Embedding

public struct Embedding<Element, Device> : LayerType, Codable where Element : RandomizableType, Device : DeviceType

Transforms discrete values, such as word indices, into a lower dimensional embedding.

  • Declaration

    Swift

    public var parameterPaths: [WritableKeyPath<`Self`, Tensor<Element, Device>>] { get }

  • Declaration

    Swift

    public var parameters: [Tensor<Element, Device>] { get }

  • Matrix of embedding vectors, shape [inputFeatures, outputSize]

    Declaration

    Swift

    public var embeddingMatrix: Tensor<Element, Device>

  • Number of input features

    Declaration

    Swift

    public var inputFeatures: Int { get }

  • Size of embedded feature vectors

    Declaration

    Swift

    public var outputSize: Int { get }

  • Index for padding that is ignored in inputs to the layer

    Declaration

    Swift

    public let ignoreIndex: Int

  • Creates an embedding layer that has an input vocabulary of size inputFeatures and returns embeddings with the size outputSize.

    The layer expects categorial inputs with a shape of [batch size] and returns embeddings with a shape of [batch size, outputSize]

    Declaration

    Swift

    public init(inputFeatures: Int, outputSize: Int, ignoreIndex: Int = -1)

    Parameters

    inputFeatures

    Vocabulary size.
    outputSize

    Embedding dimensionality.
    ignoreIndex

    Token index that is ignored when retreiving values from the embedding matrix

  • Loads pretrained word embeddings from the space / tab separated values file at the given path and arranges them according to the order of words provided.

    The embeddings are expected to be arranged using the following format:

    word1 num num num ... num
word2 num num num ... num
...

    If a word is not found in the pretrained embeddings, it is randomly created using Xavier initialization

    Declaration

    Swift

    public init?(words: [String], embeddingsURL: URL, verbose: Bool = false, ignoreIndex: Int = -1)

    Parameters

    words

    Provided word order.
    embeddingsURL

    Path to pretrained embeddings
    verbose

    If set to true, print out loading progress
    ignoreIndex

    Token index that is ignored when retreiving values from the embedding matrix

  • Declaration

    Swift

    public func callAsFunction(_ inputs: Tensor<Int32, Device>) -> Tensor<Element, Device>