LayerNorm

public struct LayerNorm<Element, Device> : LayerType, Codable where Element : RandomizableType, Device : DeviceType

A layer that normalizes its inputs along all dimensions except the batch dimension

  • Declaration

    Swift

    public var parameterPaths: [WritableKeyPath<LayerNorm<Element, Device>, Tensor<Element, Device>>] { get }

  • Declaration

    Swift

    public var parameters: [Tensor<Element, Device>] { get }

  • Whether the layer is training, currently ignored.

    Declaration

    Swift

    public var isTraining: Bool

  • Learned shift vector

    Declaration

    Swift

    public var shift: Tensor<Element, Device>

  • Learned scale vector

    Declaration

    Swift

    public var scale: Tensor<Element, Device>

  • Momentum with which to update mean and variance. Currently ignored

    Declaration

    Swift

    public var momentum: Element

  • A layer that normalizes its inputs along all dimensions except the batch dimension

    Declaration

    Swift

    public init(inputSize: [Int], momentum: Element = 0.9)

  • Declaration

    Swift

    public func callAsFunction(_ inputs: Tensor<Element, Device>) -> Tensor<Element, Device>