Tensor

public struct Tensor<Element, Device> where Element : NumericType, Device : DeviceType
extension Tensor: CustomStringConvertible, CustomDebugStringConvertible
extension Tensor: ExpressibleByFloatLiteral
extension Tensor: ExpressibleByIntegerLiteral
extension Tensor: Equatable where Element: Equatable
extension Tensor: Codable where Element: Codable

A tensor is an n-dimensional array of numbers with a given shape.

  • Shape of the tensor.

    A tensor with an empty shape is a scalar. When shape.count == 1, the tensor is a vector. When shape.count == 2, the tensor is a matrix, etc.

    Declaration

    Swift

    public let shape: [Int]

  • Whether the compute graph of operations originating from this tensor should be captured. If the compute graph is captured, the resources associated with this tensor are only released after all tensors that have been derived from this tensor are released.

    All tensors derived from gradient requiring tensors will also require a gradient.

    To compute a gradient, use the gradients(of:) function. Example:

    let a = Tensor<Float, CPU>([1,2,3,4,5], requiresGradient: true)
let result = a * a * a // [1, 8, 27, 64, 125]

let grads = result.gradients(of: [a])
let ∇a = grads[0] // [3, 12, 27, 48, 75]

    To detach a tensor from the compute graph, use tensor.detached().

    Declaration

    Swift

    public var requiresGradient: Bool
  • tag

    Debug tag for the tensor. If you use tensor.graph() to visualize the compute graph, the tensor is labelled with the appropriate tag.

    Declaration

    Swift

    public var tag: String?

  • Number of elements in the tensor.

    Declaration

    Swift

    public var count: Int { get }
  • dim

    Dimensionality of the tensor. (0: scalar, 1: vector, 2: matrix, …)

    Declaration

    Swift

    public var dim: Int { get }

  • Creates a tensor with the given shape and fills it with value

    Declaration

    Swift

    public init(repeating value: Element, shape: Int..., requiresGradient: Bool = false)

    Parameters

    value

    Value to fill tensor with
    shape

    Shape of the tensor
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with value

    Declaration

    Swift

    public init(repeating value: Element, shape: [Int], requiresGradient: Bool = false)

    Parameters

    value

    Value to fill tensor with
    shape

    Shape of the tensor
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    public init(_ v: [Element], requiresGradient: Bool = false)

    Parameters

    v

    Value to fill tensor with
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    public init(_ v: [Element], shape: [Int], requiresGradient: Bool = false)

    Parameters

    v

    Value to fill tensor with
    shape

    Shape of the tensor. The number of elements in v must be compatible with the shape.
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    public init(_ v: [Element], shape: Int..., requiresGradient: Bool = false)

    Parameters

    v

    Value to fill tensor with
    shape

    Shape of the tensor. The number of elements in v must be compatible with the shape.
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Performs backpropagation and returns the gradients for the given tensors.

    Tensors for which it is desired to compute gradients must have requiresGradient set to true. If the result is not differentiable with respect to an input tensor, a tensor of zeros will be returned.

    Example:

    let a = Tensor<Float, CPU>([1,2,3,4,5], requiresGradient: true)
let result = a * a * a // [1, 8, 27, 64, 125]

let grads = result.gradients(of: [a])
let ∇a = grads[0] // [3, 12, 27, 48, 75]

    To detach a tensor from the compute graph, use tensor.detached().

    If it is desired to compute second, third, etc. derivatives, the retainBackwardsGraph flag must be set to true. This will record the compute graph for the backpropagation operation. A second derivative can then be computed as the gradient of a gradient. If the flag is not set, the compute graph of the backwards operation will not be captured and the result is not differentiable to any variable.

    Declaration

    Swift

    public func gradients(of tensors: [`Self`], retainBackwardsGraph retainGraph: Bool = false) -> [`Self`]

    Parameters

    tensors

    Tensors to differentiate for
    retainGraph

    Whether to store the graph for the backwards pass. If enabled, higher order gradients can be computed.

  • In-place detaches the tensor from the compute graph.

    Declaration

    Swift

    public mutating func discardContext()

  • Detaches the tensor from the compute graph. No gradients can be computed for the resulting tensor.

    Declaration

    Swift

    public func detached() -> Tensor<Element, Device>

Compute Graph Debugging

  • Prints the compute graph, from which the tensor has been derived.

    The graph is in graphviz format and can be rendered with command line tools such as dot.

    Note: When running release builds, some information about the compute graph is discarded. To obtain a detailed compute graph, compile in debug mode.

    Declaration

    Swift

    func graph() -> String

Broadcasting Operators

  • Element-wise broadcast adds the given tensors

    lhs and rhs must have matching shapes, such that dimensions of the shape are either equal or 1. Shapes are matched from the right. For example, the shapes [42, 3, 1] and [3, 8] can be broadcasted and will give a tensor with the result shape [42, 3, 8].

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func + (lhs: `Self`, rhs: `Self`) -> Tensor<Element, Device>

    Parameters

    lhs

    First tensor
    rhs

    Second tensor

    Return Value

    Broadcast added result

  • Element-wise broadcast multiplies the given tensors

    lhs and rhs must have matching shapes, such that dimensions of the shape are either equal or 1. Shapes are matched from the right. For example, the shapes [42, 3, 1] and [3, 8] can be broadcasted and will give a tensor with the result shape [42, 3, 8].

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func * (lhs: `Self`, rhs: `Self`) -> Tensor<Element, Device>

    Parameters

    lhs

    First tensor
    rhs

    Second tensor

    Return Value

    Broadcast multiplied result

  • Element-wise broadcast subtracts the given tensors

    lhs and rhs must have matching shapes, such that dimensions of the shape are either equal or 1. Shapes are matched from the right. For example, the shapes [42, 3, 1] and [3, 8] can be broadcasted and will give a tensor with the result shape [42, 3, 8].

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func - (lhs: `Self`, rhs: `Self`) -> Tensor<Element, Device>

    Parameters

    lhs

    First tensor
    rhs

    Second tensor

    Return Value

    Broadcast difference

  • Element-wise broadcast divides the given tensors

    lhs and rhs must have matching shapes, such that dimensions of the shape are either equal or 1. Shapes are matched from the right. For example, the shapes [42, 3, 1] and [3, 8] can be broadcasted and will give a tensor with the result shape [42, 3, 8].

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func / (lhs: `Self`, rhs: `Self`) -> Tensor<Element, Device>

    Parameters

    lhs

    First tensor
    rhs

    Second tensor

    Return Value

    Broadcast quotient

  • Negates every element of the given tensor.

    Declaration

    Swift

    prefix static func - (value: `Self`) -> Tensor<Element, Device>

    Parameters

    value

    Tensor to negate

    Return Value

    Negated tensor

  • In-place broadcast adds the given tensors. This operation requires the resulting broadcast shape to be equivalent to the shape of lhs.

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func += (lhs: inout `Self`, rhs: `Self`)

    Parameters

    lhs

    Tensor to update
    rhs

    Tensor to add to lhs

  • In-place broadcast subtracts the given tensors. This operation requires the resulting broadcast shape to be equivalent to the shape of lhs.

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func -= (lhs: inout `Self`, rhs: `Self`)

    Parameters

    lhs

    Tensor to update
    rhs

    Tensor to subtract from lhs

  • In-place broadcast multiplies the given tensors. This operation requires the resulting broadcast shape to be equivalent to the shape of lhs.

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func *= (lhs: inout `Self`, rhs: `Self`)

    Parameters

    lhs

    Tensor to update
    rhs

    Tensor to multiply with lhs

  • In-place broadcast divides the given tensors. This operation requires the resulting broadcast shape to be equivalent to the shape of lhs.

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    static func /= (lhs: inout `Self`, rhs: `Self`)

    Parameters

    lhs

    Tensor to update
    rhs

    Tensor to divide lhs with

  • Performs a broadcasted exponentiation between self (base) and power (exponent).

    For detailed broadcasting rules, follow the numpy documentation

    Declaration

    Swift

    func raised(toPowerOf power: `Self`) -> Tensor<Element, Device>

    Parameters

    power

    Exponent

    Return Value

    self broadcast exponentiated by power

  • Computes the elementwise maxima between the given tensors

    Declaration

    Swift

    static func max(_ first: `Self`, _ second: `Self`) -> Tensor<Element, Device>

    Parameters

    first

    First tensor
    second

    Second tensor

    Return Value

    Element wise maxima between first and second value tensors

  • Computes the elementwise minima between the given tensors

    Declaration

    Swift

    static func min(_ first: `Self`, _ second: `Self`) -> Tensor<Element, Device>

    Parameters

    first

    First tensor
    second

    Other tensors

    Return Value

    Element wise minima between first and second value tensors

Img2col

  • Performs an img2col transformation, which allows convolutions to be performed by matrix multiplication.

    The source tensor is expected to have a shape of [batchSize, channels, height, width]. The result is a tensor with shape [window_size, window_count].

    The window size is the size of the kernel (width * height * depth). The window count is the number of windows that fit into the source tensor when using the given padding and stride.

    Windows are layed out from left to right and from top to bottom, where (0, 0) is the top left corner of the image.

    Declaration

    Swift

    func img2col(kernelWidth: Int, kernelHeight: Int, padding: Int, stride: Int) -> Tensor<Element, Device>

    Parameters

    kernelWidth

    Width of the convolution kernel
    kernelHeight

    Height of the convolution kernel
    padding

    Padding applied before and after the image in the horizontal and vertical direction
    stride

    Stride, with which the kernel is moved along the image

  • Computes the inverse of the img2col operation.

    The source tensor is expected to be a tensor with shape [window_size, window_count]. The result tensor will have the given result shape, which is expected to be 4-dimensional ([batch_size, channels, height, width])

    Declaration

    Swift

    func col2img(kernelWidth: Int, kernelHeight: Int, padding: Int, stride: Int, resultShape: [Int]) -> Tensor<Element, Device>

    Parameters

    kernelWidth

    Width of the convolution kernel
    kernelHeight

    Height of the convolution kernel
    padding

    Padding applied before and after the image in the horizontal and vertical direction
    stride

    Stride, with which the kernel is moved along the image
    resultShape

    Shape of the resulting tensor

Convolution

  • Performs a 2d convolution

    the source tensor is expected to have a shape of [batchSize, channels, width, height] the filters tensor is expected to have a shape of [outputChannels, inputChannels, kernelWidth, kernelHeight]

    Declaration

    Swift

    func convolved2d(filters: Tensor<Element, Device>, padding: Int? = nil, stride: Int = 1) -> Tensor<Element, Device>

    Parameters

    filters

    Filters to convolve the tensor with
    padding

    Padding applied before and after the image in the horizontal and vertical direction
    stride

    Stride, with which the kernel is moved along the image

    Return Value

    A tensor of shape [batchSize, outputChannels, (height + 2 * padding - kernelHeight) / stride + 1, (width + 2 * padding - kernelWidth) / stride + 1)

  • Performs a transposed 2d convolution (also called fractionally strided convolution).

    The source tensor is expected to have a shape of [batchSize, channels, width, height] the filters tensor is expected to have a shape of [outputChannels, inputChannels, kernelWidth, kernelHeight]

    Declaration

    Swift

    func transposedConvolved2d(filters: Tensor<Element, Device>, inset: Int? = nil, stride: Int = 1) -> Tensor<Element, Device>

    Parameters

    filters

    Filters to convolve the tensor with
    inset

    Inset from edge of the source tensor
    stride

    Stride, with which the kernel moves over the result image. Larger strides result in larger output shapes.

    Return Value

    A tensor of shape [batchSize, outputChannels, (height - 1) * stride - 2 * padding + kernelHeight, (width - 1) * stride - 2 * padding + kernelWidth]

Pooling

  • Performs max pooling on the tensor. Max pooling selects the maximum value for every given window of a tensor.

    The source tensor is expected to have a shape of [batchSize, channels, width, height].

    Declaration

    Swift

    func maxPooled2d(windowSize: Int, padding: Int? = nil, stride: Int? = nil) -> Tensor<Element, Device>

    Parameters

    windowSize

    Window size
    padding

    Padding applied before and after the image in the horizontal and vertical direction
    stride

    Stride, with which the kernel is moved along the image

    Return Value

    A tensor of shape [batchSize, channels, (height + 2 * padding - windowSize) / stride + 1, (width + 2 * padding - windowSize) / stride + 1]

  • Performs average pooling on the tensor. Average pooling computes the average of every given window of a tensor.

    The source tensor is expected to have a shape of [batchSize, channels, width, height].

    Declaration

    Swift

    func averagePooled2d(windowSize: Int, padding: Int? = nil, stride: Int? = nil) -> Tensor<Element, Device>

    Parameters

    windowSize

    Window size
    padding

    Padding applied before and after the image in the horizontal and vertical direction
    stride

    Stride, with which the kernel is moved along the image

    Return Value

    A tensor of shape [batchSize, channels, (height + 2 * padding - windowSize) / stride + 1, (width + 2 * padding - windowSize) / stride + 1]

Matrix Multiplication

  • Undocumented

    Declaration

    Swift

    func matrixMultiplied(with other: `Self`, transposeSelf: Bool = false, transposeOther: Bool = false) -> Tensor<Element, Device>

  • Broadcast matrix multiplies self with the given other operand.

    Broadcasting is applied along all axes except the last two. Operands are expected to have a dimensionality of 2 or higher.

    Declaration

    Swift

    func broadcastMatrixMultiplied(with other: `Self`, transposeSelf: Bool = false, transposeOther: Bool = false) -> Tensor<Element, Device>

    Parameters

    other

    Other operand
    transposeSelf

    Whether to transpose self before multiplication
    transposeOther

    Whether to transpose the other operand before the multiplication

Summation

  • Sums up elements along the given axes.

    Declaration

    Swift

    func reduceSum(along axes: [Int]) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to sum

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Sums up elements along the given axes

    Declaration

    Swift

    @inline(__always)
func reduceSum(along axes: Int...) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to sum

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the sum of all elements of the tensor

    Declaration

    Swift

    func reduceSum() -> Tensor<Element, Device>

    Return Value

    Scalar, sum of all elements

  • Computes the mean of the elements along the given axes

    Declaration

    Swift

    func reduceMean(along axes: [Int]) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to compute the mean of

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the mean of the elements along the given axes

    Declaration

    Swift

    func reduceMean(along axes: Int...) -> Tensor<Element, Device>

    Parameters

    axes

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the mean of all elements of the tensor

    Declaration

    Swift

    func reduceMean() -> Tensor<Element, Device>

    Return Value

    Scalar, mean of all elements

  • Computes the variance of the tensor along the given axes.

    Declaration

    Swift

    func variance(along axes: [Int]) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to compute the variance along.

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the variance of the tensor along the given axes.

    Declaration

    Swift

    func variance(along axes: Int...) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to compute the variance along.

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the variance of all elements in the tensor.

    Declaration

    Swift

    func variance() -> Tensor<Element, Device>

    Return Value

    Scalar, variance of all elements

  • Returns the index of the largest element in the tensor.

    Declaration

    Swift

    func argmax() -> Int

Min/Max

  • Computes the maximum values along the given axes of the tensor.

    Declaration

    Swift

    func reduceMax(along axes: [Int]) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to reduce along

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the maximum values along the given axes of the tensor.

    Declaration

    Swift

    func reduceMax(along axes: Int...) -> Tensor<Element, Device>

    Parameters

    axes

    Axes to reduce along

    Return Value

    Tensor with shape equal to self.shape without the given reduction axes.

  • Computes the maximum of all values in the tensor

    Declaration

    Swift

    func reduceMax() -> Tensor<Element, Device>

    Return Value

    Scalar, maximum of all elements.

Indexing

  • Gathers elements at indices determined by the context along the specified axis.

     Example: Gathering from Tensor [[1,2,3], [4,5,6], [7,8,9]]
 Context: [0, 1, 2], axis: 0
 => [1,5,9]

 Context: [2, 2, 1], axis: 1
 => [3, 6, 8]

    Declaration

    Swift

    func gather(using context: Tensor<Int32, Device>, alongAxis axis: Int, ignoreIndex: Int32 = -1) -> Tensor<Element, Device>

    Parameters

    context

    Indices along gathering axis.
    axis

    Axis to gather from

  • Scatters elements to indices determined by the context along the specified axis.

     Example: Scattering Tensor [3, 1, 4]
 Context: [0, 1, 2], axis: 0, axisSize: 3
 => [[3, 0, 0], [0, 1, 0], [0, 0, 4]]

 Context: [2, 2, 1], axis: 1, axisSize: 3
 => [[0, 0, 3], [0, 0, 1], [0, 4, 0]]

    Declaration

    Swift

    func scatter(using context: Tensor<Int32, Device>, alongAxis axis: Int, withSize axisSize: Int, ignoreIndex: Int32 = -1) -> Tensor<Element, Device>

    Parameters

    context

    Indices along scattering axis
    axis

    Axis to scatter along
    axisSize

    Number of elements along the axis in the result tensor. Must be greater than max(context)

Tensor Shape Modifiers

  • Reshapes the tensor to the given shape.

    The shape must be compatible with the source shape, i.e. the number of elements must be the same.

    The shape may contain a -1. The size of the result tensor along that axis is then computed as needed.

    Declaration

    Swift

    func view(as shape: [Int]) -> Tensor<Element, Device>

    Parameters

    shape

    Shape to view the tensor in.

    Return Value

    Tensor with given shape, where occurrences of -1 have been replaced.

  • Reshapes the tensor to the given shape.

    The shape must be compatible with the source shape, i.e. the number of elements must be the same.

    The shape may contain a -1. The size of the result tensor along that axis is then computed as needed.

    Declaration

    Swift

    func view(as shape: Int...) -> Tensor<Element, Device>

    Parameters

    shape

    Shape to view the tensor in.

    Return Value

    Tensor with given shape, where occurrences of -1 have been replaced.

  • Adds an axis to the shape of the tensor. The axis will have a size of 1.

    Declaration

    Swift

    func unsqueezed(at axis: Int) -> Tensor<Element, Device>

    Parameters

    axis

    Axis to expand at.

  • Removes an axis from the tensor if the axis has a size of 1. Otherwise, the original tensor is returned.

    Declaration

    Swift

    func squeezed(at axis: Int) -> Tensor<Element, Device>

    Parameters

    axis

    Axis to remove if possible.

  • Removes all axes from the tensor that have a size of 1.

    Declaration

    Swift

    func squeezed() -> Tensor<Element, Device>

  • Flattens the tensor into a tensor of shape [count]

    Declaration

    Swift

    func flattened() -> Tensor<Element, Device>

  • Swaps the axes of the tensor.

    The axis arangement must have a count of tensor.dim and contain all elements in 0 ..< tensor.dim.

    With axis arangement of [1, 0], this operation is equivalent to tensor.transposed()

    Declaration

    Swift

    func permuted(to axisArangement: [Int]) -> Tensor<Element, Device>

    Parameters

    axisArangement

    Arangement of axes in the resulting tensor.

  • Permutes the other tensor along the given axes and adds it to the current tensor in place.

    The permutation must have a count of tensor.dim and contain all elements in 0 ..< tensor.dim.

    Declaration

    Swift

    mutating func addingPermuted(_ other: `Self`, permutation: [Int])

    Parameters

    other

    Tensor to add permuted to the current tensor
    permutation

    Desired arangement of axes of the summand.

  • Swaps the axes of the tensor.

    The axis arangement must have a count of tensor.dim and contain all elements in 0 ..< tensor.dim.

    With axis arangement of [1, 0], this operation is equivalent to tensor.transposed()

    Declaration

    Swift

    func permuted(to axisArangement: Int...) -> Tensor<Element, Device>

    Parameters

    axisArangement

    Arangement of axes in the resulting tensor.

  • Transposes the given tensor. The tensor must have a dimensionality of 2.

    Declaration

    Swift

    func transposed() -> Tensor<Element, Device>
  • T

    Transposes the given tensor. The tensor must have a dimensionality of 2.

    Declaration

    Swift

    var T: `Self` { get }

Tensor Stacking

  • Inverts stacking of tensors.

    This operation returns a list of tensors that have equal shapes except along the unstacking axis. The number of elements of the source tensor along the unstacking axis must be equal to the sum of lengths.

    Declaration

    Swift

    func unstacked(along axis: Int, withLengths lengths: [Int]) -> [`Self`]

    Parameters

    axis

    Axis to unstack along.
    lengths

    Number of elements along the unstacking axis of the resulting tensors

  • Stacks the given tensors into a new tensor.

    The tensors must have equal shapes except along the stacking axis.

    Declaration

    Swift

    init(stacking tensors: [`Self`], along axis: Int = 0)

    Parameters

    tensors

    Tensors to stack
    axis

    Axis to stack the tensors along.

Subscripting

  • Gets or sets a subtensor at the given index.

    When an element of the index is nil, all elements along the corresponding axis are read or written.

    Example:

    let a = Tensor<Float, CPU>([[1, 2, 3], [4, 5, 6]])
print(a[nil, 1]) // [2, 5]
print(a[1]) // [4, 5, 6]
print(a[1, nil] == a[1]) // true

    Declaration

    Swift

    subscript(index: [Int?]) -> `Self` { get set }

  • Gets or sets a subtensor at the given index.

    When an element of the index is nil, all elements along the corresponding axis are read or written.

    Example:

    let a = Tensor<Float, CPU>([[1, 2, 3], [4, 5, 6]])
print(a[nil, 1]) // [2, 5]
print(a[1]) // [4, 5, 6]
print(a[1, nil] == a[1]) // true

    Declaration

    Swift

    subscript(index: Int?...) -> `Self` { get set }

  • Gets or sets a subtensor at the given window.

    When an element of the index is nil, all elements along the corresponding axis are read or written.

    Example:

    let a = Tensor<Float, CPU>([[1, 2, 3], [4, 5, 6]])
print(a[0 ..< 2]) // [[1, 2], [4, 5]]
print(a[nil, 0 ..< 1]) // [[1, 2, 3]]

    Declaration

    Swift

    subscript(index: [Range<Int>?]) -> `Self` { get set }

  • Gets or sets a subtensor at the given window.

    When an element of the index is nil, all elements along the corresponding axis are read or written.

    Example:

    let a = Tensor<Float, CPU>([[1, 2, 3], [4, 5, 6]])
print(a[0 ..< 2]) // [[1, 2], [4, 5]]
print(a[nil, 0 ..< 1]) // [[1, 2, 3]]

    Declaration

    Swift

    subscript(index: Range<Int>?...) -> `Self` { get set }

Element-wise operations

  • Element-wise exponentiates the tensor

    Declaration

    Swift

    func exp() -> Tensor<Element, Device>

  • Computes the element-wise logarithm of the tensor.

    Declaration

    Swift

    func log() -> Tensor<Element, Device>

  • Computes the element-wise hyperbolic tangent of the tensor.

    Declaration

    Swift

    func tanh() -> Tensor<Element, Device>

  • Computes the element-wise square root of the tensor.

    Declaration

    Swift

    func sqrt() -> Tensor<Element, Device>

  • Computes the element-wise heaviside step function of the tensor.

    The heaviside step function is defined as value > 0 ? 1 : 0

    Declaration

    Swift

    func heaviside() -> Tensor<Element, Device>

  • Computes the element-wise relu function.

    The relu function is defined as max(value, 0)

    Declaration

    Swift

    func rectifiedLinear() -> Tensor<Element, Device>

  • Computes the element-wise leaky relu function.

    The leaky relu function is defined as max(value, leakage * value)

    Declaration

    Swift

    func leakyRectifiedLinear(leakage: `Self`) -> Tensor<Element, Device>

  • Computes the element-wise sigmoid function.

    Declaration

    Swift

    func sigmoid() -> Tensor<Element, Device>

  • Computes the softmax function along the given axis. If no axis is provided, the softmax is computed along axis 1.

    Declaration

    Swift

    func softmax(axis: Int = 1) -> Tensor<Element, Device>

  • Computes the logarithm of the softmax function along the given axis. If no axis is provided, the softmax is computed along axis 1.

    Declaration

    Swift

    func logSoftmax(axis: Int = 1) -> Tensor<Element, Device>

  • Computes the element-wise sine.

    Declaration

    Swift

    func sine() -> Tensor<Element, Device>

  • Computes the element-wise cosine.

    Declaration

    Swift

    func cosine() -> Tensor<Element, Device>

  • Computes the element-wise GeLU activation

    See Hendrycks, Gimpel - Gaussian Error Linear Units

    Declaration

    Swift

    func gaussianErrorLinear() -> Tensor<Element, Device>

  • Computes the element-wise Swish activation

    See Ramachandran et al. - Searching for Activation Functions

    Declaration

    Swift

    func swishActivated(beta: `Self` = 1) -> Tensor<Element, Device>

  • Computes the element-wise Mish activation

    See Diganta Misra - Mish: A Self Regularized Non-Monotonic Neural Activation Function

    Declaration

    Swift

    func mishActivated() -> Tensor<Element, Device>

  • Computes the element-wise LiSHT activation

    See Roy et al. - LiSHT: Non-Parametric Linearly Scaled Hyperbolic Tangent Activation Function for Neural Networks

    Declaration

    Swift

    func lishtActivated() -> Tensor<Element, Device>

  • Element-wise exponential linear unit activation

    See [Clevert et al. - Fast And Accurate Deep Network Learning By Exponential Linear Units (ELUs)](https://arxiv.org/pdf/1511.07289.pdf

    Declaration

    Swift

    func exponentialLinearActivated(alpha: `Self` = 1) -> Tensor<Element, Device>

    Parameters

    alpha

    Scale applied to exponential part

Utility operations

  • Linearly interpolates between the lower and upper bound (both including).

    Declaration

    Swift

    init(linearRampWithLowerBound lowerBound: Element = 0, upperBound: Element, by stride: Element = 1)

    Parameters

    lowerBound

    Start
    upperBound

    End
    stride

    Increment between elements

  • Repeats the tensor times times and stacks the result along the 0th axis.

    Declaration

    Swift

    func repeated(_ times: Int) -> Tensor<Element, Device>

    Parameters

    times

    Number of repetitions

  • Pads the tensor with the given leading and trailing padding for each axis.

    Declaration

    Swift

    func padded(with value: Element = 0, padding: [(Int, Int)]) -> Tensor<Element, Device>

    Parameters

    value

    Padding value
    padding

    Number of padded elements before and after the tensor.

  • Pads the tensor with the given leading and trailing padding for each axis.

    Declaration

    Swift

    func padded(with value: Element = 0, padding: [Int]) -> Tensor<Element, Device>

    Parameters

    value

    Padding value
    padding

    Number of padded elements before and after the tensor.

  • Reverses the tensor along the 0th axis.

    Declaration

    Swift

    func reversed() -> Tensor<Element, Device>

  • Computes a diagonal matrix with the given number of elements below and above the diagonal. Remaining elements are filled with zeros.

    Declaration

    Swift

    func bandMatrix(belowDiagonal: Int?, aboveDiagonal: Int?) -> Tensor<Element, Device>

    Parameters

    belowDiagonal

    Number of elements below diagonal or nil, if all elements should be copied.
    aboveDiagonal

    Number of elements above the diagonal or nil, if all elements should be copied.

  • Computes the vector of diagonal elements of a matrix

    Source tensor must have dimensionality of 2. For backpropagation, the matrix must have square shape.

    Declaration

    Swift

    func diagonalElements() -> Tensor<Element, Device>

    Return Value

    Vector containing matrix diagonal elements

  • Computes a matrix that contains the elements of a vector in its diagonal. The remaining elements will be filled with zeros.

    The source tensor must have a dimensionality of two. The resulting matrix will have a number of rows and columns equal to number of elements in the vector

    Declaration

    Swift

    func diagonalMatrix() -> Tensor<Element, Device>

    Return Value

    Square diagonal matrix

  • Creates a matrix filled with the given value on its diagonal and zeros everywhere else

    Declaration

    Swift

    init(fillingDiagonalWith value: Element, size: Int, requiresGradient: Bool = false)

    Parameters

    value

    Value to fill diagonal with
    size

    Number of rows and columns of the resulting matrix
    requiresGradient

    Whether to include the tensor in the compute graph for gradient computation

Tensor extensions

  • Declaration

    Swift

    public var description: String { get }

  • Declaration

    Swift

    public var debugDescription: String { get }

  • Declaration

    Swift

    public init(floatLiteral value: Double)

  • Declaration

    Swift

    public init(integerLiteral value: Int)

  • Creates a scalar tensor with the given value. The tensor will have a shape of []

    Declaration

    Swift

    init(_ value: Element)

    Parameters

    value

    Value of the tensor.

  • Element at the first index in the tensor.

    Declaration

    Swift

    var item: Element { get }

Tensor - array conversion

  • Creates a tensor value holding the provided scalar. The tensor will have an empty shape.

    Declaration

    Swift

    init(_ e: Element, requiresGradient: Bool = false)

    Parameters

    e

    Element
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    init(_ v: [[Element]], requiresGradient: Bool = false)

    Parameters

    v

    Values to fill tensor with
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    init(_ v: [[[Element]]], requiresGradient: Bool = false)

    Parameters

    v

    Values to fill tensor with
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    init(_ v: [[[[Element]]]], requiresGradient: Bool = false)

    Parameters

    v

    Values to fill tensor with
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor with the given shape and fills it with the given array of elements

    Declaration

    Swift

    init(_ v: [[[[[Element]]]]], requiresGradient: Bool = false)

    Parameters

    v

    Values to fill tensor with
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

Tensor initialization

Tensor - Image conversion

  • Creates a tensor from the given CGImage

    Declaration

    Swift

    init?(_ image: CGImage, normalizedTo range: ClosedRange<Element> = 0 ... 1)

    Parameters

    image

    Image
    range

    Range to normalize pixel values to

  • Undocumented

    Declaration

    Swift

    func cgImage(normalizeFrom tensorRange: ClosedRange<Element> = 0 ... 1) -> CGImage?

  • Creates a tensor from the given NSImage

    Declaration

    Swift

    init?(_ image: NSImage, normalizedTo range: ClosedRange<Element> = 0 ... 1)

    Parameters

    image

    Image
    range

    Range to normalize pixel values to

Available where Element == Int32

  • One-hot encodes a tensor of indices

    Declaration

    Swift

    func oneHotEncoded<Target>(dim: Int, type: Target.Type = Target.self) -> Tensor<Target, Device> where Target : NumericType

    Parameters

    dim

    Size of encoding axis. max(tensor) must be less than dim.
    type

    Data type of the result.

Available where Element: Equatable

  • Declaration

    Swift

    public static func == (lhs: `Self`, rhs: `Self`) -> Bool

Available where Element: RandomizableType

  • Creates a tensor and fills it with random values sampled from a normal distribution with mean 0 and standard deviation sqrt(2 / shape[0]).

    Declaration

    Swift

    init(xavierNormalWithShape shape: [Int], requiresGradient: Bool = false)

    Parameters

    shape

    Shape of the tensor, must be two dimensional
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor and fills it with random values sampled from a normal distribution with mean 0 and standard deviation sqrt(2 / shape[0]).

    Declaration

    Swift

    init(xavierNormalWithShape shape: Int..., requiresGradient: Bool = false)

    Parameters

    shape

    Shape of the tensor, must be two dimensional
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor and fills it with random values sampled from a normal distribution with the given mean and variance.

    Declaration

    Swift

    init(normalDistributedWithShape shape: [Int], mean: Element = 0, stdev: Element = 1, requiresGradient: Bool = false)

    Parameters

    shape

    Shape of the tensor, must be two dimensional
    mean

    Mean of the normal distribution.
    stdev

    Standard deviation of the normal distribution
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor and fills it with random values sampled from a normal distribution with the given mean and variance.

    Declaration

    Swift

    init(normalDistributedWithShape shape: Int..., mean: Element = 0, stdev: Element = 1, requiresGradient: Bool = false)

    Parameters

    shape

    Shape of the tensor, must be two dimensional
    mean

    Mean of the normal distribution.
    stdev

    Standard deviation of the normal distribution
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor and fills it with random values sampled from a uniform distribution with the given minimum and maximum.

    Declaration

    Swift

    init(uniformlyDistributedWithShape shape: [Int], min: Element = 0, max: Element = 1, requiresGradient: Bool = false)

    Parameters

    shape

    Shape of the tensor, must be two dimensional
    min

    Minimum value of the uniform distribution
    max

    Maximum value of the uniform distribution
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

  • Creates a tensor and fills it with random values sampled from a uniform distribution with the given minimum and maximum.

    Declaration

    Swift

    init(uniformlyDistributedWithShape shape: Int..., min: Element = 0, max: Element = 1, requiresGradient: Bool = false)

    Parameters

    shape

    Shape of the tensor, must be two dimensional
    min

    Minimum value of the uniform distribution
    max

    Maximum value of the uniform distribution
    requiresGradient

    Whether it is desired to compute gradients of the tensor.

Available where Element: Codable

  • Declaration

    Swift

    public init(from decoder: Decoder) throws

  • Declaration

    Swift

    public func encode(to encoder: Encoder) throws