UnaryOpElementwise Type
Defines a new op implementing an elementwise unary function and its derivatives. Instances of this class are used with the Tensor.Op method to define a new differentiable tensor function that supports forward, reverse, and nested differentiation.
This type is specialized to elementwise ops. It requires the user to specify only (1) the RawTensor operation and (2) the derivative of the function with respect to its argument. The corresponding derivative propagation rules for the forward and reverse differentiation modes are automatically generated.
If you are implementing a complex op that is not elementwise, you can use the generic type UnaryOp, which allows you to define the full derivative propagation rules.
Example
{ new UnaryOpElementwise("cos") with
member _.fRaw(a) = a.CosT()
member _.dfda(a,f) = -a.sin()
}
{ new UnaryOpElementwise("exp") with
member _.fRaw(a) = a.ExpT()
member _.dfda(a,f) = f
}
{ new UnaryOpElementwise("log") with
member _.fRaw(a) = a.LogT()
member _.dfda(a,f) = 1/a
}
Constructors
| Constructor |
Description
|
|
|
Instance members
| Instance member |
Description
|
Full Usage:
this.dfda (a, f)
Parameters:
Tensor
-
The argument \( a \)
f : Tensor
-
The function's pre-computed primal evaluation result \( f(a) \), which can be one of the terms involved in the derivative computation (e.g., the derivative of the exponential function) and be used without the need to recompute it.
Returns: Tensor
The tensor corresponding to \( \frac{\partial f(a)}{\partial a} \).
Modifiers: abstract |
Derivative of the function with respect to its argument, \( \frac{\partial f(a)}{\partial a} \).
|
Inherited members
Inherited from
UnaryOp
Instance members
| Instance member |
Description
|
Full Usage:
this.ad_dfda (a, ad, f)
Parameters:
Tensor
-
The argument \( a \).
ad : Tensor
-
The argument's derivative \( \frac{\partial a}{\partial x} \).
f : Tensor
-
The function's pre-computed primal evaluation result \( f(a) \), which can be one of the terms involved in the derivative computation (e.g., the derivative of the exponential function) and be used without the need to recompute it.
Returns: Tensor
The tensor corresponding to \( \frac{\partial f(a)}{\partial x} = \frac{\partial a}{\partial x} \frac{\partial f(a)}{\partial a} \).
Modifiers: abstract |
Derivative propagation rule for forward differentiation mode. This represents the derivative of \( f(a) \) with respect a value \( x \) earlier in the computation graph than the function's argument \( a \). In other words, it computes \( \frac{\partial f(a)}{\partial x} = \frac{\partial a}{\partial x} \frac{\partial f(a)}{\partial a} \).
|
|
|
Full Usage:
this.fd_dfda (a, f, fd)
Parameters:
Tensor
-
The argument \( a \).
f : Tensor
-
The function's pre-computed primal evaluation result \( f(a) \), which can be one of the terms involved in the derivative computation (e.g., the derivative of the exponential function) and be used without the need to recompute it.
fd : Tensor
-
The derivative with respect to the function's output \( \frac{\partial y}{\partial f(a)} \).
Returns: Tensor
The tensor corresponding to \( \frac{\partial y}{\partial a} = \frac{\partial y}{\partial f(a)} \frac{\partial f(a)}{\partial a} \).
Modifiers: abstract |
Derivative propagation rule for reverse differentiation mode. This represents the derivative of a value \( y \), which comes later in the computation graph than the function's value \( f(a) \), with respect to the function's argument \( a \). In other words, it computes \( \frac{\partial y}{\partial a} = \frac{\partial y}{\partial f(a)} \frac{\partial f(a)}{\partial a} \).
|
Full Usage:
this.name
Returns: string
|
Name of the op.
|
© Copyright 2021, DiffSharp and dfx Contributors.