Automatic differentiation using dual numbers

[Albert Chan]

(06-21-2022 02:19 AM)Thomas Klemm Wrote:  Au contraire!
It's just that you misinterpreted the meaning of the coefficient of \(\varepsilon^2\).
Based on the Taylor series we have:

\(
\begin{align}
f(x + \varepsilon) = f(x) + {f}'(x) \cdot \varepsilon + \frac{1}{2} {f}''(x) \cdot \varepsilon^2
\end{align}
\)

Thank you for clearing this up!
To confirm my understanding, I redo expm1 and log1p rules, using epsilon.

Quote:-- expm1(f)' = exp(f) * f'
-- expm1(f)'' = exp(f) * f'' + exp(f)*f' * f'

Let ε2 = ε^2/2!, and assume (ε3, ε4 , ...) ≈ 0

expm1(x) = x + x^2/2! + x^3/3! + ...

expm1(f(x+ε))
≈ expm1(f + f'*ε + f''*ε2)
= exp(f) * exp(f'*ε) * exp(f''*ε2) - 1
≈ exp(f) * (1 + f'*ε + f'^2*ε2) * (1 + f''*ε2) - 1
≈ exp(f) * (1 + f'*ε + f'^2*ε2 + f''*ε2) - 1
= expm1(f) + (exp(f)*f')*ε + (exp(f)*(f'' + f'^2))*ε2       ✔️

Quote:-- log(1+f)' = f' / (1+f)
-- log(1+f)'' = ((1+f)*f'' - f'*f') / (1+f)^2

log1p(x) = x - x^2/2 + x^3/3 - ...

log1p(f(x+ε))
≈ log(1 + f + f'*ε + f''*ε2)
= log1p(f) + log1p(ε*(f' + f''/2*ε)/(1+f))
≈ log1p(f) + ε*(f' + f''/2*ε)/(1+f) - ε2*(f'/(1+f))^2
= log1p(f) + (f'/(1+f))*ε + (((1+f)*f'' - f'^2)/(1+f)^2)*ε2       ✔️