Automatic differentiation using dual numbers

[Thomas Klemm]

We have seen that in some cases fixed-point iteration doesn't converge to the solution of \(x = f(x)\).
Also the convergence can be slow.

Examples

Dottie Number

\(
\begin{align}
\cos(x) - x = 0
\end{align}
\)

Code:

11▸LBL "f(x)"
12 RCL "x"
13 XEQ "COS"
14 RCL- "x"
15 END

1 XEQ "NEWTON" R/S R/S R/S …

0.7503638678402438930349423066821769
0.7391128909113616703605852909048903
0.7390851333852839697601251208568043
0.7390851332151606416617026256850264
0.7390851332151606416553120876738734
0.7390851332151606416553120876738734

Compared to this the fixed-point iteration is rather slow.

…and you hit cosine, and you hit cosine, and you hit cosine.

1 COS COS COS …

0.5403023058681397174009366074429766
0.8575532158463934157441062727619899
0.6542897904977791499709664713278083
0.7934803587425655918260542309902841
0.7013687736227565244719705270529776
0.7394118536802670853384982393894295
0.7388650109363127073601308869127267
…

---

(10-04-2020 04:21 PM)Eddie W. Shore Wrote:  Be aware, some equations cannot be solved in this manner, such as x = π / sin x and x = ln(1 / x^4).

\(
\begin{align}
4 \ln(x) + x = 0
\end{align}
\)

Code:

11▸LBL "f(x)"
12 RCL "x"
13 XEQ "LN"
14 4
15 ×
16 RCL+ "x"
17 END

1 XEQ "NEWTON" R/S R/S R/S …

0.8
0.8154290342094731705108633935398897
0.8155534109294744392403815755194143
0.815553418808960626155252543632822
0.815553418808960657772727325308559
0.8155534188089606577727273253085595

---

\(
\begin{align}
\sin(x) x - \pi = 0
\end{align}
\)

Code:

11▸LBL "f(x)"
12 RCL "x"
13 XEQ "SIN"
14 RCL "x"
15 XEQ "*"
16 PI
17 -
18 END

6 XEQ "NEWTON" R/S R/S R/S …

6.878955081396972443178487244060421
6.76408551570638856537720551093318
6.76604838451268027345470086566581
6.766048790452746205491186938760874
6.766048790452763690963998120299702
6.766048790452763690963998120332144
6.766048790452763690963998120332144


Conclusion

Newton's method allows to transform an equation into a fixed-point iteration that usually converges much faster.
Using automatic differentiation allows us to do this with only minor additional effort.