(11C) Quadratic Regression

[Thomas Klemm]

Formula

Given a set of points \(\{(x_1, y_1), (x_2, y_2), \cdots, (x_n, y_n)\}\) where \(n \geqslant 3\) we want to find the best fit for a quadratic polynomial:

\(
\begin{align}
y = a \cdot x^2 + b \cdot x + c
\end{align}
\)

This leads to the following linear system of equations:

\(
\begin{bmatrix}
n & \Sigma x & \Sigma x^2 \\
\Sigma x & \Sigma x^2 & \Sigma x^3 \\
\Sigma x^2 & \Sigma x^3 & \Sigma x^4 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
c \\
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
\Sigma y \\
\Sigma xy \\
\Sigma x^2y \\
\end{bmatrix}
\)

We can perform Gauss elimination on the first row to get:

\(
\begin{bmatrix}
n & \Sigma x & \Sigma x^2 \\
0 & n \cdot \Sigma x^2 - (\Sigma x)^2 & n \cdot \Sigma x^3 - \Sigma x \cdot \Sigma x^2 \\
0 & n \cdot \Sigma x^3 - \Sigma x \cdot \Sigma x^2 & n \cdot \Sigma x^4 - (\Sigma x^2)^2 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
c \\
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
\Sigma y \\
n \cdot \Sigma xy - \Sigma x \cdot \Sigma y \\
n \cdot \Sigma x^2y - \Sigma x^2 \cdot \Sigma y \\
\end{bmatrix}
\)

We define the following:

\(
\begin{align}
S_{x^2} &= n \cdot \Sigma x^2 - (\Sigma x)^2 \\
S_{xy} &= n \cdot \Sigma xy - \Sigma x \cdot \Sigma y \\
S_{xx^2} &= n \cdot \Sigma x^3 - \Sigma x \cdot \Sigma x^2 \\
S_{x^2x^2} &= n \cdot \Sigma x^4 - (\Sigma x^2)^2 \\
S_{x^2y} &= n \cdot \Sigma x^2y - \Sigma x^2 \cdot \Sigma y \\
\end{align}
\)

This allows us to rewrite the linear system for \(b\) and \(a\) as:

\(
\begin{bmatrix}
S_{x^2} & S_{xx^2} \\
S_{xx^2} & S_{x^2x^2} \\
\end{bmatrix}
\cdot
\begin{bmatrix}
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
S_{xy} \\
S_{x^2y} \\
\end{bmatrix}
\)

Since the matrix is symmetric we can use the built-in function L.R. to solve it.

In case of linear regression we have the following linear system:

\(
\begin{bmatrix}
n & \Sigma x \\
\Sigma x & \Sigma x^2 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
\Sigma y \\
\Sigma xy \\
\end{bmatrix}
\)

Registers

We use the following registers:

\(
\begin{align}
R_{0} &: n \\
R_{1} &: \Sigma x \\
R_{2} &: \Sigma x^2 \\
R_{3} &: \Sigma y \\
R_{4} &: \Sigma y^2 \\
R_{5} &: \Sigma xy \\
R_{6} &: \Sigma x^3 \\
R_{7} &: \Sigma x^4 \\
R_{8} &: \Sigma x^2y \\
R_{9} &: c \\
R_{.0} &: b \\
R_{.1} &: a \\
I &: \text{scratch} \\
\end{align}
\)

Thus the function L.R. allows to solve the following linear system:

\(
\begin{bmatrix}
R_{0} & R_{1} \\
R_{1} & R_{2} \\
\end{bmatrix}
\cdot
\begin{bmatrix}
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
R_{3} \\
R_{5} \\
\end{bmatrix}
\)

This means we have to do the following assignments:

\(
\begin{align}
R_{0} &\leftarrow R_{0} \cdot R_{2} - R_{1}^2 \\
R_{1} &\leftarrow R_{0} \cdot R_{6} - R_{1} \cdot R_{2} \\
R_{2} &\leftarrow R_{0} \cdot R_{7} - R_{2}^2 \\
R_{3} &\leftarrow R_{0} \cdot R_{5} - R_{1} \cdot R_{3} \\
R_{5} &\leftarrow R_{0} \cdot R_{8} - R_{2} \cdot R_{3} \\
\end{align}
\)

We temporarily store the previous values in the following registers:

\(
\begin{align}
R_{9} &\leftarrow R_{0} \\
R_{.0} &\leftarrow R_{1} \\
R_{.1} &\leftarrow R_{2} \\
I &\leftarrow R_{3} \\
\end{align}
\)

Forecast

The linear estimate function ŷ allows to calculate:

\(
\begin{align}
\hat{y} &= a \cdot x + b
\end{align}
\)

This can be used to calculate:

\(
\begin{align}
\hat{y} \cdot x + c
&= (a \cdot x + b) \cdot x + c \\
&= a \cdot x^2 + b \cdot x + c \\
\end{align}
\)

Program

Code:

001 - 42,21,11  LBL A
002 -       36  ENTER
003 -       36  ENTER
004 -    43 11  x↑2
005 -       20  ×
006 - 44,40, 6  STO+ 6
007 -       33  R↓
008 -    43 36  LST×
009 -    43 33  R↑
010 -       34  x<>y
011 -       20  ×
012 - 44,40, 8  STO+ 8
013 -       33  R↓
014 -    43 36  LST×
015 -    43 11  x↑2
016 - 44,40, 7  STO+ 7
017 -       33  R↓
018 -       49  Σ+
019 -    43 32  RTN
020 - 42,21,12  LBL B
021 -       36  ENTER
022 -       36  ENTER
023 -    43 11  x↑2
024 -       20  ×
025 - 44,30, 6  STO- 6
026 -       33  R↓
027 -    43 36  LST×
028 -    43 33  R↑
029 -       34  x<>y
030 -       20  ×
031 - 44,30, 8  STO- 8
032 -       33  R↓
033 -    43 36  LST×
034 -    43 11  x↑2
035 - 44,30, 7  STO- 7
036 -       33  R↓
037 -    43 49  Σ-
038 -    43 32  RTN
039 - 42,21,13  LBL C
040 -    45  0  RCL 0
041 -    44  9  STO 9
042 -    45  2  RCL 2
043 -    44 .1  STO .1
044 -       20  ×
045 -    45  1  RCL 1
046 -    44 .0  STO .0
047 -    43 11  x↑2
048 -       30  -
049 -    44  0  STO 0
050 -    45  9  RCL 9
051 -    45  6  RCL 6
052 -       20  ×
053 -    45 .0  RCL .0
054 -    45 .1  RCL .1
055 -       20  ×
056 -       30  -
057 -    44  1  STO 1
058 -    45  9  RCL 9
059 -    45  7  RCL 7
060 -       20  ×
061 -    45 .1  RCL .1
062 -    43 11  x↑2
063 -       30  -
064 -    44  2  STO 2
065 -    45  9  RCL 9
066 -    45  5  RCL 5
067 -       20  ×
068 -    45 .0  RCL .0
069 -    45  3  RCL 3
070 -    44 25  STO I
071 -       20  ×
072 -       30  -
073 -    44  3  STO 3
074 -    45  9  RCL 9
075 -    45  8  RCL 8
076 -       20  ×
077 -    45 .1  RCL .1
078 -    45 25  RCL I
079 -       20  ×
080 -       30  -
081 -    44  5  STO 5
082 -    42 49  L.R.
083 -    45 .0  RCL .0
084 -       34  x<>y
085 -    44 .0  STO .0
086 -       20  ×
087 -       34  x<>y
088 -    45 .1  RCL .1
089 -       34  x<>y
090 -    44 .1  STO .1
091 -       20  ×
092 -       40  +
093 -    45 25  RCL I
094 -       34  x<>y
095 -       30  -
096 -    45  9  RCL 9
097 -       10  ÷
098 -    44  9  STO 9
099 -    45 .0  RCL .0
100 -    45 .1  RCL .1
101 -    43 32  RTN
102 - 42,21,14  LBL D
103 -    42 48  ŷ,r
104 -    43 36  LST×
105 -       20  ×
106 -    45  9  RCL 9
107 -       40  +
108 -    43 32  RTN

Usage

• A: add data point (similar to Σ+)
• B: remove data point (similar to Σ-)
• C: calculate best fit (similar to L.R.)
• D: forecast (similar to ŷ)
  

Example

Q       H
(flow)  (head)
[m3/h]  [m]
----------------
  0     90.0
 30     90.2
 60     86.4
 90     79.0
120     67.7
150     52.4





90.0  ENTER    0  A
1

90.2  ENTER   30  A
2

86.4  ENTER   60  A
3

79.0  ENTER   90  A
4

67.7  ENTER  120  A
5

52.4  ENTER  150  A
6

C
-0.002132936508 (a)

R↓
0.06955952381 (b)

R↓
89.99642857 (c)


100  D
75.62301587 (y)


Hint

A linear regression can be performed before calculating the best fit.
This result can then later be compared with the quadratic regression.

References

• Quadratic fit without linear algebra (32SII)
• Small Solver Program
• Polynomial regression
• Least Squares Fitting--Polynomial
• Quadratic regression Calculator
  

[Csaba Tizedes]

Thanks for the credit at the end!
In the first matrix-coefficient equation row 3-coloumn 3 the sum(x^3) is wrong. Pls update.

Cs.

[Thomas Klemm]

Thank you for pointing out the typo.