Post Reply 
(11C) Quadratic Regression
11-20-2022, 10:28 PM (This post was last modified: 11-24-2022 06:00 PM by Thomas Klemm.)
Post: #1
(11C) Quadratic Regression
Formula

Given a set of points \(\{(x_1, y_1), (x_2, y_2), \cdots, (x_n, y_n)\}\) where \(n \geqslant 3\) we want to find the best fit for a quadratic polynomial:

\(
\begin{align}
y = a \cdot x^2 + b \cdot x + c
\end{align}
\)

This leads to the following linear system of equations:

\(
\begin{bmatrix}
n & \Sigma x & \Sigma x^2 \\
\Sigma x & \Sigma x^2 & \Sigma x^3 \\
\Sigma x^2 & \Sigma x^3 & \Sigma x^4 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
c \\
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
\Sigma y \\
\Sigma xy \\
\Sigma x^2y \\
\end{bmatrix}
\)

We can perform Gauss elimination on the first row to get:

\(
\begin{bmatrix}
n & \Sigma x & \Sigma x^2 \\
0 & n \cdot \Sigma x^2 - (\Sigma x)^2 & n \cdot \Sigma x^3 - \Sigma x \cdot \Sigma x^2 \\
0 & n \cdot \Sigma x^3 - \Sigma x \cdot \Sigma x^2 & n \cdot \Sigma x^4 - (\Sigma x^2)^2 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
c \\
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
\Sigma y \\
n \cdot \Sigma xy - \Sigma x \cdot \Sigma y \\
n \cdot \Sigma x^2y - \Sigma x^2 \cdot \Sigma y \\
\end{bmatrix}
\)

We define the following:

\(
\begin{align}
S_{x^2} &= n \cdot \Sigma x^2 - (\Sigma x)^2 \\
S_{xy} &= n \cdot \Sigma xy - \Sigma x \cdot \Sigma y \\
S_{xx^2} &= n \cdot \Sigma x^3 - \Sigma x \cdot \Sigma x^2 \\
S_{x^2x^2} &= n \cdot \Sigma x^4 - (\Sigma x^2)^2 \\
S_{x^2y} &= n \cdot \Sigma x^2y - \Sigma x^2 \cdot \Sigma y \\
\end{align}
\)

This allows us to rewrite the linear system for \(b\) and \(a\) as:

\(
\begin{bmatrix}
S_{x^2} & S_{xx^2} \\
S_{xx^2} & S_{x^2x^2} \\
\end{bmatrix}
\cdot
\begin{bmatrix}
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
S_{xy} \\
S_{x^2y} \\
\end{bmatrix}
\)

Since the matrix is symmetric we can use the built-in function L.R. to solve it.

In case of linear regression we have the following linear system:

\(
\begin{bmatrix}
n & \Sigma x \\
\Sigma x & \Sigma x^2 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
\Sigma y \\
\Sigma xy \\
\end{bmatrix}
\)

Registers

We use the following registers:

\(
\begin{align}
R_{0} &: n \\
R_{1} &: \Sigma x \\
R_{2} &: \Sigma x^2 \\
R_{3} &: \Sigma y \\
R_{4} &: \Sigma y^2 \\
R_{5} &: \Sigma xy \\
R_{6} &: \Sigma x^3 \\
R_{7} &: \Sigma x^4 \\
R_{8} &: \Sigma x^2y \\
R_{9} &: c \\
R_{.0} &: b \\
R_{.1} &: a \\
I &: \text{scratch} \\
\end{align}
\)

Thus the function L.R. allows to solve the following linear system:

\(
\begin{bmatrix}
R_{0} & R_{1} \\
R_{1} & R_{2} \\
\end{bmatrix}
\cdot
\begin{bmatrix}
b \\
a \\
\end{bmatrix}
=
\begin{bmatrix}
R_{3} \\
R_{5} \\
\end{bmatrix}
\)

This means we have to do the following assignments:

\(
\begin{align}
R_{0} &\leftarrow R_{0} \cdot R_{2} - R_{1}^2 \\
R_{1} &\leftarrow R_{0} \cdot R_{6} - R_{1} \cdot R_{2} \\
R_{2} &\leftarrow R_{0} \cdot R_{7} - R_{2}^2 \\
R_{3} &\leftarrow R_{0} \cdot R_{5} - R_{1} \cdot R_{3} \\
R_{5} &\leftarrow R_{0} \cdot R_{8} - R_{2} \cdot R_{3} \\
\end{align}
\)

We temporarily store the previous values in the following registers:

\(
\begin{align}
R_{9} &\leftarrow R_{0} \\
R_{.0} &\leftarrow R_{1} \\
R_{.1} &\leftarrow R_{2} \\
I &\leftarrow R_{3} \\
\end{align}
\)

Forecast

The linear estimate function ŷ allows to calculate:

\(
\begin{align}
\hat{y} &= a \cdot x + b
\end{align}
\)

This can be used to calculate:

\(
\begin{align}
\hat{y} \cdot x + c
&= (a \cdot x + b) \cdot x + c \\
&= a \cdot x^2 + b \cdot x + c \\
\end{align}
\)

Program

Code:
001 - 42,21,11  LBL A
002 -       36  ENTER
003 -       36  ENTER
004 -    43 11  x↑2
005 -       20  ×
006 - 44,40, 6  STO+ 6
007 -       33  R↓
008 -    43 36  LST×
009 -    43 33  R↑
010 -       34  x<>y
011 -       20  ×
012 - 44,40, 8  STO+ 8
013 -       33  R↓
014 -    43 36  LST×
015 -    43 11  x↑2
016 - 44,40, 7  STO+ 7
017 -       33  R↓
018 -       49  Σ+
019 -    43 32  RTN
020 - 42,21,12  LBL B
021 -       36  ENTER
022 -       36  ENTER
023 -    43 11  x↑2
024 -       20  ×
025 - 44,30, 6  STO- 6
026 -       33  R↓
027 -    43 36  LST×
028 -    43 33  R↑
029 -       34  x<>y
030 -       20  ×
031 - 44,30, 8  STO- 8
032 -       33  R↓
033 -    43 36  LST×
034 -    43 11  x↑2
035 - 44,30, 7  STO- 7
036 -       33  R↓
037 -    43 49  Σ-
038 -    43 32  RTN
039 - 42,21,13  LBL C
040 -    45  0  RCL 0
041 -    44  9  STO 9
042 -    45  2  RCL 2
043 -    44 .1  STO .1
044 -       20  ×
045 -    45  1  RCL 1
046 -    44 .0  STO .0
047 -    43 11  x↑2
048 -       30  -
049 -    44  0  STO 0
050 -    45  9  RCL 9
051 -    45  6  RCL 6
052 -       20  ×
053 -    45 .0  RCL .0
054 -    45 .1  RCL .1
055 -       20  ×
056 -       30  -
057 -    44  1  STO 1
058 -    45  9  RCL 9
059 -    45  7  RCL 7
060 -       20  ×
061 -    45 .1  RCL .1
062 -    43 11  x↑2
063 -       30  -
064 -    44  2  STO 2
065 -    45  9  RCL 9
066 -    45  5  RCL 5
067 -       20  ×
068 -    45 .0  RCL .0
069 -    45  3  RCL 3
070 -    44 25  STO I
071 -       20  ×
072 -       30  -
073 -    44  3  STO 3
074 -    45  9  RCL 9
075 -    45  8  RCL 8
076 -       20  ×
077 -    45 .1  RCL .1
078 -    45 25  RCL I
079 -       20  ×
080 -       30  -
081 -    44  5  STO 5
082 -    42 49  L.R.
083 -    45 .0  RCL .0
084 -       34  x<>y
085 -    44 .0  STO .0
086 -       20  ×
087 -       34  x<>y
088 -    45 .1  RCL .1
089 -       34  x<>y
090 -    44 .1  STO .1
091 -       20  ×
092 -       40  +
093 -    45 25  RCL I
094 -       34  x<>y
095 -       30  -
096 -    45  9  RCL 9
097 -       10  ÷
098 -    44  9  STO 9
099 -    45 .0  RCL .0
100 -    45 .1  RCL .1
101 -    43 32  RTN
102 - 42,21,14  LBL D
103 -    42 48  ŷ,r
104 -    43 36  LST×
105 -       20  ×
106 -    45  9  RCL 9
107 -       40  +
108 -    43 32  RTN

Usage
  • A: add data point (similar to Σ+)
  • B: remove data point (similar to Σ-)
  • C: calculate best fit (similar to L.R.)
  • D: forecast (similar to ŷ)

Example

Q       H
(flow)  (head)
[m3/h]  [m]
----------------
  0     90.0
 30     90.2
 60     86.4
 90     79.0
120     67.7
150     52.4


[Image: attachment.php?aid=11441]


90.0  ENTER    0  A
1

90.2  ENTER   30  A
2

86.4  ENTER   60  A
3

79.0  ENTER   90  A
4

67.7  ENTER  120  A
5

52.4  ENTER  150  A
6

C
-0.002132936508 (a)

R↓
0.06955952381 (b)

R↓
89.99642857 (c)


100  D
75.62301587 (y)


Hint

A linear regression can be performed before calculating the best fit.
This result can then later be compared with the quadratic regression.

References


Attached File(s) Thumbnail(s)
   
Find all posts by this user
Quote this message in a reply
11-24-2022, 01:24 PM
Post: #2
RE: (11C) Quadratic Regression
Thanks for the credit at the end! Wink
In the first matrix-coefficient equation row 3-coloumn 3 the sum(x^3) is wrong. Pls update.

Cs.
Find all posts by this user
Quote this message in a reply
11-24-2022, 06:01 PM
Post: #3
RE: (11C) Quadratic Regression
Thank you for pointing out the typo.
Find all posts by this user
Quote this message in a reply
Post Reply 




User(s) browsing this thread: 1 Guest(s)