Ranks calculation

[robmio]

Hello everyone, some statistical tests, especially non-parametric ones, require the calculation of ranks. Is there a way to calculate them with HP PRIME?

For example, given a column vector:

[[1],[3],[5],[1],[4],[5],[6],[1]]

the ranks are:

[[2],[4],[6.5],[2],[5],[6.5],[8],[2]].

Thanks for your help, Roberto

[Albert Chan]

(10-31-2024 01:39 PM)robmio Wrote:  For example, given a column vector:

[[1],[3],[5],[1],[4],[5],[6],[1]]

the ranks are:

[[2],[4],[6.5],[2],[5],[6.5],[8],[2]].

Is rank of matrix supposed to be a single number?
What does above list of numbers mean?

[robmio]

(10-31-2024 02:38 PM)Albert Chan Wrote:  

(10-31-2024 01:39 PM)robmio Wrote:  For example, given a column vector:

[[1],[3],[5],[1],[4],[5],[6],[1]]

the ranks are:

[[2],[4],[6.5],[2],[5],[6.5],[8],[2]].

Is rank of matrix supposed to be a single number?
What does above list of numbers mean?

It is not the rank of a matrix, but the scores to be given to the various values. For example, ranks are used in the statistical test "Wilcoxon signed-rank test": https://www.technologynetworks.com/infor...est-370384

[robmio]

See also: https://real-statistics.com/non-parametr...anks-test/

[Albert Chan]

Code:

def ranking(m):
    hi = len(m)
    m2 = zip(m, range(hi))
    m2.sort()        # sort with index
    lo, m3 = 0, m[:] # work with a copy
    for i in range(hi):
        j = i+1
        if j != hi and m2[i][0]==m2[j][0]: continue    
        r = (lo+i)/2 # j-lo items have same ranking
        for k in range(lo,j): m3[m2[k][1]] = r+1
        lo = j
    return m3

p2> ranking([1,3,5,1,4,5,6,1])
[2.0, 4.0, 6.5, 2.0, 5.0, 6.5, 8.0, 2.0]

[robmio]

(10-31-2024 04:41 PM)Albert Chan Wrote:  

Code:

def ranking(m):
    hi = len(m)
    m2 = zip(m, range(hi))
    m2.sort()        # sort with index
    lo, m3 = 0, m[:] # work with a copy
    for i in range(hi):
        j = i+1
        if j != hi and m2[i][0]==m2[j][0]: continue    
        r = (lo+i)/2 # j-lo items have same ranking
        for k in range(lo,j): m3[m2[k][1]] = r+1
        lo = j
    return m3

p2> ranking([1,3,5,1,4,5,6,1])
[2.0, 4.0, 6.5, 2.0, 5.0, 6.5, 8.0, 2.0]

Thank you so much, Albert Chan. I really appreciate it! Kind regards, Roberto

[Albert Chan]

Note that Python has 0-based array.
If code were for Python use, I would assign r back to the ranking list, instead of r+1

Lua array is 1-based, and probably easier to port to HP Prime

Code:

function ranking(m)
    local m2, m3, lo, hi = {}, {}, 1, #m
    for i=1, hi do m2[i] = {m[i],i} end
    table.sort(m2, function(x,y) return x[1]<y[1] end)
    for i=1, hi do
        local j = i+1
        if i<hi and m2[i][1]==m2[j][1] then continue end
        local r = (lo+i)/2
        for k=lo, i do m3[m2[k][2]] = r end
        lo = j
    end
    return m3
end

lua> pprint( ranking{1,3,5,1,4,5,6,1} )
{ 2, 4, 6.5, 2, 5, 6.5, 8, 2 }

[robmio]

(10-31-2024 08:42 PM)Albert Chan Wrote:  Note that Python has 0-based array.
If code were for Python use, I would assign r back to the ranking list, instead of r+1

Lua array is 1-based, and probably easier to port to HP Prime

Code:

function ranking(m)
    local m2, m3, lo, hi = {}, {}, 1, #m
    for i=1, hi do m2[i] = {m[i],i} end
    table.sort(m2, function(x,y) return x[1]<y[1] end)
    for i=1, hi do
        local j = i+1
        if i<hi and m2[i][1]==m2[j][1] then continue end
        local r = (lo+i)/2
        for k=lo, i do m3[m2[k][2]] = r end
        lo = j
    end
    return m3
end

lua> pprint( ranking{1,3,5,1,4,5,6,1} )
{ 2, 4, 6.5, 2, 5, 6.5, 8, 2 }

Thank you very much, Albert Chan. I have translated your Python program into a Cas program:

Code:


#cas
ranking(m):=
BEGIN
LOCAL hi, m2, m3, lo, j, ii, r, k;
hi:= length(m);
m2:=zip(m, range(hi).+1);
m2:=sort(m2);
lo:=1;
m3:=m;
FOR ii FROM 1 TO hi DO
j:=ii+1;
IF j≠hi AND m2[ii][1]==m2[j][1] THEN
CONTINUE;
END;
r:=(lo+ii)/2;
FOR k FROM lo TO ii DO
m3[m2[k][2]]:=r;
END;
lo:=j;
END;
RETURN m3;
END;
#end

[robmio]

Unfortunately, there is an error in the following program, and I can't figure out what it is:

Code:


#cas
ranking(m):=
BEGIN
LOCAL hi, m2, m3, lo, j, ii, r, k;
hi:= length(m);
m2:=zip(m, range(hi).+1);
m2:=sort(m2);
lo:=1;
m3:=m;
FOR ii FROM 1 TO hi DO
j:=ii+1;
IF j≠hi AND m2[ii][1]==m2[j][1] THEN
CONTINUE;
END;
r:=(lo+ii)/2;
FOR k FROM lo TO ii DO
m3[m2[k][2]]:=r;
END;
lo:=j;
END;
RETURN m3;
END;
#end

For example:

ranking([7,5,7]) --> [2,1,3]

instead of:

ranking([7,5,7]) --> [2.5,1,2.5]

[Albert Chan]

(11-01-2024 09:04 AM)robmio Wrote:  Unfortunately, there is an error in the following program, and I can't figure out what it is:

Off-by-1 error (Lua code use i<hi, with same effect)
i≠hi guard is to *ALWAYS* update last ranking. (*)

< IF j≠hi AND m2[ii][1]==m2[j][1] THEN
> IF ii≠hi AND m2[ii][1]==m2[j][1] THEN

(*) It is weird bad code does not generate Index outside range error, when j = hi+1

Update: fixed typo, should be ii instead of i

[robmio]

(11-01-2024 09:38 AM)Albert Chan Wrote:  

(11-01-2024 09:04 AM)robmio Wrote:  Unfortunately, there is an error in the following program, and I can't figure out what it is:

Off-by-1 error (Lua code use i<hi, with same effect)
i≠hi guard is to *ALWAYS* update last ranking. (*)

< IF j≠hi AND m2[ii][1]==m2[j][1] THEN
> IF i≠hi AND m2[ii][1]==m2[j][1] THEN

(*) It is weird bad code does not generate Index outside range error, when j = hi+1

Hi Albert Chan, your fix works fine. Also, if we replace "i" with "j" in the following line:

"FOR k FROM lo TO ii DO"

the code generates Index outside range error.

Thanks again, many thanks!

[robmio]

This version works fine.

Code:


#cas
ranking(m):=
BEGIN
LOCAL hi, m2, m3, lo, j, ii, r, k;
hi:= length(m);
m2:=zip(m, range(hi).+1);
m2:=sort(m2);
lo:=1;
m3:=m;
FOR ii FROM 1 TO hi DO
j:=ii+1;
IF ii≠hi AND m2[ii][1]==m2[j][1] THEN
CONTINUE;
END;
r:=(lo+ii)/2;
FOR k FROM lo TO ii DO
m3[m2[k][2]]:=r;
END;
lo:=j;
END;
RETURN m3;
END;
#end

[Albert Chan]

(11-01-2024 10:03 AM)robmio Wrote:  the code generates Index outside range error.

I mean when we do equality test, Index outside range not producing error

Cas> m := [1,2,3]
Cas> m[3] == m[4]      → 0
Cas> m[4] == m[5]      → 0
Cas> m[5] == m[6]      → 0

m2[hi][1]==m2[hi+1][1] guard should also generate 0, i.e. always update last ranking.

If we use this undefined behavior (not recommended), we can drop ii≠hi guard.

Interestingly, I discovered this due to a typo , i≠hi instead of ii≠hi
Since i = √(-1) is imaginary, i≠hi is always true, same as no guard.