Solved: what is wrong with this LatLong->Distance code? A Trigonometry "feature".

[Guenter Schink]

(10-12-2016 09:46 PM)StephenG1CMZ Wrote:  Thank you for taking the time to look at this.
When I first saw your code I was thinking the lower-case Sin was exact and avoiding a rounding error - but it doesn't fix this example, which is still 1 km rather than 54 km.

Sorry the lower case "sin" was not intentionally.

But now I think we're coming closer. Don't know what really happens (a bug?). I added commands to print DLON and DLAT. And much to my surprise they showed up in DMS format. Consequently I removed the transformation to radians et voilá the result comes pretty close to what's expected.

Code:

EXPORT Haversine(LAT1,LON1,LAT2,LON2)
 BEGIN
  LOCAL RR:=6378.137;
  LOCAL DLON:=(LON2-LON1);
  LOCAL DLAT:=(LAT2-LAT1);
  LOCAL AA:=(SIN(DLAT/2))^2 + 
   COS(LAT1)*
   COS(LAT2)*(SIN(DLON/2))^2;
  LOCAL CC:=2*ASIN(MIN(1,√(AA)));//MIN 1 AVOIDS ROUNDING OUTRANGE
  LOCAL DD:=RR*CC;
  PRINT("DLAT " + DLAT);
  PRINT("DLON " + DLON);
  RETURN ("DD " + DD);
 END;

That's really weird. The "Angle Measure" is set to radians! I guess you have it too. The problem seems not to show up when entering the co-ordinates not in DMS but degrees (with decimal degrees)**. On the other hand, I don't understand, at least in the Haversine formula, why a transformation to radians is needed after all.

It will happen on the command line too. Just enter "12°30' * PI / 180" the expression will stay in DMS mode.

HTH, Günter

** That's obviously the reason why I didn't encounter the problem, as I entered the data in degrees format.
Günter