# Ask HN: How do web graphing calculators zoom without floating point issues?

7 points by xeonmc 4 days ago

As an exercise, I've been trying my hands at implementing an interactive plotting backend, starting with rendering gridlines which are simple enough using homogeneous matrices, and auto-ticking are just a matter of finding the closest decade and deciding between half, single, and double decade delineations.

``````    pan(hcoords, dx, dy) = {
for x,y,w in hcoords
x <- 1*x + 0*y + dx*w
y <- 0*x + 1*y + dy*w
w <- 0*x + 0*y +  1*w
next
}

tickGap (range,nTick)
| delta<0           = 10^decade / 2
| delta>0           = 10^decade * 2
where thresh = log10(2)/2
delta  = power-round(power)
power  = log10(range/nTicks)
``````
However, for zooming, while it should also be straightforward using homogeneous coordinates, I run into an issue where continuously zooming into arbitrary points eventually causes floating point precision loss

``````    0.1 < (x=0.15) < 0.2
0.149 < (x=0.15) < 0.151
0.14999999999 < (x=0.15) < 0.15000000000
``````
which results in a mess of un-renderable gridlines leaving a blank canvas.

I thought this was just an unavoidable issue due to the nature of condition numbers in limited precision arithmetic, unless I try to figure out how to write my own bignum algorithm.

However, it seems that web-based graphing calculators such as Desmos render the canvas gridlines just fine, and seemingly without bignum arithmetics.

Is there perhaps some optimization "tricks" that I'm missing which lets you "fake" render the canvas just fine without being explicitly beholden to fp precision?

``     https://desmos.com/calculator``
xeonmc 2 days ago

eventually bandaid-ed it by preventing further zooming based on current precision

``````    zoom = {
fac ctr min max -> newMin newMax
newMin =
|     eps < 2^(-2^11) =  min
|     eps > 2^( 2^11) =  min
| avg/eps > 2^52      =  min
| otherwise           = (min-ctr)*fac + ctr
newMax =
|     eps < 2^(-2^11) =  max
|     eps > 2^( 2^11) =  max
| avg/eps > 2^25      =  max
| otherwise           = (max-ctr)*fac + ctr
avg = (max+min)/2
eps = (max-min)/2
}``````
pestatije 4 days ago

Maybe you can "fake" render the canvas, but what happens to the actual plot graph? Do you see fp artifacts there?