Remove long integer output.
This commit is contained in:
parent
3d1c7dec4a
commit
bae1b94d9e
|
@ -173,24 +173,24 @@ and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``),
|
|||
the best value for *N* is 56::
|
||||
|
||||
>>> 2**52
|
||||
4503599627370496L
|
||||
4503599627370496
|
||||
>>> 2**53
|
||||
9007199254740992L
|
||||
9007199254740992
|
||||
>>> 2**56/10
|
||||
7205759403792793L
|
||||
7205759403792794.0
|
||||
|
||||
That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The
|
||||
best possible value for *J* is then that quotient rounded::
|
||||
|
||||
>>> q, r = divmod(2**56, 10)
|
||||
>>> r
|
||||
6L
|
||||
6
|
||||
|
||||
Since the remainder is more than half of 10, the best approximation is obtained
|
||||
by rounding up::
|
||||
|
||||
>>> q+1
|
||||
7205759403792794L
|
||||
7205759403792794
|
||||
|
||||
Therefore the best possible approximation to 1/10 in 754 double precision is
|
||||
that over 2\*\*56, or ::
|
||||
|
@ -211,7 +211,7 @@ If we multiply that fraction by 10\*\*30, we can see the (truncated) value of
|
|||
its 30 most significant decimal digits::
|
||||
|
||||
>>> 7205759403792794 * 10**30 / 2**56
|
||||
100000000000000005551115123125L
|
||||
100000000000000005551115123125
|
||||
|
||||
meaning that the exact number stored in the computer is approximately equal to
|
||||
the decimal value 0.100000000000000005551115123125. Rounding that to 17
|
||||
|
|
Loading…
Reference in New Issue