traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
cpython/Lib/test/decimaltestdata/quantize.decTest

949 lines
46 KiB
Plaintext
Raw Normal View History

Move Decimal from the sandbox into production.
2004-07-01 08:01:35 -03:00
------------------------------------------------------------------------
-- quantize.decTest -- decimal quantize operation --
Issue #3449: Update decimal module to use most recent specification (v. 1.68) and tests (v. 2.58) from IBM.
2008-07-27 03:39:07 -03:00
-- Copyright (c) IBM Corporation, 1981, 2008. All rights reserved. --
Move Decimal from the sandbox into production.
2004-07-01 08:01:35 -03:00
------------------------------------------------------------------------
-- Please see the document "General Decimal Arithmetic Testcases" --
-- at http://www2.hursley.ibm.com/decimal for the description of --
-- these testcases. --
-- --
-- These testcases are experimental ('beta' versions), and they --
-- may contain errors. They are offered on an as-is basis. In --
-- particular, achieving the same results as the tests here is not --
-- a guarantee that an implementation complies with any Standard --
-- or specification. The tests are not exhaustive. --
-- --
-- Please send comments, suggestions, and corrections to the author: --
-- Mike Cowlishaw, IBM Fellow --
-- IBM UK, PO Box 31, Birmingham Road, Warwick CV34 5JL, UK --
-- mfc@uk.ibm.com --
------------------------------------------------------------------------
Merged revisions 75286 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r75286 | mark.dickinson | 2009-10-08 17:28:39 +0100 (Thu, 08 Oct 2009) | 2 lines Update decimal test data to the most recent set from Mike Cowlishaw. ........
2009-10-27 14:02:46 -03:00
version: 2.59
Move Decimal from the sandbox into production.
2004-07-01 08:01:35 -03:00
-- Most of the tests here assume a "regular pattern", where the
-- sign and coefficient are +1.
-- 2004.03.15 Underflow for quantize is suppressed
Merged the decimal-branch (revisions 54886 to 58140). Decimal is now fully updated to the latests Decimal Specification (v1.66) and the latests test cases (v2.56). Thanks to Mark Dickinson for all his help during this process.
2007-09-13 15:13:15 -03:00
-- 2005.06.08 More extensive tests for 'does not fit'
Move Decimal from the sandbox into production.
2004-07-01 08:01:35 -03:00
extended: 1
precision: 9
rounding: half_up
maxExponent: 999
minexponent: -999
-- sanity checks
quax001 quantize 0 1e0 -> 0
quax002 quantize 1 1e0 -> 1
quax003 quantize 0.1 1e+2 -> 0E+2 Inexact Rounded
quax005 quantize 0.1 1e+1 -> 0E+1 Inexact Rounded
quax006 quantize 0.1 1e0 -> 0 Inexact Rounded
quax007 quantize 0.1 1e-1 -> 0.1
quax008 quantize 0.1 1e-2 -> 0.10
quax009 quantize 0.1 1e-3 -> 0.100
quax010 quantize 0.9 1e+2 -> 0E+2 Inexact Rounded
quax011 quantize 0.9 1e+1 -> 0E+1 Inexact Rounded
quax012 quantize 0.9 1e+0 -> 1 Inexact Rounded
quax013 quantize 0.9 1e-1 -> 0.9
quax014 quantize 0.9 1e-2 -> 0.90
quax015 quantize 0.9 1e-3 -> 0.900
-- negatives
quax021 quantize -0 1e0 -> -0
quax022 quantize -1 1e0 -> -1
quax023 quantize -0.1 1e+2 -> -0E+2 Inexact Rounded
quax025 quantize -0.1 1e+1 -> -0E+1 Inexact Rounded
quax026 quantize -0.1 1e0 -> -0 Inexact Rounded
quax027 quantize -0.1 1e-1 -> -0.1
quax028 quantize -0.1 1e-2 -> -0.10
quax029 quantize -0.1 1e-3 -> -0.100
quax030 quantize -0.9 1e+2 -> -0E+2 Inexact Rounded
quax031 quantize -0.9 1e+1 -> -0E+1 Inexact Rounded
quax032 quantize -0.9 1e+0 -> -1 Inexact Rounded
quax033 quantize -0.9 1e-1 -> -0.9
quax034 quantize -0.9 1e-2 -> -0.90
quax035 quantize -0.9 1e-3 -> -0.900
quax036 quantize -0.5 1e+2 -> -0E+2 Inexact Rounded
quax037 quantize -0.5 1e+1 -> -0E+1 Inexact Rounded
quax038 quantize -0.5 1e+0 -> -1 Inexact Rounded
quax039 quantize -0.5 1e-1 -> -0.5
quax040 quantize -0.5 1e-2 -> -0.50
quax041 quantize -0.5 1e-3 -> -0.500
quax042 quantize -0.9 1e+2 -> -0E+2 Inexact Rounded
quax043 quantize -0.9 1e+1 -> -0E+1 Inexact Rounded
quax044 quantize -0.9 1e+0 -> -1 Inexact Rounded
quax045 quantize -0.9 1e-1 -> -0.9
quax046 quantize -0.9 1e-2 -> -0.90
quax047 quantize -0.9 1e-3 -> -0.900
-- examples from Specification
quax060 quantize 2.17 0.001 -> 2.170
quax061 quantize 2.17 0.01 -> 2.17
quax062 quantize 2.17 0.1 -> 2.2 Inexact Rounded
quax063 quantize 2.17 1e+0 -> 2 Inexact Rounded
quax064 quantize 2.17 1e+1 -> 0E+1 Inexact Rounded
quax065 quantize -Inf Inf -> -Infinity
quax066 quantize 2 Inf -> NaN Invalid_operation
quax067 quantize -0.1 1 -> -0 Inexact Rounded
quax068 quantize -0 1e+5 -> -0E+5
quax069 quantize +35236450.6 1e-2 -> NaN Invalid_operation
quax070 quantize -35236450.6 1e-2 -> NaN Invalid_operation
quax071 quantize 217 1e-1 -> 217.0
quax072 quantize 217 1e+0 -> 217
quax073 quantize 217 1e+1 -> 2.2E+2 Inexact Rounded
quax074 quantize 217 1e+2 -> 2E+2 Inexact Rounded
-- general tests ..
quax089 quantize 12 1e+4 -> 0E+4 Inexact Rounded
quax090 quantize 12 1e+3 -> 0E+3 Inexact Rounded
quax091 quantize 12 1e+2 -> 0E+2 Inexact Rounded
quax092 quantize 12 1e+1 -> 1E+1 Inexact Rounded
quax093 quantize 1.2345 1e-2 -> 1.23 Inexact Rounded
quax094 quantize 1.2355 1e-2 -> 1.24 Inexact Rounded
quax095 quantize 1.2345 1e-6 -> 1.234500
quax096 quantize 9.9999 1e-2 -> 10.00 Inexact Rounded
quax097 quantize 0.0001 1e-2 -> 0.00 Inexact Rounded
quax098 quantize 0.001 1e-2 -> 0.00 Inexact Rounded
quax099 quantize 0.009 1e-2 -> 0.01 Inexact Rounded
quax100 quantize 92 1e+2 -> 1E+2 Inexact Rounded
quax101 quantize -1 1e0 -> -1
quax102 quantize -1 1e-1 -> -1.0
quax103 quantize -1 1e-2 -> -1.00
quax104 quantize 0 1e0 -> 0
quax105 quantize 0 1e-1 -> 0.0
quax106 quantize 0 1e-2 -> 0.00
quax107 quantize 0.00 1e0 -> 0
quax108 quantize 0 1e+1 -> 0E+1
quax109 quantize 0 1e+2 -> 0E+2
quax110 quantize +1 1e0 -> 1
quax111 quantize +1 1e-1 -> 1.0
quax112 quantize +1 1e-2 -> 1.00
quax120 quantize 1.04 1e-3 -> 1.040
quax121 quantize 1.04 1e-2 -> 1.04
quax122 quantize 1.04 1e-1 -> 1.0 Inexact Rounded
quax123 quantize 1.04 1e0 -> 1 Inexact Rounded
quax124 quantize 1.05 1e-3 -> 1.050
quax125 quantize 1.05 1e-2 -> 1.05
quax126 quantize 1.05 1e-1 -> 1.1 Inexact Rounded
quax131 quantize 1.05 1e0 -> 1 Inexact Rounded
quax132 quantize 1.06 1e-3 -> 1.060
quax133 quantize 1.06 1e-2 -> 1.06
quax134 quantize 1.06 1e-1 -> 1.1 Inexact Rounded
quax135 quantize 1.06 1e0 -> 1 Inexact Rounded
quax140 quantize -10 1e-2 -> -10.00
quax141 quantize +1 1e-2 -> 1.00
quax142 quantize +10 1e-2 -> 10.00
quax143 quantize 1E+10 1e-2 -> NaN Invalid_operation
quax144 quantize 1E-10 1e-2 -> 0.00 Inexact Rounded
quax145 quantize 1E-3 1e-2 -> 0.00 Inexact Rounded
quax146 quantize 1E-2 1e-2 -> 0.01
quax147 quantize 1E-1 1e-2 -> 0.10
quax148 quantize 0E-10 1e-2 -> 0.00
quax150 quantize 1.0600 1e-5 -> 1.06000
quax151 quantize 1.0600 1e-4 -> 1.0600
quax152 quantize 1.0600 1e-3 -> 1.060 Rounded
quax153 quantize 1.0600 1e-2 -> 1.06 Rounded
quax154 quantize 1.0600 1e-1 -> 1.1 Inexact Rounded
quax155 quantize 1.0600 1e0 -> 1 Inexact Rounded
-- base tests with non-1 coefficients
quax161 quantize 0 -9e0 -> 0
quax162 quantize 1 -7e0 -> 1
quax163 quantize 0.1 -1e+2 -> 0E+2 Inexact Rounded
quax165 quantize 0.1 0e+1 -> 0E+1 Inexact Rounded
quax166 quantize 0.1 2e0 -> 0 Inexact Rounded
quax167 quantize 0.1 3e-1 -> 0.1
quax168 quantize 0.1 44e-2 -> 0.10
quax169 quantize 0.1 555e-3 -> 0.100
quax170 quantize 0.9 6666e+2 -> 0E+2 Inexact Rounded
quax171 quantize 0.9 -777e+1 -> 0E+1 Inexact Rounded
quax172 quantize 0.9 -88e+0 -> 1 Inexact Rounded
quax173 quantize 0.9 -9e-1 -> 0.9
quax174 quantize 0.9 0e-2 -> 0.90
quax175 quantize 0.9 1.1e-3 -> 0.9000
-- negatives
quax181 quantize -0 1.1e0 -> -0.0
quax182 quantize -1 -1e0 -> -1
quax183 quantize -0.1 11e+2 -> -0E+2 Inexact Rounded
quax185 quantize -0.1 111e+1 -> -0E+1 Inexact Rounded
quax186 quantize -0.1 71e0 -> -0 Inexact Rounded
quax187 quantize -0.1 -91e-1 -> -0.1
quax188 quantize -0.1 -.1e-2 -> -0.100
quax189 quantize -0.1 -1e-3 -> -0.100
quax190 quantize -0.9 0e+2 -> -0E+2 Inexact Rounded
quax191 quantize -0.9 -0e+1 -> -0E+1 Inexact Rounded
quax192 quantize -0.9 -10e+0 -> -1 Inexact Rounded
quax193 quantize -0.9 100e-1 -> -0.9
quax194 quantize -0.9 999e-2 -> -0.90
-- +ve exponents ..
quax201 quantize -1 1e+0 -> -1
quax202 quantize -1 1e+1 -> -0E+1 Inexact Rounded
quax203 quantize -1 1e+2 -> -0E+2 Inexact Rounded
quax204 quantize 0 1e+0 -> 0
quax205 quantize 0 1e+1 -> 0E+1
quax206 quantize 0 1e+2 -> 0E+2
quax207 quantize +1 1e+0 -> 1
quax208 quantize +1 1e+1 -> 0E+1 Inexact Rounded
quax209 quantize +1 1e+2 -> 0E+2 Inexact Rounded
quax220 quantize 1.04 1e+3 -> 0E+3 Inexact Rounded
quax221 quantize 1.04 1e+2 -> 0E+2 Inexact Rounded
quax222 quantize 1.04 1e+1 -> 0E+1 Inexact Rounded
quax223 quantize 1.04 1e+0 -> 1 Inexact Rounded
quax224 quantize 1.05 1e+3 -> 0E+3 Inexact Rounded
quax225 quantize 1.05 1e+2 -> 0E+2 Inexact Rounded
quax226 quantize 1.05 1e+1 -> 0E+1 Inexact Rounded
quax227 quantize 1.05 1e+0 -> 1 Inexact Rounded
quax228 quantize 1.05 1e+3 -> 0E+3 Inexact Rounded
quax229 quantize 1.05 1e+2 -> 0E+2 Inexact Rounded
quax230 quantize 1.05 1e+1 -> 0E+1 Inexact Rounded
quax231 quantize 1.05 1e+0 -> 1 Inexact Rounded
quax232 quantize 1.06 1e+3 -> 0E+3 Inexact Rounded
quax233 quantize 1.06 1e+2 -> 0E+2 Inexact Rounded
quax234 quantize 1.06 1e+1 -> 0E+1 Inexact Rounded
quax235 quantize 1.06 1e+0 -> 1 Inexact Rounded
quax240 quantize -10 1e+1 -> -1E+1 Rounded
quax241 quantize +1 1e+1 -> 0E+1 Inexact Rounded
quax242 quantize +10 1e+1 -> 1E+1 Rounded
quax243 quantize 1E+1 1e+1 -> 1E+1 -- underneath this is E+1
quax244 quantize 1E+2 1e+1 -> 1.0E+2 -- underneath this is E+1
quax245 quantize 1E+3 1e+1 -> 1.00E+3 -- underneath this is E+1
quax246 quantize 1E+4 1e+1 -> 1.000E+4 -- underneath this is E+1
quax247 quantize 1E+5 1e+1 -> 1.0000E+5 -- underneath this is E+1
quax248 quantize 1E+6 1e+1 -> 1.00000E+6 -- underneath this is E+1
quax249 quantize 1E+7 1e+1 -> 1.000000E+7 -- underneath this is E+1
quax250 quantize 1E+8 1e+1 -> 1.0000000E+8 -- underneath this is E+1
quax251 quantize 1E+9 1e+1 -> 1.00000000E+9 -- underneath this is E+1
-- next one tries to add 9 zeros
quax252 quantize 1E+10 1e+1 -> NaN Invalid_operation
quax253 quantize 1E-10 1e+1 -> 0E+1 Inexact Rounded
quax254 quantize 1E-2 1e+1 -> 0E+1 Inexact Rounded
quax255 quantize 0E-10 1e+1 -> 0E+1
quax256 quantize -0E-10 1e+1 -> -0E+1
quax257 quantize -0E-1 1e+1 -> -0E+1
quax258 quantize -0 1e+1 -> -0E+1
quax259 quantize -0E+1 1e+1 -> -0E+1
quax260 quantize -10 1e+2 -> -0E+2 Inexact Rounded
quax261 quantize +1 1e+2 -> 0E+2 Inexact Rounded
quax262 quantize +10 1e+2 -> 0E+2 Inexact Rounded
quax263 quantize 1E+1 1e+2 -> 0E+2 Inexact Rounded
quax264 quantize 1E+2 1e+2 -> 1E+2
quax265 quantize 1E+3 1e+2 -> 1.0E+3
quax266 quantize 1E+4 1e+2 -> 1.00E+4
quax267 quantize 1E+5 1e+2 -> 1.000E+5
quax268 quantize 1E+6 1e+2 -> 1.0000E+6
quax269 quantize 1E+7 1e+2 -> 1.00000E+7
quax270 quantize 1E+8 1e+2 -> 1.000000E+8
quax271 quantize 1E+9 1e+2 -> 1.0000000E+9
quax272 quantize 1E+10 1e+2 -> 1.00000000E+10
quax273 quantize 1E-10 1e+2 -> 0E+2 Inexact Rounded
quax274 quantize 1E-2 1e+2 -> 0E+2 Inexact Rounded
quax275 quantize 0E-10 1e+2 -> 0E+2
quax280 quantize -10 1e+3 -> -0E+3 Inexact Rounded
quax281 quantize +1 1e+3 -> 0E+3 Inexact Rounded
quax282 quantize +10 1e+3 -> 0E+3 Inexact Rounded
quax283 quantize 1E+1 1e+3 -> 0E+3 Inexact Rounded
quax284 quantize 1E+2 1e+3 -> 0E+3 Inexact Rounded
quax285 quantize 1E+3 1e+3 -> 1E+3
quax286 quantize 1E+4 1e+3 -> 1.0E+4
quax287 quantize 1E+5 1e+3 -> 1.00E+5
quax288 quantize 1E+6 1e+3 -> 1.000E+6
quax289 quantize 1E+7 1e+3 -> 1.0000E+7
quax290 quantize 1E+8 1e+3 -> 1.00000E+8
quax291 quantize 1E+9 1e+3 -> 1.000000E+9
quax292 quantize 1E+10 1e+3 -> 1.0000000E+10
quax293 quantize 1E-10 1e+3 -> 0E+3 Inexact Rounded
quax294 quantize 1E-2 1e+3 -> 0E+3 Inexact Rounded
quax295 quantize 0E-10 1e+3 -> 0E+3
-- round up from below [sign wrong in JIT compiler once]
quax300 quantize 0.0078 1e-5 -> 0.00780
quax301 quantize 0.0078 1e-4 -> 0.0078
quax302 quantize 0.0078 1e-3 -> 0.008 Inexact Rounded
quax303 quantize 0.0078 1e-2 -> 0.01 Inexact Rounded
quax304 quantize 0.0078 1e-1 -> 0.0 Inexact Rounded
quax305 quantize 0.0078 1e0 -> 0 Inexact Rounded
quax306 quantize 0.0078 1e+1 -> 0E+1 Inexact Rounded
quax307 quantize 0.0078 1e+2 -> 0E+2 Inexact Rounded
quax310 quantize -0.0078 1e-5 -> -0.00780
quax311 quantize -0.0078 1e-4 -> -0.0078
quax312 quantize -0.0078 1e-3 -> -0.008 Inexact Rounded
quax313 quantize -0.0078 1e-2 -> -0.01 Inexact Rounded
quax314 quantize -0.0078 1e-1 -> -0.0 Inexact Rounded
quax315 quantize -0.0078 1e0 -> -0 Inexact Rounded
quax316 quantize -0.0078 1e+1 -> -0E+1 Inexact Rounded
quax317 quantize -0.0078 1e+2 -> -0E+2 Inexact Rounded
quax320 quantize 0.078 1e-5 -> 0.07800
quax321 quantize 0.078 1e-4 -> 0.0780
quax322 quantize 0.078 1e-3 -> 0.078
quax323 quantize 0.078 1e-2 -> 0.08 Inexact Rounded
quax324 quantize 0.078 1e-1 -> 0.1 Inexact Rounded
quax325 quantize 0.078 1e0 -> 0 Inexact Rounded
quax326 quantize 0.078 1e+1 -> 0E+1 Inexact Rounded
quax327 quantize 0.078 1e+2 -> 0E+2 Inexact Rounded
quax330 quantize -0.078 1e-5 -> -0.07800
quax331 quantize -0.078 1e-4 -> -0.0780
quax332 quantize -0.078 1e-3 -> -0.078
quax333 quantize -0.078 1e-2 -> -0.08 Inexact Rounded
quax334 quantize -0.078 1e-1 -> -0.1 Inexact Rounded
quax335 quantize -0.078 1e0 -> -0 Inexact Rounded
quax336 quantize -0.078 1e+1 -> -0E+1 Inexact Rounded
quax337 quantize -0.078 1e+2 -> -0E+2 Inexact Rounded
quax340 quantize 0.78 1e-5 -> 0.78000
quax341 quantize 0.78 1e-4 -> 0.7800
quax342 quantize 0.78 1e-3 -> 0.780
quax343 quantize 0.78 1e-2 -> 0.78
quax344 quantize 0.78 1e-1 -> 0.8 Inexact Rounded
quax345 quantize 0.78 1e0 -> 1 Inexact Rounded
quax346 quantize 0.78 1e+1 -> 0E+1 Inexact Rounded
quax347 quantize 0.78 1e+2 -> 0E+2 Inexact Rounded
quax350 quantize -0.78 1e-5 -> -0.78000
quax351 quantize -0.78 1e-4 -> -0.7800
quax352 quantize -0.78 1e-3 -> -0.780
quax353 quantize -0.78 1e-2 -> -0.78
quax354 quantize -0.78 1e-1 -> -0.8 Inexact Rounded
quax355 quantize -0.78 1e0 -> -1 Inexact Rounded
quax356 quantize -0.78 1e+1 -> -0E+1 Inexact Rounded
quax357 quantize -0.78 1e+2 -> -0E+2 Inexact Rounded
quax360 quantize 7.8 1e-5 -> 7.80000
quax361 quantize 7.8 1e-4 -> 7.8000
quax362 quantize 7.8 1e-3 -> 7.800
quax363 quantize 7.8 1e-2 -> 7.80
quax364 quantize 7.8 1e-1 -> 7.8
quax365 quantize 7.8 1e0 -> 8 Inexact Rounded
quax366 quantize 7.8 1e+1 -> 1E+1 Inexact Rounded
quax367 quantize 7.8 1e+2 -> 0E+2 Inexact Rounded
quax368 quantize 7.8 1e+3 -> 0E+3 Inexact Rounded
quax370 quantize -7.8 1e-5 -> -7.80000
quax371 quantize -7.8 1e-4 -> -7.8000
quax372 quantize -7.8 1e-3 -> -7.800
quax373 quantize -7.8 1e-2 -> -7.80
quax374 quantize -7.8 1e-1 -> -7.8
quax375 quantize -7.8 1e0 -> -8 Inexact Rounded
quax376 quantize -7.8 1e+1 -> -1E+1 Inexact Rounded
quax377 quantize -7.8 1e+2 -> -0E+2 Inexact Rounded
quax378 quantize -7.8 1e+3 -> -0E+3 Inexact Rounded
-- some individuals
precision: 9
quax380 quantize 352364.506 1e-2 -> 352364.51 Inexact Rounded
quax381 quantize 3523645.06 1e-2 -> 3523645.06
quax382 quantize 35236450.6 1e-2 -> NaN Invalid_operation
quax383 quantize 352364506 1e-2 -> NaN Invalid_operation
quax384 quantize -352364.506 1e-2 -> -352364.51 Inexact Rounded
quax385 quantize -3523645.06 1e-2 -> -3523645.06
quax386 quantize -35236450.6 1e-2 -> NaN Invalid_operation
quax387 quantize -352364506 1e-2 -> NaN Invalid_operation
rounding: down
quax389 quantize 35236450.6 1e-2 -> NaN Invalid_operation
-- ? should that one instead have been:
-- quax389 quantize 35236450.6 1e-2 -> NaN Invalid_operation
rounding: half_up
-- and a few more from e-mail discussions
precision: 7
quax391 quantize 12.34567 1e-3 -> 12.346 Inexact Rounded
quax392 quantize 123.4567 1e-3 -> 123.457 Inexact Rounded
quax393 quantize 1234.567 1e-3 -> 1234.567
quax394 quantize 12345.67 1e-3 -> NaN Invalid_operation
quax395 quantize 123456.7 1e-3 -> NaN Invalid_operation
quax396 quantize 1234567. 1e-3 -> NaN Invalid_operation
-- some 9999 round-up cases
precision: 9
quax400 quantize 9.999 1e-5 -> 9.99900
quax401 quantize 9.999 1e-4 -> 9.9990
quax402 quantize 9.999 1e-3 -> 9.999
quax403 quantize 9.999 1e-2 -> 10.00 Inexact Rounded
quax404 quantize 9.999 1e-1 -> 10.0 Inexact Rounded
quax405 quantize 9.999 1e0 -> 10 Inexact Rounded
quax406 quantize 9.999 1e1 -> 1E+1 Inexact Rounded
quax407 quantize 9.999 1e2 -> 0E+2 Inexact Rounded
quax410 quantize 0.999 1e-5 -> 0.99900
quax411 quantize 0.999 1e-4 -> 0.9990
quax412 quantize 0.999 1e-3 -> 0.999
quax413 quantize 0.999 1e-2 -> 1.00 Inexact Rounded
quax414 quantize 0.999 1e-1 -> 1.0 Inexact Rounded
quax415 quantize 0.999 1e0 -> 1 Inexact Rounded
quax416 quantize 0.999 1e1 -> 0E+1 Inexact Rounded
quax420 quantize 0.0999 1e-5 -> 0.09990
quax421 quantize 0.0999 1e-4 -> 0.0999
quax422 quantize 0.0999 1e-3 -> 0.100 Inexact Rounded
quax423 quantize 0.0999 1e-2 -> 0.10 Inexact Rounded
quax424 quantize 0.0999 1e-1 -> 0.1 Inexact Rounded
quax425 quantize 0.0999 1e0 -> 0 Inexact Rounded
quax426 quantize 0.0999 1e1 -> 0E+1 Inexact Rounded
quax430 quantize 0.00999 1e-5 -> 0.00999
quax431 quantize 0.00999 1e-4 -> 0.0100 Inexact Rounded
quax432 quantize 0.00999 1e-3 -> 0.010 Inexact Rounded
quax433 quantize 0.00999 1e-2 -> 0.01 Inexact Rounded
quax434 quantize 0.00999 1e-1 -> 0.0 Inexact Rounded
quax435 quantize 0.00999 1e0 -> 0 Inexact Rounded
quax436 quantize 0.00999 1e1 -> 0E+1 Inexact Rounded
quax440 quantize 0.000999 1e-5 -> 0.00100 Inexact Rounded
quax441 quantize 0.000999 1e-4 -> 0.0010 Inexact Rounded
quax442 quantize 0.000999 1e-3 -> 0.001 Inexact Rounded
quax443 quantize 0.000999 1e-2 -> 0.00 Inexact Rounded
quax444 quantize 0.000999 1e-1 -> 0.0 Inexact Rounded
quax445 quantize 0.000999 1e0 -> 0 Inexact Rounded
quax446 quantize 0.000999 1e1 -> 0E+1 Inexact Rounded
precision: 8
quax449 quantize 9.999E-15 1e-23 -> NaN Invalid_operation
quax450 quantize 9.999E-15 1e-22 -> 9.9990000E-15
quax451 quantize 9.999E-15 1e-21 -> 9.999000E-15
quax452 quantize 9.999E-15 1e-20 -> 9.99900E-15
quax453 quantize 9.999E-15 1e-19 -> 9.9990E-15
quax454 quantize 9.999E-15 1e-18 -> 9.999E-15
quax455 quantize 9.999E-15 1e-17 -> 1.000E-14 Inexact Rounded
quax456 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded
quax457 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded
quax458 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded
quax459 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded
quax460 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded
quax461 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded
quax462 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded
quax463 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded
quax464 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded
quax465 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded
quax466 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded
quax467 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded
quax468 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded
quax469 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded
quax470 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded
quax471 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded
quax472 quantize 9.999E-15 1e0 -> 0 Inexact Rounded
quax473 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded
Merged the decimal-branch (revisions 54886 to 58140). Decimal is now fully updated to the latests Decimal Specification (v1.66) and the latests test cases (v2.56). Thanks to Mark Dickinson for all his help during this process.
2007-09-13 15:13:15 -03:00
precision: 7
quax900 quantize 9.999E-15 1e-22 -> NaN Invalid_operation
quax901 quantize 9.999E-15 1e-21 -> 9.999000E-15
quax902 quantize 9.999E-15 1e-20 -> 9.99900E-15
quax903 quantize 9.999E-15 1e-19 -> 9.9990E-15
quax904 quantize 9.999E-15 1e-18 -> 9.999E-15
quax905 quantize 9.999E-15 1e-17 -> 1.000E-14 Inexact Rounded
quax906 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded
quax907 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded
quax908 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded
quax909 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded
quax910 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded
quax911 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded
quax912 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded
quax913 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded
quax914 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded
quax915 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded
quax916 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded
quax917 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded
quax918 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded
quax919 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded
quax920 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded
quax921 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded
quax922 quantize 9.999E-15 1e0 -> 0 Inexact Rounded
quax923 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded
precision: 6
quax930 quantize 9.999E-15 1e-22 -> NaN Invalid_operation
quax931 quantize 9.999E-15 1e-21 -> NaN Invalid_operation
quax932 quantize 9.999E-15 1e-20 -> 9.99900E-15
quax933 quantize 9.999E-15 1e-19 -> 9.9990E-15
quax934 quantize 9.999E-15 1e-18 -> 9.999E-15
quax935 quantize 9.999E-15 1e-17 -> 1.000E-14 Inexact Rounded
quax936 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded
quax937 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded
quax938 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded
quax939 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded
quax940 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded
quax941 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded
quax942 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded
quax943 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded
quax944 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded
quax945 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded
quax946 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded
quax947 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded
quax948 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded
quax949 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded
quax950 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded
quax951 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded
quax952 quantize 9.999E-15 1e0 -> 0 Inexact Rounded
quax953 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded
precision: 3
quax960 quantize 9.999E-15 1e-22 -> NaN Invalid_operation
quax961 quantize 9.999E-15 1e-21 -> NaN Invalid_operation
quax962 quantize 9.999E-15 1e-20 -> NaN Invalid_operation
quax963 quantize 9.999E-15 1e-19 -> NaN Invalid_operation
quax964 quantize 9.999E-15 1e-18 -> NaN Invalid_operation
quax965 quantize 9.999E-15 1e-17 -> NaN Invalid_operation
quax966 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded
quax967 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded
quax968 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded
quax969 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded
quax970 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded
quax971 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded
quax972 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded
quax973 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded
quax974 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded
quax975 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded
quax976 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded
quax977 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded
quax978 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded
quax979 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded
quax980 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded
quax981 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded
quax982 quantize 9.999E-15 1e0 -> 0 Inexact Rounded
quax983 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded
-- Fung Lee's case & similar
precision: 3
quax1001 quantize 0.000 0.001 -> 0.000
quax1002 quantize 0.001 0.001 -> 0.001
quax1003 quantize 0.0012 0.001 -> 0.001 Inexact Rounded
quax1004 quantize 0.0018 0.001 -> 0.002 Inexact Rounded
quax1005 quantize 0.501 0.001 -> 0.501
quax1006 quantize 0.5012 0.001 -> 0.501 Inexact Rounded
quax1007 quantize 0.5018 0.001 -> 0.502 Inexact Rounded
quax1008 quantize 0.999 0.001 -> 0.999
quax1009 quantize 0.9992 0.001 -> 0.999 Inexact Rounded
quax1010 quantize 0.9998 0.001 -> NaN Invalid_operation
quax1011 quantize 1.0001 0.001 -> NaN Invalid_operation
quax1012 quantize 1.0051 0.001 -> NaN Invalid_operation
quax1013 quantize 1.0551 0.001 -> NaN Invalid_operation
quax1014 quantize 1.5551 0.001 -> NaN Invalid_operation
quax1015 quantize 1.9999 0.001 -> NaN Invalid_operation
Move Decimal from the sandbox into production.
2004-07-01 08:01:35 -03:00
-- long operand checks [rhs checks removed]
maxexponent: 999
minexponent: -999
precision: 9
quax481 quantize 12345678000 1e+3 -> 1.2345678E+10 Rounded
quax482 quantize 1234567800 1e+1 -> 1.23456780E+9 Rounded
quax483 quantize 1234567890 1e+1 -> 1.23456789E+9 Rounded
quax484 quantize 1234567891 1e+1 -> 1.23456789E+9 Inexact Rounded
quax485 quantize 12345678901 1e+2 -> 1.23456789E+10 Inexact Rounded
quax486 quantize 1234567896 1e+1 -> 1.23456790E+9 Inexact Rounded
-- a potential double-round
quax487 quantize 1234.987643 1e-4 -> 1234.9876 Inexact Rounded
quax488 quantize 1234.987647 1e-4 -> 1234.9876 Inexact Rounded
precision: 15
quax491 quantize 12345678000 1e+3 -> 1.2345678E+10 Rounded
quax492 quantize 1234567800 1e+1 -> 1.23456780E+9 Rounded
quax493 quantize 1234567890 1e+1 -> 1.23456789E+9 Rounded
quax494 quantize 1234567891 1e+1 -> 1.23456789E+9 Inexact Rounded
quax495 quantize 12345678901 1e+2 -> 1.23456789E+10 Inexact Rounded
quax496 quantize 1234567896 1e+1 -> 1.23456790E+9 Inexact Rounded
quax497 quantize 1234.987643 1e-4 -> 1234.9876 Inexact Rounded
quax498 quantize 1234.987647 1e-4 -> 1234.9876 Inexact Rounded
-- Zeros
quax500 quantize 0 1e1 -> 0E+1
quax501 quantize 0 1e0 -> 0
quax502 quantize 0 1e-1 -> 0.0
quax503 quantize 0.0 1e-1 -> 0.0
quax504 quantize 0.0 1e0 -> 0
quax505 quantize 0.0 1e+1 -> 0E+1
quax506 quantize 0E+1 1e-1 -> 0.0
quax507 quantize 0E+1 1e0 -> 0
quax508 quantize 0E+1 1e+1 -> 0E+1
quax509 quantize -0 1e1 -> -0E+1
quax510 quantize -0 1e0 -> -0
quax511 quantize -0 1e-1 -> -0.0
quax512 quantize -0.0 1e-1 -> -0.0
quax513 quantize -0.0 1e0 -> -0
quax514 quantize -0.0 1e+1 -> -0E+1
quax515 quantize -0E+1 1e-1 -> -0.0
quax516 quantize -0E+1 1e0 -> -0
quax517 quantize -0E+1 1e+1 -> -0E+1
-- Suspicious RHS values
maxexponent: 999999999
minexponent: -999999999
precision: 15
quax520 quantize 1.234 1e999999000 -> 0E+999999000 Inexact Rounded
quax521 quantize 123.456 1e999999000 -> 0E+999999000 Inexact Rounded
quax522 quantize 1.234 1e999999999 -> 0E+999999999 Inexact Rounded
quax523 quantize 123.456 1e999999999 -> 0E+999999999 Inexact Rounded
quax524 quantize 123.456 1e1000000000 -> NaN Invalid_operation
quax525 quantize 123.456 1e12345678903 -> NaN Invalid_operation
-- next four are "won't fit" overflows
quax526 quantize 1.234 1e-999999000 -> NaN Invalid_operation
quax527 quantize 123.456 1e-999999000 -> NaN Invalid_operation
quax528 quantize 1.234 1e-999999999 -> NaN Invalid_operation
quax529 quantize 123.456 1e-999999999 -> NaN Invalid_operation
quax530 quantize 123.456 1e-1000000014 -> NaN Invalid_operation
quax531 quantize 123.456 1e-12345678903 -> NaN Invalid_operation
maxexponent: 999
minexponent: -999
precision: 15
quax532 quantize 1.234E+999 1e999 -> 1E+999 Inexact Rounded
quax533 quantize 1.234E+998 1e999 -> 0E+999 Inexact Rounded
quax534 quantize 1.234 1e999 -> 0E+999 Inexact Rounded
quax535 quantize 1.234 1e1000 -> NaN Invalid_operation
quax536 quantize 1.234 1e5000 -> NaN Invalid_operation
quax537 quantize 0 1e-999 -> 0E-999
-- next two are "won't fit" overflows
quax538 quantize 1.234 1e-999 -> NaN Invalid_operation
quax539 quantize 1.234 1e-1000 -> NaN Invalid_operation
quax540 quantize 1.234 1e-5000 -> NaN Invalid_operation
-- [more below]
-- check bounds (lhs maybe out of range for destination, etc.)
precision: 7
quax541 quantize 1E+999 1e+999 -> 1E+999
quax542 quantize 1E+1000 1e+999 -> NaN Invalid_operation
quax543 quantize 1E+999 1e+1000 -> NaN Invalid_operation
quax544 quantize 1E-999 1e-999 -> 1E-999
quax545 quantize 1E-1000 1e-999 -> 0E-999 Inexact Rounded
quax546 quantize 1E-999 1e-1000 -> 1.0E-999
quax547 quantize 1E-1005 1e-999 -> 0E-999 Inexact Rounded
quax548 quantize 1E-1006 1e-999 -> 0E-999 Inexact Rounded
quax549 quantize 1E-1007 1e-999 -> 0E-999 Inexact Rounded
quax550 quantize 1E-998 1e-1005 -> NaN Invalid_operation -- won't fit
quax551 quantize 1E-999 1e-1005 -> 1.000000E-999
quax552 quantize 1E-1000 1e-1005 -> 1.00000E-1000 Subnormal
quax553 quantize 1E-999 1e-1006 -> NaN Invalid_operation
quax554 quantize 1E-999 1e-1007 -> NaN Invalid_operation
-- related subnormal rounding
quax555 quantize 1.666666E-999 1e-1005 -> 1.666666E-999
quax556 quantize 1.666666E-1000 1e-1005 -> 1.66667E-1000 Subnormal Inexact Rounded
quax557 quantize 1.666666E-1001 1e-1005 -> 1.6667E-1001 Subnormal Inexact Rounded
quax558 quantize 1.666666E-1002 1e-1005 -> 1.667E-1002 Subnormal Inexact Rounded
quax559 quantize 1.666666E-1003 1e-1005 -> 1.67E-1003 Subnormal Inexact Rounded
quax560 quantize 1.666666E-1004 1e-1005 -> 1.7E-1004 Subnormal Inexact Rounded
quax561 quantize 1.666666E-1005 1e-1005 -> 2E-1005 Subnormal Inexact Rounded
quax562 quantize 1.666666E-1006 1e-1005 -> 0E-1005 Inexact Rounded
quax563 quantize 1.666666E-1007 1e-1005 -> 0E-1005 Inexact Rounded
-- Specials
quax580 quantize Inf -Inf -> Infinity
quax581 quantize Inf 1e-1000 -> NaN Invalid_operation
quax582 quantize Inf 1e-1 -> NaN Invalid_operation
quax583 quantize Inf 1e0 -> NaN Invalid_operation
quax584 quantize Inf 1e1 -> NaN Invalid_operation
quax585 quantize Inf 1e1000 -> NaN Invalid_operation
quax586 quantize Inf Inf -> Infinity
quax587 quantize -1000 Inf -> NaN Invalid_operation
quax588 quantize -Inf Inf -> -Infinity
quax589 quantize -1 Inf -> NaN Invalid_operation
quax590 quantize 0 Inf -> NaN Invalid_operation
quax591 quantize 1 Inf -> NaN Invalid_operation
quax592 quantize 1000 Inf -> NaN Invalid_operation
quax593 quantize Inf Inf -> Infinity
quax594 quantize Inf 1e-0 -> NaN Invalid_operation
quax595 quantize -0 Inf -> NaN Invalid_operation
quax600 quantize -Inf -Inf -> -Infinity
quax601 quantize -Inf 1e-1000 -> NaN Invalid_operation
quax602 quantize -Inf 1e-1 -> NaN Invalid_operation
quax603 quantize -Inf 1e0 -> NaN Invalid_operation
quax604 quantize -Inf 1e1 -> NaN Invalid_operation
quax605 quantize -Inf 1e1000 -> NaN Invalid_operation
quax606 quantize -Inf Inf -> -Infinity
quax607 quantize -1000 Inf -> NaN Invalid_operation
quax608 quantize -Inf -Inf -> -Infinity
quax609 quantize -1 -Inf -> NaN Invalid_operation
quax610 quantize 0 -Inf -> NaN Invalid_operation
quax611 quantize 1 -Inf -> NaN Invalid_operation
quax612 quantize 1000 -Inf -> NaN Invalid_operation
quax613 quantize Inf -Inf -> Infinity
quax614 quantize -Inf 1e-0 -> NaN Invalid_operation
quax615 quantize -0 -Inf -> NaN Invalid_operation
quax621 quantize NaN -Inf -> NaN
quax622 quantize NaN 1e-1000 -> NaN
quax623 quantize NaN 1e-1 -> NaN
quax624 quantize NaN 1e0 -> NaN
quax625 quantize NaN 1e1 -> NaN
quax626 quantize NaN 1e1000 -> NaN
quax627 quantize NaN Inf -> NaN
quax628 quantize NaN NaN -> NaN
quax629 quantize -Inf NaN -> NaN
quax630 quantize -1000 NaN -> NaN
quax631 quantize -1 NaN -> NaN
quax632 quantize 0 NaN -> NaN
quax633 quantize 1 NaN -> NaN
quax634 quantize 1000 NaN -> NaN
quax635 quantize Inf NaN -> NaN
quax636 quantize NaN 1e-0 -> NaN
quax637 quantize -0 NaN -> NaN
quax641 quantize sNaN -Inf -> NaN Invalid_operation
quax642 quantize sNaN 1e-1000 -> NaN Invalid_operation
quax643 quantize sNaN 1e-1 -> NaN Invalid_operation
quax644 quantize sNaN 1e0 -> NaN Invalid_operation
quax645 quantize sNaN 1e1 -> NaN Invalid_operation
quax646 quantize sNaN 1e1000 -> NaN Invalid_operation
quax647 quantize sNaN NaN -> NaN Invalid_operation
quax648 quantize sNaN sNaN -> NaN Invalid_operation
quax649 quantize NaN sNaN -> NaN Invalid_operation
quax650 quantize -Inf sNaN -> NaN Invalid_operation
quax651 quantize -1000 sNaN -> NaN Invalid_operation
quax652 quantize -1 sNaN -> NaN Invalid_operation
quax653 quantize 0 sNaN -> NaN Invalid_operation
quax654 quantize 1 sNaN -> NaN Invalid_operation
quax655 quantize 1000 sNaN -> NaN Invalid_operation
quax656 quantize Inf sNaN -> NaN Invalid_operation
quax657 quantize NaN sNaN -> NaN Invalid_operation
quax658 quantize sNaN 1e-0 -> NaN Invalid_operation
quax659 quantize -0 sNaN -> NaN Invalid_operation
-- propagating NaNs
quax661 quantize NaN9 -Inf -> NaN9
quax662 quantize NaN8 919 -> NaN8
quax663 quantize NaN71 Inf -> NaN71
quax664 quantize NaN6 NaN5 -> NaN6
quax665 quantize -Inf NaN4 -> NaN4
quax666 quantize -919 NaN31 -> NaN31
quax667 quantize Inf NaN2 -> NaN2
quax671 quantize sNaN99 -Inf -> NaN99 Invalid_operation
quax672 quantize sNaN98 -11 -> NaN98 Invalid_operation
quax673 quantize sNaN97 NaN -> NaN97 Invalid_operation
quax674 quantize sNaN16 sNaN94 -> NaN16 Invalid_operation
quax675 quantize NaN95 sNaN93 -> NaN93 Invalid_operation
quax676 quantize -Inf sNaN92 -> NaN92 Invalid_operation
quax677 quantize 088 sNaN91 -> NaN91 Invalid_operation
quax678 quantize Inf sNaN90 -> NaN90 Invalid_operation
quax679 quantize NaN sNaN88 -> NaN88 Invalid_operation
quax681 quantize -NaN9 -Inf -> -NaN9
quax682 quantize -NaN8 919 -> -NaN8
quax683 quantize -NaN71 Inf -> -NaN71
quax684 quantize -NaN6 -NaN5 -> -NaN6
quax685 quantize -Inf -NaN4 -> -NaN4
quax686 quantize -919 -NaN31 -> -NaN31
quax687 quantize Inf -NaN2 -> -NaN2
quax691 quantize -sNaN99 -Inf -> -NaN99 Invalid_operation
quax692 quantize -sNaN98 -11 -> -NaN98 Invalid_operation
quax693 quantize -sNaN97 NaN -> -NaN97 Invalid_operation
quax694 quantize -sNaN16 sNaN94 -> -NaN16 Invalid_operation
quax695 quantize -NaN95 -sNaN93 -> -NaN93 Invalid_operation
quax696 quantize -Inf -sNaN92 -> -NaN92 Invalid_operation
quax697 quantize 088 -sNaN91 -> -NaN91 Invalid_operation
quax698 quantize Inf -sNaN90 -> -NaN90 Invalid_operation
quax699 quantize NaN -sNaN88 -> -NaN88 Invalid_operation
-- subnormals and underflow
precision: 4
maxexponent: 999
minexponent: -999
quax710 quantize 1.00E-999 1e-999 -> 1E-999 Rounded
quax711 quantize 0.1E-999 2e-1000 -> 1E-1000 Subnormal
quax712 quantize 0.10E-999 3e-1000 -> 1E-1000 Subnormal Rounded
quax713 quantize 0.100E-999 4e-1000 -> 1E-1000 Subnormal Rounded
quax714 quantize 0.01E-999 5e-1001 -> 1E-1001 Subnormal
-- next is rounded to Emin
quax715 quantize 0.999E-999 1e-999 -> 1E-999 Inexact Rounded
quax716 quantize 0.099E-999 10e-1000 -> 1E-1000 Inexact Rounded Subnormal
quax717 quantize 0.009E-999 1e-1001 -> 1E-1001 Inexact Rounded Subnormal
quax718 quantize 0.001E-999 1e-1001 -> 0E-1001 Inexact Rounded
quax719 quantize 0.0009E-999 1e-1001 -> 0E-1001 Inexact Rounded
quax720 quantize 0.0001E-999 1e-1001 -> 0E-1001 Inexact Rounded
quax730 quantize -1.00E-999 1e-999 -> -1E-999 Rounded
quax731 quantize -0.1E-999 1e-999 -> -0E-999 Rounded Inexact
quax732 quantize -0.10E-999 1e-999 -> -0E-999 Rounded Inexact
quax733 quantize -0.100E-999 1e-999 -> -0E-999 Rounded Inexact
quax734 quantize -0.01E-999 1e-999 -> -0E-999 Inexact Rounded
-- next is rounded to Emin
quax735 quantize -0.999E-999 90e-999 -> -1E-999 Inexact Rounded
quax736 quantize -0.099E-999 -1e-999 -> -0E-999 Inexact Rounded
quax737 quantize -0.009E-999 -1e-999 -> -0E-999 Inexact Rounded
quax738 quantize -0.001E-999 -0e-999 -> -0E-999 Inexact Rounded
quax739 quantize -0.0001E-999 0e-999 -> -0E-999 Inexact Rounded
quax740 quantize -1.00E-999 1e-1000 -> -1.0E-999 Rounded
quax741 quantize -0.1E-999 1e-1000 -> -1E-1000 Subnormal
quax742 quantize -0.10E-999 1e-1000 -> -1E-1000 Subnormal Rounded
quax743 quantize -0.100E-999 1e-1000 -> -1E-1000 Subnormal Rounded
quax744 quantize -0.01E-999 1e-1000 -> -0E-1000 Inexact Rounded
-- next is rounded to Emin
quax745 quantize -0.999E-999 1e-1000 -> -1.0E-999 Inexact Rounded
quax746 quantize -0.099E-999 1e-1000 -> -1E-1000 Inexact Rounded Subnormal
quax747 quantize -0.009E-999 1e-1000 -> -0E-1000 Inexact Rounded
quax748 quantize -0.001E-999 1e-1000 -> -0E-1000 Inexact Rounded
quax749 quantize -0.0001E-999 1e-1000 -> -0E-1000 Inexact Rounded
quax750 quantize -1.00E-999 1e-1001 -> -1.00E-999
quax751 quantize -0.1E-999 1e-1001 -> -1.0E-1000 Subnormal
quax752 quantize -0.10E-999 1e-1001 -> -1.0E-1000 Subnormal
quax753 quantize -0.100E-999 1e-1001 -> -1.0E-1000 Subnormal Rounded
quax754 quantize -0.01E-999 1e-1001 -> -1E-1001 Subnormal
-- next is rounded to Emin
quax755 quantize -0.999E-999 1e-1001 -> -1.00E-999 Inexact Rounded
quax756 quantize -0.099E-999 1e-1001 -> -1.0E-1000 Inexact Rounded Subnormal
quax757 quantize -0.009E-999 1e-1001 -> -1E-1001 Inexact Rounded Subnormal
quax758 quantize -0.001E-999 1e-1001 -> -0E-1001 Inexact Rounded
quax759 quantize -0.0001E-999 1e-1001 -> -0E-1001 Inexact Rounded
quax760 quantize -1.00E-999 1e-1002 -> -1.000E-999
quax761 quantize -0.1E-999 1e-1002 -> -1.00E-1000 Subnormal
quax762 quantize -0.10E-999 1e-1002 -> -1.00E-1000 Subnormal
quax763 quantize -0.100E-999 1e-1002 -> -1.00E-1000 Subnormal
quax764 quantize -0.01E-999 1e-1002 -> -1.0E-1001 Subnormal
quax765 quantize -0.999E-999 1e-1002 -> -9.99E-1000 Subnormal
quax766 quantize -0.099E-999 1e-1002 -> -9.9E-1001 Subnormal
quax767 quantize -0.009E-999 1e-1002 -> -9E-1002 Subnormal
quax768 quantize -0.001E-999 1e-1002 -> -1E-1002 Subnormal
quax769 quantize -0.0001E-999 1e-1002 -> -0E-1002 Inexact Rounded
-- rhs must be no less than Etiny
quax770 quantize -1.00E-999 1e-1003 -> NaN Invalid_operation
quax771 quantize -0.1E-999 1e-1003 -> NaN Invalid_operation
quax772 quantize -0.10E-999 1e-1003 -> NaN Invalid_operation
quax773 quantize -0.100E-999 1e-1003 -> NaN Invalid_operation
quax774 quantize -0.01E-999 1e-1003 -> NaN Invalid_operation
quax775 quantize -0.999E-999 1e-1003 -> NaN Invalid_operation
quax776 quantize -0.099E-999 1e-1003 -> NaN Invalid_operation
quax777 quantize -0.009E-999 1e-1003 -> NaN Invalid_operation
quax778 quantize -0.001E-999 1e-1003 -> NaN Invalid_operation
quax779 quantize -0.0001E-999 1e-1003 -> NaN Invalid_operation
quax780 quantize -0.0001E-999 1e-1004 -> NaN Invalid_operation
precision: 9
maxExponent: 999999999
minexponent: -999999999
-- some extremes derived from Rescale testcases
quax801 quantize 0 1e1000000000 -> NaN Invalid_operation
quax802 quantize 0 1e-1000000000 -> 0E-1000000000
quax803 quantize 0 1e2000000000 -> NaN Invalid_operation
quax804 quantize 0 1e-2000000000 -> NaN Invalid_operation
quax805 quantize 0 1e3000000000 -> NaN Invalid_operation
quax806 quantize 0 1e-3000000000 -> NaN Invalid_operation
quax807 quantize 0 1e4000000000 -> NaN Invalid_operation
quax808 quantize 0 1e-4000000000 -> NaN Invalid_operation
quax809 quantize 0 1e5000000000 -> NaN Invalid_operation
quax810 quantize 0 1e-5000000000 -> NaN Invalid_operation
quax811 quantize 0 1e6000000000 -> NaN Invalid_operation
quax812 quantize 0 1e-6000000000 -> NaN Invalid_operation
quax813 quantize 0 1e7000000000 -> NaN Invalid_operation
quax814 quantize 0 1e-7000000000 -> NaN Invalid_operation
quax815 quantize 0 1e8000000000 -> NaN Invalid_operation
quax816 quantize 0 1e-8000000000 -> NaN Invalid_operation
quax817 quantize 0 1e9000000000 -> NaN Invalid_operation
quax818 quantize 0 1e-9000000000 -> NaN Invalid_operation
quax819 quantize 0 1e9999999999 -> NaN Invalid_operation
quax820 quantize 0 1e-9999999999 -> NaN Invalid_operation
quax821 quantize 0 1e10000000000 -> NaN Invalid_operation
quax822 quantize 0 1e-10000000000 -> NaN Invalid_operation
quax843 quantize 0 1e999999999 -> 0E+999999999
quax844 quantize 0 1e1000000000 -> NaN Invalid_operation
quax845 quantize 0 1e-999999999 -> 0E-999999999
quax846 quantize 0 1e-1000000000 -> 0E-1000000000
quax847 quantize 0 1e-1000000001 -> 0E-1000000001
quax848 quantize 0 1e-1000000002 -> 0E-1000000002
quax849 quantize 0 1e-1000000003 -> 0E-1000000003
quax850 quantize 0 1e-1000000004 -> 0E-1000000004
quax851 quantize 0 1e-1000000005 -> 0E-1000000005
quax852 quantize 0 1e-1000000006 -> 0E-1000000006
quax853 quantize 0 1e-1000000007 -> 0E-1000000007
quax854 quantize 0 1e-1000000008 -> NaN Invalid_operation
quax861 quantize 1 1e+2147483649 -> NaN Invalid_operation
quax862 quantize 1 1e+2147483648 -> NaN Invalid_operation
quax863 quantize 1 1e+2147483647 -> NaN Invalid_operation
quax864 quantize 1 1e-2147483647 -> NaN Invalid_operation
quax865 quantize 1 1e-2147483648 -> NaN Invalid_operation
quax866 quantize 1 1e-2147483649 -> NaN Invalid_operation
Merged the decimal-branch (revisions 54886 to 58140). Decimal is now fully updated to the latests Decimal Specification (v1.66) and the latests test cases (v2.56). Thanks to Mark Dickinson for all his help during this process.
2007-09-13 15:13:15 -03:00
-- More from Fung Lee
precision: 16
rounding: half_up
maxExponent: 384
minExponent: -383
quax1021 quantize 8.666666666666000E+384 1.000000000000000E+384 -> 8.666666666666000E+384
quax1022 quantize 64#8.666666666666000E+384 64#1.000000000000000E+384 -> 8.666666666666000E+384
quax1023 quantize 64#8.666666666666000E+384 128#1.000000000000000E+384 -> 8.666666666666000E+384
quax1024 quantize 64#8.666666666666000E+384 64#1E+384 -> 8.666666666666000E+384
quax1025 quantize 64#8.666666666666000E+384 64#1E+384 -> 64#8.666666666666000E+384
quax1026 quantize 64#8.666666666666000E+384 128#1E+384 -> 64#9E+384 Inexact Rounded Clamped
quax1027 quantize 64#8.666666666666000E+323 64#1E+31 -> NaN Invalid_operation
quax1028 quantize 64#8.666666666666000E+323 128#1E+31 -> NaN Invalid_operation
quax1029 quantize 64#8.66666666E+3 128#1E+10 -> 64#0E10 Inexact Rounded
quax1030 quantize 8.66666666E+3 1E+3 -> 9E+3 Inexact Rounded
-- Int and uInt32 edge values for testing conversions
quax1040 quantize -2147483646 0 -> -2147483646
quax1041 quantize -2147483647 0 -> -2147483647
quax1042 quantize -2147483648 0 -> -2147483648
quax1043 quantize -2147483649 0 -> -2147483649
quax1044 quantize 2147483646 0 -> 2147483646
quax1045 quantize 2147483647 0 -> 2147483647
quax1046 quantize 2147483648 0 -> 2147483648
quax1047 quantize 2147483649 0 -> 2147483649
quax1048 quantize 4294967294 0 -> 4294967294
quax1049 quantize 4294967295 0 -> 4294967295
quax1050 quantize 4294967296 0 -> 4294967296
quax1051 quantize 4294967297 0 -> 4294967297
-- and powers of ten for same
quax1101 quantize 5000000000 0 -> 5000000000
quax1102 quantize 4000000000 0 -> 4000000000
quax1103 quantize 2000000000 0 -> 2000000000
quax1104 quantize 1000000000 0 -> 1000000000
quax1105 quantize 0100000000 0 -> 100000000
quax1106 quantize 0010000000 0 -> 10000000
quax1107 quantize 0001000000 0 -> 1000000
quax1108 quantize 0000100000 0 -> 100000
quax1109 quantize 0000010000 0 -> 10000
quax1110 quantize 0000001000 0 -> 1000
quax1111 quantize 0000000100 0 -> 100
quax1112 quantize 0000000010 0 -> 10
quax1113 quantize 0000000001 0 -> 1
quax1114 quantize 0000000000 0 -> 0
-- and powers of ten for same
quax1121 quantize -5000000000 0 -> -5000000000
quax1122 quantize -4000000000 0 -> -4000000000
quax1123 quantize -2000000000 0 -> -2000000000
quax1124 quantize -1000000000 0 -> -1000000000
quax1125 quantize -0100000000 0 -> -100000000
quax1126 quantize -0010000000 0 -> -10000000
quax1127 quantize -0001000000 0 -> -1000000
quax1128 quantize -0000100000 0 -> -100000
quax1129 quantize -0000010000 0 -> -10000
quax1130 quantize -0000001000 0 -> -1000
quax1131 quantize -0000000100 0 -> -100
quax1132 quantize -0000000010 0 -> -10
quax1133 quantize -0000000001 0 -> -1
quax1134 quantize -0000000000 0 -> -0
-- Some miscellany
precision: 34
rounding: half_up
maxExponent: 6144
minExponent: -6143
-- 1 2 3
-- 1 234567890123456789012345678901234
quax0a1 quantize 8.555555555555555555555555555555555E+6143 1E+6143 -> 9E+6143 Inexact Rounded
quax0a2 quantize 128#8.555555555555555555555555555555555E+6143 128#1E+6143 -> 8.55555555555555555555555555555556E+6143 Rounded Inexact
quax0a3 quantize 128#8.555555555555555555555555555555555E+6144 128#1E+6144 -> 8.555555555555555555555555555555555E+6144
-- payload decapitate
precision: 5
quax62100 quantize 11 -sNaN1234567890 -> -NaN67890 Invalid_operation
Move Decimal from the sandbox into production.
2004-07-01 08:01:35 -03:00
-- Null tests
Merged the decimal-branch (revisions 54886 to 58140). Decimal is now fully updated to the latests Decimal Specification (v1.66) and the latests test cases (v2.56). Thanks to Mark Dickinson for all his help during this process.
2007-09-13 15:13:15 -03:00
quax998 quantize 10 # -> NaN Invalid_operation
quax999 quantize # 1e10 -> NaN Invalid_operation