This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
This issue arises from an XSLT test case bug: http://www.w3.org/Member/bugzilla/show_bug.cgi?id=624 (member-only) but it is not specific to XSLT. The question is, what exactly does it mean in F+O 6.4.5 when it says "If two such values are equally near (e.g. if the fractional part in $arg is exactly .500...), returns the one whose least significant digit is even. " The case in question is round-half-to-even(xs:float(150.0150e0), 2) At first sight 150.015 is equally close to 150.01 and 150.02, so it should be rounded up. However, xs:float(150.015e0) actually returns a number whose precise value is 150.0149993896484375, which would suggest rounding down. (This is what I think should be done, and what the published test results do). However, Joanne has argued: xs:float(150.01) = 150.00999450683594 xs:float(150.015) = 150.01499938964844 xs:float(150.02) = 150.02000427246094 and therefore 150.015 is equally close to both. But I can't see any justification for this line of reasoning in the spec. The spec says we should compare with "a multiple of ten to the power of minus $precision.", not with an xs:float approximation to this.
I'm not sure what to do abt this. I understand both Mike and Joanne's arguments. The summary of the function currently says "The value returned is the nearest (that is, numerically closest) value to $arg that is a multiple of ten to the power of minus $precision. If two such values are equally near (e.g. if the fractional part in $arg is exactly .500...), returns the one whose least significant digit is even." Would it help to add something like "floating point numbers are compared using a decimal representation"? But then we need to say something abt the number of decimal places.
I'm not sure what do about it either. I'd like to be consistent with other parts of the spec, but we are very strict about numeric precision in some places, and very lax in other places. My own implementation of round-half-to-even(xs:float) does the arithmetic in double precision, which I think is reasonable but it's hard to argue that it's the only reasonable approach - you still get rounding errors, but they are smaller than with float. I think at this stage it's probably best to add a note saying that in the case of xs:float and xs:double, the arithmetic used to implement round-half-to-even may use floating point arithmetic, and that we don't prescribe the exact algorithm which means that rounding errors may occur and are implementation-dependent; the effect is that a number that is close to half-way between two powers of ten may round either up or down.
The working group accepted the following proposal, which was made by email at http://lists.w3.org/Archives/Member/w3c-xsl-query/2006Oct/0005.html (member-only): OPTION 2: define the function to work by converting the argument to decimal, and then converting the result back to the original type. Below is suggested text for that option. <proposal> fn:round-half-to-even($arg as numeric?) as numeric? fn:round-half-to-even($arg as numeric?, $precision as xs:integer) as numeric? Summary: The value returned is the nearest (that is, numerically closest) numeric to $arg that is a multiple of ten to the power of minus $precision. If two such values are equally near (e.g. if the fractional part in $arg is exactly .500...), the function returns the one whose least significant digit is even. If type of $arg is one of the four numeric types xs:float, xs:double, xs:decimal or xs:integer the type of the result is the same as the type of $arg. If the type of $arg is a type derived from one of the numeric types, the result is an instance of the base numeric type. The first signature of this function produces the same result as the second signature with $precision=0. For arguments of type xs:float and xs:double, if the argument is NaN, positive or negative zero, or positive or negative infinity, then the result is the same as the argument. In all other cases, the argument is cast to xs:decimal, the function is applied to this xs:decimal value, and the resulting xs:decimal value is cast back to xs:float or xs:double as appropriate to form the function result. If the resulting xs:decimal value is zero, then positive or negative zero is returned according to the sign of the original argument. Note that the process of casting to xs:decimal may result in an error [FOCA0001]. </proposal> This produces a well-defined and interoperable result provided that the implementation supports sufficient decimal range and precision: though not always the result that a naive user might expect. For the example in question round-half-to-even(xs:float(150.0150e0), 2) any implementation that supports the required 18 digits for xs:decimal will convert the argument to the xs:decimal 150.014999389.... which will then be rounded to the xs:decimal 150.01 which will be converted back to the xs:float whose exact value is 150.0099945068..... which will typically be displayed as 150.01 Michael Kay http://www.saxonica.com/
Closing bug because commenter has not objected to the resolution posted and more than two weeks have passed.