(In reply to comment #0)
> There is a mismatch between the terminology of XSD 1.1 part 2 regarding
> precisionDecimal, and the terminology of IEEE 754 - 2008. In particular, the
> term "precision" is used in a quite different way. In XSD 1.1 it is used to
> mean (essentially) the exponent when the significand is treated as an integer,
> while in IEEE it is used to mean the (maximum) number of digits in the
> significand.

It's expressed in terms of the exponent, but what it means is the number of digits to the right of the decimal  point in the significand.  (Both of us conflating the significand as a number and as a string of decimal digits, a numeral).  The number of digits to the right of the decimal point is often called precision as well--corresponds to "plus or minus xx", which, if you wish terminology to differentiate that from precision such as "plus or minus xx percent", is quite properly called arithmetic precision.  The "precision" that IEEE is describing is related in a complicated way to both of these two kinds of precision.

More comments to come.  (Surprise!  ;-)  )

>It [arithmeticPrecision] expressed in terms of the exponent, but what it means is the number of digits to the right of the decimal  point in the significand.

If that's what it meant, it could never be negative.

Michael Kay writes:

> If that's what it meant, it could never be negative.

Consider the case where a scientist wishes to express the number 1000 plus or minus 100.  In that case, it would be negative.  That is, of the 4 digits that appear before the decimal point, only two are significant.

Noah

Indeed. So what we're actually saying is that P is the position of the least significant digit relative to the decimal point, counting towards the right, with the digit immediately before the decimal point being numbered zero. The question is what to call this quantity. Currently the XSD spec has three names for it: scale, arithmeticPrecision, and precision. I'm advocating we should call it scale, because 

(a) we use that in the concrete syntax (in the facets minScale and maxScale), and we don't want to change the concrete syntax, and 

(b) because the IEEE spec uses the word "precision" to mean something completely different, which makes things very confusing for someone who is reading both specs, and will also make life very difficult for authors of other specifications (such as XPath functions and operators) that will need to refer to both.

(In reply to comment #0)

> See http://lists.w3.org/Archives/Member/w3c-xml-schema-ig/2010Dec/0006.html
> (member only) for further details.

Much of Mike's IG was discussion leading up to his recommendation for changes to the Datatypes spec.  I've added to that discussion (mostly adding, not correcting) in http://lists.w3.org/Archives/Member/w3c-xml-schema-ig/2010Dec/0007.html (also member only).  In that note I said I'd make my recommendations here.  Will try not to clutter my recommendations with a lot of discussion.  (Those who know me know that won't be easy.)

In addition, two months after Mike accepted the task of writing up a description of the relationship between IEEE 754 and precisionDecimal (culminatine in the IG messages noted above), I wrote a more technical description of that relationship and submitted it as a PDF attachment to http://lists.w3.org/Archives/Member/w3c-xml-schema-ig/2010May/0010.html .  I'll refer to this further on.

Mike recommended that we change the name of the arithmeticPrecision property of precisionDecimal values to 'scale'.  I'm comfortable with this (based on BigDecimal's use of "scale", not IEEE 754's) provided we have a note somewhere explaining what precisionDecimal has to do with precision (in the various generic senses, not just 754's specialized use of the word without an adjective).

I concur that we explain the relationship between IEEE "exponent" and our arithmeticPrecison/scale.  (Mike is wrong that our arithmeticPrecision/scale is IEEE's "exponent - precision + 1"; IEEE describes both and calls both "exponent".  It uses the variable "q" for ours and "e" for the other one, stating that q = e - p + 1 .  I believe we should, in fact, explain how all our representations of numerical values (sign, numericalValue, arithmeticPrecision/scale) are in one-to-one correspondence with those of IEEE (sign, significand, and exponent).

Mike recommended that we "Include in the specification a summary of the analysis above  describing the relationship of the precisionDecimal type to the abstract  decimal formats of IEEE."  I concur, provided we also include a summary describing the relationship to BigDecimal.

Mike wishes to avoid ever using the words "type' or "datatype" when mentioning IEEE "formats".  In fact, IEEE has simply used the word "format" to mean essentially the same thing as our "datatype".  I'm quite comfortable with always using "format" provided the equivalence of concept is noted somewhere.

Mike finally recommends that the relationship between our NaN and IEEE NaNs (various qNaNs and sNaNs) be explained.  I concur.  I included a paragraph (next to the last) in my May IG submission (see URL above) which can be a first cut at such a description.

Finally, and along that line, we should complete the description of the differences between precisionDecimal and IEEE 754 radix-10 formats by describing the differences in allowed lexical representations.  A first cut at such a description can also be found in the last several paragraphs of that May submission.  Indeed, in some ways these may be the most striking differences (other than
terminology) between our approach and that of IEEE.

[It's late at night.  Apologies if I've garbled anything in this comment.]

(In reply to comment #4)
> Indeed. So what we're actually saying is that P is the position of the least
> significant digit relative to the decimal point, counting towards the right,
> with the digit immediately before the decimal point being numbered zero. The
> question is what to call this quantity. Currently the XSD spec has three names
> for it: scale, arithmeticPrecision, and precision.

Well, we never call the property of values "scale", though one could infer that from the names of those facets.  The facets were created in Jan 2005; before then we just used additional clauses in the fractionDigits facet and added a "minFractionDigits" facet.  Some folks had the same problem you did with negative counts of fractionDigits, so the name was changed.

I *think* that the only place we just say "precision" is in the very introduction to precisionDecimal, where we say "...(arithmetic) precision..." the first time, to indicate that that's the kind of precision we're talking about.  We must remember that 754 isn't the whole universe, and "precision" means different things to different people.  (N.B.:  The formal name of the property is "arithmeticPrecision"; when simply referring to that particular kind of precision, in ordinary English we say simply "arithmetic precision" ( no fancy made-up words).

(For thaqt matter, 754 just uses "precision" because they only talk about one kind of precision.  By the same logic, we could just say "precision" because we only talk about one kind of precision.  ;-)  )

Created attachment 965 [details]
change proposal for 11531

RESOLVED: proposal in attachment 965 [details] adopted as ammended.
AMMENDMENT: make references to Java non-normative, with an infomative Java reference added to the appendix.

The proposal adopted on 11 March has now been integrated into the status-quo document.