Personal response.

I agree the design is wrong. (I say as much in my book XSLT/XPath Programmer's Reference 4th ed). The question is what we can now do about it.

I think that issuing an erratum to change the behaviour of the existing function in such an incompatible way would be very damaging. Not just because it would break existing code, but also because there would be a period of at least three years where different implementations in active use would give different results, and also because books (like my own) and other reference and tutorial materials will be on sale and in use for many years and will be giving incorrect information to users, leading to general confusion in the user community. And because the release cycle for some products is long (up to 3 years in the case of major relational database systems), the goal that XPointer could use an XPath implementation "out of the box" would not be achieved for several years.

One solution might be to define a new function with the "improved" behaviour, to exist alongside the old. This isn't ideal either, because there will still be an interoperability problem and an education problem, but at least there won't be a compatibility problem, and the interoperability problem will be that some implementations accept the new function and others don't, which is a lot better than having them return fundamentally different results.

Another softer approach might be to introduce a new way of calling the existing function: if the supplied ID value is prefixed with "#" then the function exhibits the new behaviour. This approach has the benefit that it makes it easier for applications to handle the coexistence of implementations that do or don't implement the erratum - the current specification is that if a value such as "#D001" is passed to the function, no error is reported and nothing is selected. So the expression

(id("#D001"),id("D001")/..)[1]

would give the desired result on both pre-and post-erratum implementations.

But of course this isn't a panacea - one aim of the function is that you can use it to follow a link from an IDREF or IDREFS attribute, and this would require some string manipulation to make it work.

I think we should probably see this problem with a sense of perspective. Many users are reluctant to use id() anyway, because it only works when the input document is validated, and because some parsers don't report IDness even when they are performing validation. Under XSLT, many users prefer to use the key() function. Under XQuery, they prefer to write an expression such as //A[@id="D001"] and rely on the optimizer to use whatever indexes are available. Even if it's broken, do we really need to fix it?

On 2008-09-05 the XML Schema WG resolved (minutes to be published RSN) to endorse this issue, and respectfully request that QT accept this issue as requiring its immediate attention.  The Schema WG believes that the most satisfactory dispensation would be to open an erratum against F&O.

Thank you.

I was asked today by the joint XQuery/XSLWG meeting to produce a proposal. The proposal is to add a new section, identical to 15.5.2 fn:id, with the following differences:

(a) the function name is fn:element-with-id. (It has the same two signatures as fn:id).

(b) Rule 2, bullet 1, sub-bullet 1 is changed to read:

The element has a child element node whose is-id property property (See Section 5.5 is-id AccessorDM.) is true, and whose typed value is equal to V under the rules of the eq operator using the Unicode code point collation (http://www.w3.org/2005/xpath-functions/collation/codepoint).

(c) The existing Notes under fn:id() are referenced rather than being duplicated

(d) A new note is added explaining the problem and the difference between the two functions, and the approach to introducing it as an optional function by means of an erratum.

(e) The new function is added by Erratum to XPath 2.0/XQuery 1.0, with a statement that it is an optional feature for this version. To localize the changes, we can do this, I think, by means of a statement within the F+O specification of the function that says "Where any statement in the conformance rules of a host language requires that all functions in this function library must be provided, such a statement does not apply to the fn:element-with-id function (in both its variants): processors may provide this function but are not required to do so."

(f) The function becomes an integral part of the function library for XPath 2.1/XQuery 1.1.

Erratum E31 has been drafted to define the new function fn:element-with-id().

I'm marking this bug FIXED on the grounds that a new function has been added to satisfy the requirement as stated without "breaking" the existing function. 

If you are satisfied with this resolution, please mark the bug CLOSED. 

I'm marking this as closed, since it has been fixed by erratum (E31) and the new text is in the master document for both the 1.02e and 1.1 branches.

I'm posting even though I agree that this bug is properly closed, because Michael Kay's comment #1 contains a broken meme I am trying to stamp out.

"id() [...] only works when the input document is validated" is not true.  Conforming non-validating XML parsers MUST process ATTLIST declarations in the internal subset that appear before the first reference to an external parameter entity. Conforming processors that read external DTD subsets and parameter entities, whether validating or non-validating, MUST process all ATTLIST declarations.  Once an ATTLIST declaration for a particular attribute has been processed, the XML parser MUST report it to the application.  So if you write:

<!DOCTYPE root [<!ATTLIST root foo ID #REQUIRED>]>
<root foo="FOO">

every conforming XML parser MUST report the IDness of root/@foo.  What is more, if you write:

<!DOCTYPE root [<!ATTLIST root foo ID #REQUIRED>]>
<root foo="FOO">

every conforming processor MUST still report the IDness of root/@foo, even though 32 is not a valid ID.

Erm, make the second example:

<!DOCTYPE root [<!ATTLIST root foo ID #REQUIRED>]>
<root foo="32">

(thanks to Paul Grosso)