This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
Checking the CVS history of XQTSCatalog.xml and of the files in Queries/XQuery/Expressions/PrologExpr/VersionProlog/, I find an alarming amount of churn, all revolving around the following paragraph in 4.1 [Version Declaration]: If a version declaration is present, no Comment may occur before the end of the version declaration. If such a Comment is present, the result is implementation-dependent. (I refer to this as "the comment-before-versiondecl clause".) I've appended the whole history below. The upshot is: (1) The test cases that originally (or nearly originally) were intended to address this clause no longer do so. (2) For test cases version_declaration-001 version_declaration-002 version_declaration-003 version_declaration-004 K-VersionProlog-1 the catalog says <expected-error>*</expected-error> which is inconsistent with the query. So: Is there any point having test cases with comment-before-versiondecl, when the resulting behaviour is implementation-dependent? (Do the results tell us anything?) If yes, then we should undo most of the changes to all the version_declaration-nnn test cases. If no, we should probably just remove them. ----------------------------------------------------------------- Abbreviations: vd = version_declaration pv = prolog-version KVP = K-VersionProlog error=* = <expected-error>*</expected-error> TTF = Testing Task Force 2005-10-05: Ravi Chennoju (rchennoj) adds test cases vd-001 through vd-009 (XQTSCatalog.xml rev 1.349): 001-005 expect no error 006-008 expect XPST0031 009 expects XPST0003 All have a comment preceding the VersionDecl, and so the behaviour is implementation-defined according to the XQuery spec, but Ravi is presumably not aware of this. 2005-11-01: Ravi adds XQST0038 to vd-005's expectations. (XQTSCatalog.xml rev 1.389) 2006-02-06: Carmelo Montanez (cmontane) adds vd-010, expects XQST0087. 2006-02-15/17: Apparently, Carmelo becomes aware of the comment-before-versiondecl clause, because he adds pv-1 thru pv-12, all of which have comment-AFTER-versiondecl. In fact, pv-4 through pv-12 are simply copies of vd-001 through vd-009 with the versiondecl moved up. At about the same time, he adds <expected-error>*</expected-error> to vd-001 through vd-010 (XQTSCatalog.xml rev 1.522), presumably to reflect their implementation-dependent behaviour. 2006-05-11: In Bug 3275, Jinghao Liu thinks vd-1 shouldn't expect error=*. Andrew and Ravi agree that it's there because of the comment-before-versiondecl clause. [So, the intent of the tests seems fairly clear: test cases vd-* exercise the imp-dep behaviour of comment-before-versiondecl, and pv-* exercise the non-imp-dep behaviour that vd-* were originally intended to cover. Call this the Revised Intent. ] --- 2006-05-22: On the TTF mailing list, Franz Englich (fenglich) rightly points out that vd-005 thru vd-010 expect both a specific error code and error=*, which is redundant. In each case, one or the other should be removed. http://lists.w3.org/Archives/Member/member-query-test/2006May/0045.html 2006-05-22: cmontane removes error=* from vd-009 (XQTSCatalog.xml rev 1.603) 2006-06-08: TTF meeting 64 decides that in each case, error=* should be removed: > ACTION TTF-064-03: Carmello to fix version_declaration-010 test and > remove '*' for expected error and use specific error > ACTION TTF-064-04: Ravi to fix version_declaration-00[5,6,7,8] test > and remove '*' for expected error and use specific error http://lists.w3.org/Archives/Member/member-query-test/2006Jun/0054.html (That is, it seems the TTF forgot that error=* was there to cover the imp-dep behaviour of comment-before-versiondecl. As far as I can see, they should have decided to remove the specific error.) 2006-06-08: cmontane removes error=* from vd-010 (rev 1.618) 2006-06-09: rchennoj removes error=* from vd-005 thru 008 (rev 1.619) (This leaves vd-001 thru 004 expecting a value or error=*, still in line with the Revised Intent, but vd-005 thru 010 expecting specific errors, which is incorrect, and will bite us later.) --- (2006-07-19: cmontane moves vd-005) (2006-09-15: In Bug 3731, Per Bothner questions error=* for vd-001 thru 005, but he's just misreading the metadata.) 2007-01-26 In Bug 4281, Michael Kay points out the comment-before-versiondecl in vd-010, which is inconsistent with its specific error message. 2007-07-23 In vd-010, Andrew moves up the versiondecl, making it consistent with the error message, though inconsistent with the Revised Intent. (2007-08-09: In the Bug, Andrew reports making this change.) 2007-11-22 fenglich adds KVP-1 through KVP-5, all with comment-before-versiondecl. The comment for KVP-1 clearly indicates that this is intentional, but oddly, only KVP-1 has error=*; the others expect a specific error or a value. 2009-05-20 In Bug 6934, Josh Spiegel lists many cases of comment-before-versiondecl, including vd-* and KVP-*. His implementation, which (conformantly) raises an error in such cases, only passes vd-001 thru 0004. 2009-10-14 To "fix" bug 6934, fenglich moves up the versiondecl in vd-001 thru 008, and KVP-1 through 5. However: vd-001 thru 004 and KVP-1 still expect error=*, which is inconsistent with their (changed) queries. None of vd-* adhere to the Revised Intent any more. That is, there are no longer any tests that exercise the comment-before-versiondecl clause. The vd-* queries are basically duplicates of pv-* queries. (But missing a couple changes that some pv-queries received.) xxx
Just to add to this, I'm sure the conversion to QT3 format hasn't helped matters. Generally, the automatic conversion moved comments out of the query into the metadata, and where comments in the query were part of the essence of the test, manual intervention was needed. If running the query produced the correct result, the need for this intervention might well have gone unnoticed. (These tests are by no means the only ones that don't do anything useful, of course; I didn't regard that as something to worry about during the conversion).
> Is there any point having test cases with comment-before-versiondecl, > when the resulting behaviour is implementation-dependent? (Do the > results tell us anything?) I agree that these test cases won't tell us anything with regards to our specification. I would still leave them in the test suite. Every so often there will be an implementation that hasn't prepared for this situation and ends abnormally. Leaving these test cases in the test suite will help catch such omissions.
This bug issue was fixed in the bug #14955 for the QT3 test suite therefore I am marking this resolved - moved