This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
Section 3.2 (FTMatchOptions) Second sentence after EBNF. (Technical) Most of the match options do not, in fact, modify the sets of tokens and phrases in the query; the way the semantics are defined these days, they impact how the tokens and phrases in the query are matched against tokens and phrases in the search items. True, an implementation may process, say, a stemming option, by constructing a different set of search items. All that said, we also need to be crisper about whether match options apply to how the query string is processed or how the documents are processed. Should the language of the document really affect how search strings are stemmed? And a search against a collection of documents? Here is a case where I think we're saying you cannot tokenize your documents until you see the query, which is wrong. What we should be saying is that these options may apply to which query tokens are produced and either thereby or separately affect how matching is done. We should not say that they affect how documents are tokenized, because then you are precluding any but small-scale implementations. For up-front document indexing, tokenization may produce several different results in different indices, and the match options then become matters of selection of the appropriate index.
The WG agreed to change the specification as follows: In 3.2 FTMatchOptions first para after the BNF, remove the second sentence and close bug #3747.