This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
The section 4 Semantics of full text processing diagram, 'step 2 'convert a sequence of items to a tokenized text or atomic values' should be stated as 'convert a sequence of items to a tokenized text'
Why? Do you think that step 2 can't result in atomic values, or do you think that it can, but that "atomic values" are subsumed by "a tokenized text"? Note that the prose for step 2 describes various ways that XQuery/XPath expressions can be nested within Full-Text expressions, some that involve tokenization and some that don't.
yes, certain xquery/xpath expressions result are used for, say distance value. In such context, it is used as atomic value instead of tokenized text.