I think it should say that the format modifier if present consists of a single character optionally followed by a sequence of characters starting with '(' and ending with ')' and not containing either '(' or ')'; and then  for 'final character' read, the last character in the string or the last character before the '('. Editorial work needed to find the right way to say this...

The WG encouraged the editor to specify the format of the picture by means of a formal grammar or regular expression. The following regular expression has been submitted for review by the WG, and is included in the status quo text on the presumption that it is OK:

^((\p{Nd}|#|[^\p{N}\p{L}])+?)(([co](\([^()]\))?)?[at]?)$

(this needs to be supplemented by additional prose rules, of course)

The last call draft of 13 December 2011 includes the a regular expression to define the picture string as given in comment #2, but unfortunately (as pointed out in subsequent email) this is incorrect: it works for the "decimal" form of picture string, but not for other forms such as A, I, or Ww.

I think the regex for the format modifier should be as follows: if the string ends with

(([co](\([^()]+\))?)?[at]?)

then this is taken as the format modifier. This differs from the published regex by the addition of the "+" quantifier. There may be better ways of expressing this (or paraphrasing it).

The regex for the first part of the picture is wrong. The part preceding the format modifier must either be a decimal digit pattern, which matches

((\p{Nd}|#|[^\p{N}\p{L}])+?)

or some other format token defined in the specification, which matches

(A|a|I|i|W|w|Ww)

or an implementation-defined format token, which I propose should be unrestricted except that it must not be empty.

I think we should express this as follows.

(1) The value of $picture must match the regular expression ^ (.+) ( ([co]( \([^()]\))? )? [at]? )$ according to the rules of the matches() function with flags "xs". (This is not a severe restriction: only the zero-length string fails to match this pattern.)

(2) Following this match, the content of captured group 1 is referred to as the primary format token, and the content of captured group 2 is referred to as the format modifier. The semantics of the format modifier are covered by existing rules.

(3) The primary format token is classified as follows:

(3a) if it contains a decimal digit or "#" then it is taken as a decimal-digit-pattern and must follow the rules for decimal digit patterns (otherwise, error)

(3b) if it is one of (A|a|I|i|W|w|Ww) then it is handled as defined in the specification for that format token

(3c) otherwise, its meaning (if any) is implementation-defined; if the implementation does not attach any other meaning to the format token then it is handled in the same way as the primary format token "1" (Currently the spec says it must use a format token of "1", which discards the modifier: this is a change.)

These rules change the result of some tests that are currently deemed errors, to being implementation-defined, with a fallback that uses format picture "1". Examples of such tests include format-integer-038, whose picture is "()Wwo", and format-integer-057, which uses a picture of "boo".

The WG was minded to improve the syntax of the picture string to avoid these ambiguuties by introducing a delimiter between the primary and secondary format tokens. A new proposal will be forthcoming.

Proposal:

(a) replace this text:

<old>
The value of $picture must match the regular expression:

^((\p{Nd}|#|[^\p{N}\p{L}])+?)(([co](\([^()]\))?)?[at]?)$

The substring that matches the first capturing group in this regular expression are referred to as the primary format token. The substring that matches the second capturing group (which may be empty) is referred to as the format modifier. A picture thus consists of a primary format token, followed by an optional format modifier.
</old>

with this:

<new>

The value of $picture consists of a primary format token, optionally followed by a format modifier. If the string contains a semicolon then everything that precedes the last semicolon is taken as the primary format token and everything that follows is taken as the format modifier; if the string contains no semicolon then the entire picture is taken as the primary format token, and the format modifier is taken to be absent (which is equivalent to supplying a zero-length string).

</new>

(b) add the semicolon to the examples that require it

Proposal accepted with two amendments: change "contains a semicolon" to "contains one or more semicolons", and keep the regex for the case of numeric primary format tokens.

I think I'm correct in saying that anything before the semicolon delimiter is a valid primary format token.

However, is the format modifier required to match some regex?

If it does not match, is that an error or a silent failure?

The outstanding point in comment 8 is covered by bug 17099. Closing this one.