17:45:19 RRSAgent has joined #widereview 17:45:19 logging to http://www.w3.org/2014/10/29-widereview-irc 17:45:28 Ralph has joined #widereview 17:45:31 mdjp has joined #widereview 17:45:36 scribe: timeless 17:45:39 chair: SteveZ 17:46:07 meeting: What is Wide Review and How do we achieve it 17:46:12 RRSAgent, draft mintues 17:46:12 I'm logging. I don't understand 'draft mintues', timeless. Try /msg RRSAgent help 17:46:15 RRSAgent, draft minutes 17:46:15 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 17:46:17 RRSAgent, make logs world 17:46:18 -> https://www.w3.org/wiki/TPAC2014/SessionIdeas#What_is_Wide_Review_and_How_do_we_achieve_it Session proposal 17:46:31 +RalphS 17:46:33 Present+ nigel 17:46:59 Present+ mdjp 17:47:26 Present+ Joanmarie_Diggs 17:47:38 SteveZ: this session is because there were a lot of concerns about what wide-review means in the Process 2014 document 17:47:51 ... so I wanted to describe the rationale between Process 2005 and Process 2014 17:48:02 ... and b) suggest things that could be done to achieve Wide Review 17:48:09 ... and c) answer questions / issues 17:48:19 ... no slides for this 17:48:33 ... in decisions that went into Process 2014 (revision of Chapter 7) 17:48:47 ... a key thing was to enable groups to begin work earlier in their process than implied in Process 2005 17:49:04 ... over the years, we've discovered that groups begin getting reviews/testcases/implementations earlier in their process 17:49:14 ... are more effective at getting out of the process 17:49:22 ... one of the stages was Last Call 17:49:30 ... which unfortunately had two things attached 17:49:50 ... when it was created it was "The WG thinks it has completed its work, Object now or forever hold your peace" 17:50:00 ... when the Patent Policy was created, it needed an anchor point 17:50:20 ... they says "since, LC is when the group finished its work", "we'll attach the Patent process to that" 17:50:26 ... but, it turns out the group wasn't finished there 17:50:45 ... and secondly, WGs weren't serious about LC = Done 17:50:49 ... we'd get a series of LCs 17:50:56 ... there's an objection that the series of LCs 17:51:32 ... you could do LC->CR->LC->CR 17:51:32 ... because patent consideration was tied to LC 17:51:34 ... we've transferred the Patent Commitment to CR 17:51:40 ... it's less likely to cycle 17:51:47 ... and if you cycle, you cycle in CR 17:51:52 ... this had a bad side-effect 17:51:56 ... it removed an opportunity to say 17:52:04 ... "speak now or forever hold your peace" 17:52:08 ... before i go down that route 17:52:14 ... the other thing we did in terms of wide-review 17:52:37 present+ Pete_Resnick 17:52:52 ... reviews at LC was way too late in many circumstances 17:53:03 ... another thing, was to drop LC as the review step 17:53:09 ... saying "review when things are stable" 17:53:14 ... review incrementally as things go along 17:53:23 ... review continuously 17:53:34 ... changing from "met requirement of making an announcement" 17:53:42 ... to "met requirement if people have reviewed it" 17:53:59 [ 7.2.3.1 Wide Review ] 17:54:12 SteveZ: so instead of a check for publishing a request for review 17:54:19 ... you have to show reviews 17:54:27 ... one thing was Disposition of Comments (DoC) 17:54:42 ... if all comments came from Implementers, it probably wasn't Wide Review 17:54:48 ... there's an emphasis to reach out early 17:54:55 ... some groups have an IG (Interest Group) list 17:54:59 ... and send things out there 17:55:08 ... show by getting comments from outside, that they've met the requirement 17:55:25 ... so, change from checkbox, to actual substantive response 17:55:49 myakura_ has joined #widereview 17:56:24 nigel: going through this, the W3C has a Liaison list 17:56:38 ... in addition to the Charter dependencies, I also went to the Liaison list 17:56:57 ... perhaps it would be beneficial to update that list to include Liaisons in the process 17:57:02 SteveZ: you're doing Timed Text 17:57:10 ... clearly an area where groups outside W3C have a documented interest 17:57:16 ... other things where it would be less useful 17:57:22 ... because the technology is more internal 17:57:34 nigel: but then there wouldn't be anything in the liaisons page, so it wouldn't occur 17:58:17 [ nigel will create an action in #w3process to open an issue to add liaisons to 7.2.3.1 ] 17:58:43 present+ Barry_Leiba 17:58:52 barry: Pete and I are IETF application areas 17:58:55 [Ralph departs] 17:58:59 trackbot: Created ACTION-37 - Open an issue to add liaisons to what's considered in the wide review paragraph 7.2.3.1 [on Steve Zilles - due 2014-11-05]. 17:58:59 Sorry, nigel, I don't understand 'trackbot: Created ACTION-37 - Open an issue to add liaisons to what's considered in the wide review paragraph 7.2.3.1 [on Steve Zilles - due 2014-11-05].'. Please refer to for help. 17:59:03 ... i was attracted by this 17:59:07 ... we too tried to solicit it 17:59:20 ... people are often trying to do a semi-final review 17:59:27 ... but resist multiple reviews 17:59:50 SteveZ: the process we did before was largely based on IETF process at the time this was written 17:59:55 barry: we thought as well 18:00:02 SteveZ: what are we trying to do? 18:00:06 ... one is dependencies 18:00:13 ... adding liaisons is relevant there 18:00:33 ... we're trying to advise people to, if you have a dependency, go talk to those people and work out a schedule for review 18:00:42 ... some are dependent to horizontal WGs 18:00:47 ... who are way overloaded w/ work 18:01:00 ... so you want to give them time when it's stable enough to review 18:01:13 ... but not before it's concrete -- that it can't be changed 18:01:29 ... LC isn't that, because that's you think it's complete, what did we miss 18:01:40 ... this is a goal, i don't how it will work out 18:01:45 ... it's an extra burden on the Chairs 18:01:52 barry: on the Chairs, and the reviewers 18:02:03 SteveZ: presumably the reviewers want to review, because they don't want their ox gored 18:02:22 barry: we're trying on a case-by-case basis, you as a chair can ask for a review from a particular group for review 18:02:25 ... it doesn't scale 18:02:35 ... it's only seldom requested, there's a hope 18:02:39 ... we're only getting 15% success 18:02:47 ... it's hard to get people to review early/intermediate versions 18:02:50 ... we're stumped 18:02:56 SteveZ: i'm not offering a panacea 18:03:09 ... mentioning a discussion, the other group may come back and say "here's what's critical to us" 18:03:24 ... so you know what might trigger them, to be more successful in getting reviews 18:03:33 ... In our WG, we'll get a Review request, and the answer is no 18:03:54 ... unless you can get something likely to be in the document that's likely to be a problem, no one wants to look at it 18:04:07 ... hopefully conversations may lead to better results, than just throwing it over the wall 18:04:11 barry: agreed 18:04:19 SteveZ: WG members may be active and doing this 24 hours a day 18:04:23 ... other reviewers aren't 18:04:31 ... you're lucky to get one review, you need to make it count 18:04:34 Zakim has joined #widereview 18:04:46 ... when you clearly have an identified dependency, you can reach out 18:04:50 ... the other thing suggested 18:05:05 ... fantasi had an action item to propose 18:05:16 ... something to help w/ the review process 18:05:33 ... we've observed that the development of a standard doesn't progress at a uniform rate 18:05:42 ... some pieces (often core) progress faster than others 18:05:50 ... it would be useful to mark sections of the document w/ stability 18:05:59 ... CSS developed a system called Shepherd 18:06:08 ... that scans through our documents, we have markup, and ToC 18:06:15 ... and our test-database is driven off those IDs 18:06:22 ... we use IDs, section numbers may change 18:06:27 ... IDs don't change as much 18:06:46 ... we can automatically generate a version of the document indicating which tests have passed in which browsers 18:06:57 ... it wouldn't take too much effort to do this, if we could indicate stability 18:07:04 ... to automatically generate section warnings 18:07:17 ... and if changes are tied to ids, we could include links to change fields 18:07:32 ... for people reviewing it more than once, they can see which sections have changed recently 18:07:37 ... since they last saw it 18:07:45 ... and therefore see what they need to re-review 18:07:53 ... trying to make the documents reviewer friendly is another piece 18:07:58 ... we don't have that technology in place 18:08:01 ... there's no funding 18:08:10 ... it's a couple of CSS WG members working on it 18:08:12 +1 to making it easier for people to answer the question "how stable is this [section of this] document?" 18:08:16 ... we have it for testing level 18:08:21 ... we can in principle extend it 18:08:34 barry: are all W3C drafts developed with the same markup 18:08:39 timeless: there are 2 that are used 18:08:41 barry: that helps 18:08:54 ... we have many different 18:08:59 SteveZ: i understand you have an effort to try to standardize 18:09:04 barry: yes 18:09:19 SteveZ: there's a wiki, has most of the information that I said 18:09:28 ... except why the process changed 18:09:52 ... in the previous session, nigel pointed out it'd be useful to say why the process changed 18:10:04 barry: in IETF, groups will write a document, and not know who their dependencies are 18:10:19 ... they'll stick something in, and not realize that they've created a dependency 18:10:22 s/why the process changed/what the goals of wide review are 18:10:27 ... getting it wrong, they'll break things 18:10:38 ... getting them to know that when they say URI/UTF, 18:10:41 ... that they need to consult 18:10:48 SteveZ: we have more process in forming a Group 18:10:54 ... we say a Charter needs to list dependencies 18:11:01 ... Horizontal WGs need to review charters 18:11:06 ... doesn't really address your problem 18:11:16 ... because Charters are approved by AC members as a whole 18:11:18 ... there's a Vote 18:11:39 ... there's a better chance that the AC member looks at a Charter to see if a reviewer thinks there's a relation 18:11:43 barry: we have a similar process 18:11:52 ... but someone says "this string is a character encoded in UTF-8" 18:12:07 ... all of a sudden it has comparison 18:12:07 ... but it wasn't in the Charter 18:12:25 pete: often in a routing protocol 18:12:57 barry: we have this problem, you have this problem, we're both trying to solve it 18:13:06 SteveZ: no, we're both doing experiments to try to improve this 18:13:15 q+ to talk about flagging content 18:13:25 nigel: you talked about typical duration for review 18:13:48 [ Working Groups should announce to other W3C Working Groups as well as the general public, ] 18:13:49 [ especially those affected by this specification, a proposal to enter Candidate Recommendation (for example in approximately four weeks). ] 18:13:57 nigel: when you talk to industry bodies 18:14:01 ... they might not meet very often 18:14:10 ... and their secretarial skills aren't great 18:14:15 ... they might sit down in six months 18:14:21 ... it probably conflicts with a good practice 18:14:34 ... to state in the document when the end of review period is 18:14:40 SteveZ: i think that review end is required 18:14:47 resnick has joined #widereview 18:14:51 nigel: reviewers will see the document after the review ends 18:14:55 ... and not know what to do 18:15:08 SteveZ: for a Liaison, you should have set this up 18:16:23 timeless: is there a stopword list that you could write a bot for? 18:16:28 s/is/barry, is/ 18:16:31 barry: for some, yes 18:18:40 consider: https://www.w3.org/wiki/DocumentReview 18:18:41 timeless: types of reviews 18:19:02 barry: if we flag "this mentions UTF8", that's easier to get someone to review 18:19:04 ... since they only have to look for UTF8 in the document 18:19:18 pete: security has been good to have a directorate 18:19:24 ... they farm out documents to people 18:19:28 barry: that works at LC 18:19:36 ... the area has asked about early reviews 18:19:48 ... and Directorate has said NO 18:19:57 pete: because they'd get double their work 18:20:04 SteveZ: i discussed that w/ Accessibility 18:20:11 ... and they said it creates as many problems as it solves 18:20:28 SteveZ: this document (DocumentReview) 18:20:33 barry: we could have written that 18:20:50 barry: we can get more than one shot for a particular document 18:20:58 ... but if we try for lots of documents, it would collapse 18:21:16 SteveZ: if the reviewing group indicates the kinds of concerns the chair should be aware of 18:21:25 ... he may go ask for advice earlier if he sees those red flags being raised 18:21:36 ... i think opening dialogue will make this work better 18:21:41 nigel: back to time-frames 18:21:47 ... how do you prevent that from happening 18:21:54 ... the other question for the WG 18:22:03 ... what is the status for comments after Review closes 18:22:16 ... you got a bunch of comments that came after 18:22:36 s/after/during and you got out of LC/ 18:22:42 ... then you get more comments 18:22:51 SteveZ: there's an official answer and an unofficial answer 18:22:55 ... there's a risk of a DoS 18:23:09 ... practically, if the problems are real, the WG should want to try to fix it 18:23:12 s/it/them/ 18:23:23 ... if someone feels they're being unfairly treated, there's an appeals process 18:23:28 nigel: worst-case scenario 18:23:34 ... comments come @ CR 18:23:39 ... you decide to go to PR 18:23:45 ... you have comments w/o DoC 18:23:53 ... it'd be clear if you should have a DoC on all, or all 18:24:06 SteveZ: typically the DoC would be postponed to a future version 18:24:29 nigel: so there's a requirement on the WG to make a decision (postponed) on the comment 18:24:38 SteveZ: there's a default-decision "came after comment deadline" 18:24:45 ... you could record that w/o the WG making a decision 18:24:51 ... good practice would be that you should note the comment 18:25:07 ... and the person noting should respond saying "i think the WG should postpone, do you agree?" 18:25:17 ... we've had cases where a group was totally overloaded 18:29:48 Ralph_ has joined #widereview 18:31:42 barry: we have who are people who aren't totally abusive, but they keep drawing the discussion out beyond where it makes sense 18:31:42 SteveZ: these are all judgement calls, you don't want to make rules that don't make them judgement 18:31:42 mdjp: github comments 18:31:42 ... we have lots of issues, they're commented on outside W3C 18:31:42 ... that's continuous 18:31:42 SteveZ: if you have a way of quickly summarizing that, i think that would be perfectly adequate 18:31:42 ... there's no requirement for an official wide review 18:31:42 ... if you can provide evidence 18:31:43 ... i believe you've established there's been wide review 18:31:43 ... it's important to show people beyond implementers 18:31:43 ... and the key groups are in that set 18:31:44 mdjp: you can identify logs where you have gaps 18:31:44 ... and you can target those groups 18:31:44 barry: now one who's expert in that has reviewed it 18:31:45 SteveZ: automating is great 18:31:45 ... we're not trying to make work 18:31:45 ... we're trying to make sure the right eyes have seen it 18:31:46 barry: a lot of this is having Chairs do the right thing 18:31:46 ... and training the chairs 18:31:46 ... what do you do in the line of Chair Training? 18:31:47 SteveZ: not much/enough 18:31:47 ... this is the only SDO i know of where an appointment doesn't involve Chair training 18:31:47 barry: there's Chair training they sometimes do before the meeting 18:31:48 ... and there's a Wednesday lunch for Chair training 18:31:48 s/before/the Sunday before/ 18:32:23 SteveZ: w3c has recently established a number of Chair Training Phone Calls 18:32:27 ... for topics relevant to Chairs 18:32:29 ... Tools 18:32:33 ... Human Element 18:32:36 ... How to be Effective 18:32:40 http://www.w3.org/2014/Talks/chairs-part4/#/ 18:32:49 mdjp: As a new chair, these sessions would have been really useful before the F2F 18:33:04 http://www.w3.org/2014/06/17-chairing-minutes.html 18:33:05 ... without the crossover to the outgoing chair, it would have been really difficult to run the meetings 18:33:29 s/fantasi/fantasai/ 18:33:34 http://www.w3.org/2014/04/24-chairing-minutes 18:33:36 RRSAgent, draft minutes 18:33:36 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 18:33:57 nigel: There are others too - I'm not sure if they're collated onto a single page for ease of reference 18:34:00 SteveZ2 has joined #widereview 18:34:01 fantasai has joined #widereview 18:34:07 RRSAgent, draft minutes 18:34:07 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 18:34:27 SteveZ2: no rules, and talk to the people you need to talk to 18:34:31 s/SteveZ2/SteveZ/G 18:34:45 nigel: you mentioned if all reviews are from implementers, that's not enough, it's not wide 18:34:53 ... but evidence of implementation can be evidence of review 18:35:00 ... since you can't implement without reading 18:35:06 SteveZ2: the way our process is 18:35:17 ... you weren't going to get anywhere unless there were implementations anyway 18:35:31 ... but it isn't the reviewers we're concerned about 18:35:49 ... it's dealing with the communities that could be disrupted or disenfranchized if this becomes a standard 18:36:14 timeless: I've dealt with implementers who see problems and ignore them 18:36:18 fantasai: +1 18:36:26 Ralph has joined #widereview 18:36:43 timeless: They pick an answer silently, without notifying anyone. They do /something/ but nobody realises it's not what was intended in the spec until much later. 18:36:51 ... Most implementers aren't noisy like that. 18:37:11 fantasai: An implementer who says something would be counted for wide review evidence 18:37:32 Timeless: there are implementers that will just jump over problems without reporting them so implementations are not total proof of review 18:37:33 timeless: When browser devs implement something what actually matters is usage in websites. 18:37:44 ... Website devs are even worse than browser devs! 18:37:55 scribeNick: nigel 18:38:28 timeless: You have to go and look at website code to see if they've put nasty comments in about your spec! 18:38:28 ... It's a nasty thing to go and hunt those comments down. 18:38:47 ... We've begged for years for this kind of feedback, and we're lucky to get feedback from 5 sites, out of billions! 18:38:57 ... A way to harvest those comments would be beneficial. 18:39:26 ... A way to demonstrate searches on stackoverflow, etc would be good. 18:39:49 ... Conversely if people don't comment on the likely sites e.g. stackoverflow, then that may be evidence that there isn't a problem 18:40:05 timeless: reviewing the comments in code on key websites and/or on stack overflow will show evidence of wide review 18:40:47 RRSAgent, draft minutes 18:40:47 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 18:41:03 i/fantasai: An implementer/scribe: nigel/ 18:41:06 s/scribeNick: nigel// 18:41:12 scribe: timeless 18:41:32 nigel: W3C generally wants to make 18:41:42 ... everyone wants to include "the general public" in "wide review" 18:41:48 ... the best way to make things visible 18:41:54 ... is "the w3c home page" 18:42:08 ... if you put things in /TR/, you can ask staff to put things on the w3 home page 18:42:18 ... it's been proposed to have a Blog or something for announcing reviews 18:42:28 SteveZ2: the DocumentReview wiki documents making reviews happen 18:42:37 ... it's asking the W3C to make a mailing list 18:42:42 ... there's an issue of what the name should be 18:42:52 ... i think that will happen 18:42:56 ... it's not last-call 18:43:08 ... the intent is that you identify what you want reviewed in the Status section 18:43:23 ... because it may not be the whole document, perhaps just a recent change 18:43:30 nigel: some organizations have an "Official Journal" 18:43:34 ... it's widely known to the world 18:43:40 ... a clear statement from W3 that this is the place 18:43:53 ... your XXXs should be looking there 18:43:58 ... boring, but effective 18:44:04 q+ To suggest Twitter and hashtag that stateholders are likely to see. (Not a joke) 18:44:06 SteveZ2: the issue of publicizing this list is known to Team 18:44:12 ack timeless 18:44:12 timeless, you wanted to talk about flagging content 18:44:25 ack joanie 18:44:25 joanie, you wanted to suggest Twitter and hashtag that stateholders are likely to see. (Not a joke) 18:44:42 joanie: i did this by ranting, and i got a lot of replies 18:45:02 ... if your stakeholders are web developers, and spit out a single tweet with a link to the document and a few appropriate hashtags 18:45:12 ... a couple of stakeholders will retweet 18:45:30 SteveZ2: we should consider using all channels that are available 18:45:53 joanie: i've used Twitter to complain about web developers doing things 18:46:05 ... and then developers reply explaining why they did it 18:46:15 SteveZ2: two other things in consideration 18:46:18 ... besides the ML 18:46:28 ... one is a dashboard page listing which active reviews 18:46:32 ... and secondly a Calendar 18:46:35 ... listing reviews closing 18:46:44 ... I think, if we could automate it 18:46:51 ... different views of the same information 18:47:08 ... you have an announcement, it shows up in the right places 18:47:09 fantasai: CSS WG 18:47:14 ... W3 will announce it 18:47:21 ... we'll announce it on our blog 18:47:26 ... we'll send it to places 18:47:31 ... we'll tweet a link 18:47:46 ... we make our editors write the announcement, because we want them to explain the key points 18:47:55 ... otherwise, W3 staff does boilerplate 18:48:06 ... there's people out there, you want their feedback, try to get it 18:48:24 SteveZ2: developing an Interest community is a valuable way to get comments from others than implementers 18:48:34 ... not the public exactly, but people who are following 18:48:48 ... for CSS, it's www-style 18:48:55 ... but we can get answers in the 50s 18:48:57 ... when we ask 18:49:06 fantasai: i think we'd benefit from a less technical channel 18:49:11 ... lower traffic 18:49:27 ... people have wanted to participate, but don't want to subscribe to the ML 18:49:31 ... it's high traffic 18:49:37 ... high signal to noise 18:49:43 SteveZ2: we tag messages 18:49:50 ... you can sort, i consciously sort 18:49:54 ... i don't do animation 18:50:02 fantasai: we ask people to tag w/ the spec shortname 18:50:09 RRSAgent, draft minutes 18:50:09 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 18:50:22 SteveZ2: i wish to thank our scribe, immensely 18:50:25 barry: indeed 18:50:33 SteveZ2: and i'd like to thank everyone for participating 18:50:47 ... i'm trying to do the process, not as written, but how it is/should be in practice... 18:50:52 ... a very different question 18:50:56 [ Adjourned ] 18:50:58 [ Lunch ] 18:51:01 RRSAgent, draft minutes 18:51:01 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 18:52:04 s/ Timeless:/ timeless:/ 18:52:24 RRSAgent, draft minutes 18:52:24 I have made the request to generate http://www.w3.org/2014/10/29-widereview-minutes.html timeless 20:05:52 Ralph has joined #widereview 20:19:56 timeless has left #widereview 21:10:36 Zakim has left #widereview 21:28:24 Ralph has left #widereview 22:49:12 myakura has joined #widereview