15:05:14 <RRSAgent> RRSAgent has joined #htmlt
15:05:14 <RRSAgent> logging to http://www.w3.org/2010/08/24-htmlt-irc
15:05:22 <krisk> zakim, who is here?
15:05:22 <Zakim> On the phone I see krisk
15:05:23 <Zakim> On IRC I see RRSAgent, plh, trackbot, Zakim, krisk, Ms2ger, gsnedders, jgraham
15:05:38 <jgraham> I think gsnedders and I will not be around much
15:05:44 <krisk> Will anyone else be dialing in?
15:05:53 <Zakim> +Plh
15:06:22 <jgraham> We have to leave at 17:25 
15:06:50 <krisk> That is fine - should be a quick meeting
15:07:14 <krisk> Agenda-> http://lists.w3.org/Archives/Public/public-html-testsuite/2010Aug/0010.html
15:07:48 <krisk> Chair: krisk
15:08:08 <krisk> Scribe: krisk (note most folks are on IRC)
15:08:23 <krisk> Agenda Item #1 Check for any bugs on approved tests (currently zero)
15:09:07 <krisk> Ms2ger - posted about XHTML5 tests see -> http://lists.w3.org/Archives/Public/public-html-testsuite/2010Aug/0011.html
15:10:16 <krisk> I'll take a peek at this feedback and report back, if others have feedback (not on default == automated) please respond
15:11:55 <krisk> Now the other feedback is to have automated be the default for tests - manual tests need a reason
15:12:10 <Ms2ger> krisk, FWIW, I can rewrite those xhtml test as reftests if you'd like
15:13:18 <plh> the problem is that we have no way to run reftests
15:13:28 <jgraham> Ms2ger: (I have a mild preference for javascript tests since there is somewhat less that can go wrong)
15:13:34 <plh> ie, the tests would still be ran manually
15:13:51 <krisk> For simple DOM tests (e.g. getElementsbyClassName) they should be automated
15:13:53 <jgraham> plh: The CSSWG are using reftests exclusively for CSS3
15:14:06 <plh> and do they have a way to run them?
15:14:21 <jgraham> Not sure, I will talk to fantasai/Tab
15:14:28 <plh> I doubt they do
15:14:35 <jgraham> But browser vendors can all run reftests now
15:14:39 <krisk> Nope - they are device/os dependent
15:15:03 <plh> I guess one thing we can do: let the harness run the reftests manually, and if someone has a better way to run those tests, good for them
15:15:18 <jgraham> That is better
15:15:42 <plh> so, maybe we need to modify the harness to allow reftests to be ran manually for now
15:15:47 <krisk> Let's agree to that
15:15:56 <plh> at least, we'll be able to accept reftests
15:16:02 <Ms2ger> Sounds good
15:16:05 <jgraham> In practice being able to run reftests is necessary to automate other W3C testsuites so I don't think it is any problem for us to have the same requirement
15:16:17 <jgraham> Having manual tests is always a problem though
15:16:27 <jgraham> and is causing problems in practice with CSS 2.1
15:16:49 <plh> yes, but I also don't want top exclude some class of mobile user agents for example in the process of developing reftests
15:16:57 <plh> s/top/to/
15:17:02 <jgraham> Nor do we
15:17:23 <krisk> OK then let's state this as our plan...
15:17:26 <jgraham> So reftests should always be possible to run manually too
15:17:38 <plh> sounds great to me
15:17:40 <krisk> Tests that can be tested via javascript should not be manual
15:17:57 <plh> so Kris, how hard would it be for you to allow reftests in the harness?
15:17:58 <jgraham> i.e. they should always have human-readable instructions
15:18:24 <krisk> Tests that need some non-javascript verification need to have manual instructions in the test
15:18:40 <plh> for reftests, it's a simple comparison
15:18:46 <krisk> That should not be a problem
15:18:50 <plh> so you need to be able to display two files
15:19:31 <krisk> Let's move on to the next agenda item
15:19:34 <jgraham> You want to display them in a way that allows you to flip between them with the tests in the same place in the viewport
15:19:43 <jgraham> Like in two tabs
15:19:54 <jgraham> Makes spotting small differences easier
15:19:58 <Ms2ger> What jgraham said
15:20:05 <plh> and the default instruction should be "For this test to pass, the two following pages must be exactly identically."
15:20:28 <krisk> sounds good
15:20:42 <krisk> glad to see we are making progress
15:20:55 <plh> I guess, we need a file naming convention or something
15:21:23 <krisk> a simple .ref. in the file name - e.g. test.html and test.ref.html
15:21:33 <plh> yep
15:21:40 <Ms2ger> Mozilla uses -ref, btw
15:21:59 <plh> we're using names like a-href at the moment
15:22:15 <plh> using -ref might clash with that
15:22:16 <krisk> they only do internally - not for tests on the w3 site
15:22:41 <krisk> Agenda Item #2 #2 Approve 25 more of Philip Taylors Canvas Tests
15:23:14 <krisk> I looked at tests from http://test.w3.org/html/tests/submission/PhilipTaylor/canvas/size.attributes.setAttribute.trailingjunk.html to http://test.w3.org/html/tests/submission/PhilipTaylor/canvas/toDataURL.arguments.3.html
15:23:35 <krisk> The look fine to me - any objections?
15:23:41 <plh> no objection from me
15:24:12 <krisk> gsnedders - I assume you are OK given the past IRC chats
15:24:33 <plh> let's assume so
15:24:40 <krisk> OK
15:25:23 <krisk> Agenda item #3 Conditionally approve Opera and Microsofts getElementsByClassName tests - additional work is test harness integration.
15:25:55 <krisk> I'd like to approve these and move them into the harness so they can be good examples of automated tests
15:26:13 <plh> sounds good to me as well. do we have overlap between the two series of tests?
15:26:14 <krisk> We have discussed this in the past, I just want to formalize this work
15:26:53 <krisk> I don't think so - their is only like ~40 total tests
15:27:08 <plh> I'm fine with approving both series of tests
15:27:11 <krisk> Given the ways that this API can be used, it's possible to create alot more tests
15:27:25 <krisk> Though the value of additional tests goes down pretty fast...
15:28:00 <krisk> Which is why the API works well today across borwsers (except in cases like namespaces)
15:28:16 <krisk> OK then let's conditionally approve these tests
15:28:42 <krisk> Any other items people want to discuss? Or shall we adjourn?
15:29:25 <Ms2ger> krisk, I'd like to submit some of my own tests
15:29:45 <Zakim> -Plh
15:29:47 <Zakim> -krisk
15:29:47 <Zakim> HTML_WG(HTMLT)11:00AM has ended
15:29:49 <Zakim> Attendees were krisk, Plh
15:29:57 <krisk> rrsagent, generate minutes
15:29:57 <RRSAgent> I have made the request to generate http://www.w3.org/2010/08/24-htmlt-minutes.html krisk
15:30:03 <Ms2ger> krisk, is there any documentation for external contributors?
15:30:11 <krisk> on our wiki 
15:30:34 <krisk> see -> http://www.w3.org/html/wg/wiki/Testing
15:30:46 <krisk> basically submit tests into Hg
15:31:12 <Ms2ger> I need a W3C account for that, right?
15:31:22 <krisk> Yes you need an account
15:31:36 <Ms2ger> I'd rather not go there, tbh
15:31:59 <plh> are you listed as a participant in html wg?
15:32:02 <Ms2ger> No
15:32:33 <plh> one simple way is that you request IE status in that group, and that will give you an account for Hg
15:33:07 <Ms2ger> Yeah, but I'd rather not
15:33:20 <krisk> You also need to make sure you properly license your or others work see -> http://www.w3.org/html/wg/wiki/Testing/Submission/
15:33:57 <Ms2ger> Right
15:34:49 <krisk> Do you have tests today or just want to help?
15:36:03 <Ms2ger> I've got some at http://bitbucket.org/ms2ger/html-tests
15:36:29 <Ms2ger> I guess I'll just stick a license file in there
15:37:27 <krisk> How about you do that (use BSD license) and then send mail to the list
15:37:39 <Ms2ger> Okay, will do
15:37:40 <krisk> I can help move them from your site to Hg
15:38:01 <krisk> You'll need to be open to feedback
15:38:08 <Ms2ger> Of course
15:58:19 <plh> plh has left #htmlt
17:34:10 <Zakim> Zakim has left #htmlt