IRC log of htmlt on 2010-05-04

Timestamps are in UTC.

14:55:12 [RRSAgent]
RRSAgent has joined #htmlt
14:55:12 [RRSAgent]
logging to http://www.w3.org/2010/05/04-htmlt-irc
14:55:18 [gsnedders]
zakim, this will be htmlt
14:55:18 [Zakim]
ok, gsnedders; I see HTML_WG(HTMLT)11:00AM scheduled to start in 5 minutes
14:55:24 [gsnedders]
zakim, what's the code?
14:55:24 [Zakim]
the conference code is 48658 (tel:+1.617.761.6200 tel:+33.4.89.06.34.99 tel:+44.117.370.6152), gsnedders
14:59:06 [Zakim]
HTML_WG(HTMLT)11:00AM has now started
14:59:14 [Zakim]
+[Microsoft]
15:01:59 [krisk]
krisk has joined #HTMLT
15:02:33 [krisk]
hello
15:02:44 [krisk]
zakim, what's the code?
15:02:44 [Zakim]
the conference code is 48658 (tel:+1.617.761.6200 tel:+33.4.89.06.34.99 tel:+44.117.370.6152), krisk
15:03:12 [krisk]
trackbot, status
15:03:12 [trackbot]
This channel is not configured
15:03:40 [krisk]
trackbot, prepare telcon
15:03:40 [trackbot]
Sorry... I don't know anything about this channel
15:03:40 [trackbot]
If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
15:04:00 [krisk]
trackbot #htmlt
15:04:42 [krisk]
trackbot, prepare telcon
15:04:42 [trackbot]
Sorry... I don't know anything about this channel
15:04:42 [trackbot]
If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
15:05:24 [plh]
plh has joined #htmlt
15:05:36 [krisk]
Zakim, this will be htmlt
15:05:36 [Zakim]
ok, krisk, I see HTML_WG(HTMLT)11:00AM already started
15:05:48 [krisk]
trackbot, prepare telcon
15:05:48 [trackbot]
Sorry... I don't know anything about this channel
15:05:48 [trackbot]
If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
15:05:59 [plh]
trackbot-ng, start telcon
15:06:01 [trackbot]
RRSAgent, make logs public
15:06:03 [trackbot]
Zakim, this will be HTML
15:06:04 [trackbot]
Meeting: HTML Weekly Teleconference
15:06:04 [trackbot]
Date: 04 May 2010
15:06:04 [Zakim]
ok, trackbot, I see HTML_WG(HTMLT)11:00AM already started
15:06:16 [Zakim]
+Plh
15:07:30 [adrianba]
adrianba has joined #htmlt
15:08:53 [krisk]
zakim, Microsoft is adrianb, kkrueger
15:08:53 [Zakim]
I don't understand 'Microsoft is adrianb, kkrueger', krisk
15:09:01 [Zakim]
+ +04613479aaaa
15:09:12 [gsnedders]
zakim, aaa is me
15:09:14 [Zakim]
sorry, gsnedders, I do not recognize a party named 'aaa'
15:09:19 [gsnedders]
zakim, aaaa is me
15:09:19 [Zakim]
+gsnedders; got it
15:09:19 [krisk]
should we start the meeting?
15:09:29 [adrianba]
zakim, Microsoft has adrianba, krisk
15:09:30 [Zakim]
+adrianba, krisk; got it
15:09:30 [krisk]
anyone want to scribe?
15:09:32 [plh]
--> http://lists.w3.org/Archives/Public/public-html-testsuite/2010May/0000.html agenda
15:10:06 [adrianba]
scribenick: adrianba
15:10:11 [adrianba]
scribe: adrianba
15:10:15 [adrianba]
chair: krisk
15:10:21 [adrianba]
agenda: http://lists.w3.org/Archives/Public/public-html-testsuite/2010May/0000.html
15:10:38 [adrianba]
TOPIC: Check for any bugs on approved tests (currently zero)
15:11:21 [adrianba]
krisk: currently zero, but we can have some bugs once we make changes to the harness
15:11:37 [adrianba]
...to support james graham's async feedback
15:12:18 [adrianba]
gsnedders: don't want to start with one harness and to keep hacking another on top and one on top of that
15:12:28 [adrianba]
krisk: do you have an idea what that should look like?
15:12:35 [adrianba]
gsnedders: not really
15:12:42 [gsnedders]
jgraham: ping
15:12:49 [adrianba]
krisk: in the windows tests i think i did something along those lines
15:13:08 [adrianba]
plh: i wonder how independent we can be from the harness
15:14:01 [adrianba]
...so that the tests can be written in a way that makes it easy to switch to a new harness
15:14:13 [adrianba]
...if we get another harness that is better
15:14:29 [adrianba]
...we don't have to re-write everything
15:14:34 [jgraham]
FWIW I think the async stuff is the stuff that makes it hard to switch harnesses too
15:14:45 [adrianba]
krisk: is this about consistency so that we can depend upon certain things being in the test
15:15:07 [adrianba]
gsnedders: the main thing is about having the assert functions that check for what the test is testing
15:15:12 [adrianba]
...assertEquals, etc.
15:15:43 [adrianba]
krisk: if you are too specific about what has to be written then you lose flexibility
15:15:51 [adrianba]
...but it has to say pass/fail in a consistent manner
15:16:09 [jgraham]
Given a set of assertFunctions and two harnesses that implement them, it should be quite easy to swicth synchronous tests between the two harnesses
15:16:18 [adrianba]
...there will be some tests where assertEquals isn't going to work for you
15:16:30 [adrianba]
gsnedders: most harnesses have these kind of functions
15:16:41 [adrianba]
...the main thing that varies is how you say what tests should be run
15:17:32 [adrianba]
krisk: nothing would prevent you from changing later - for example i changed from jsunit by replacing a lot of the include files
15:17:50 [jgraham]
but with an async test, you are typically deep into the harness-specific behaviour to determine what you need to write, when it gets called, how you express failure, and so on
15:17:58 [adrianba]
...in that regard, if a browser vendor didn't like how the tests ran they could make updates to run in their own way
15:18:49 [adrianba]
krisk: the basic contract between the test and the harness is all that has to be defined
15:19:01 [adrianba]
...even if you had an async test you could poll and know the test was starting to run
15:19:11 [adrianba]
...and through some setTimeouts continue to check if the work has been done
15:19:25 [adrianba]
...and at some point the harness can say the work didn't complete and move on
15:19:33 [jgraham]
That sounds like a harness-specific design
15:19:38 [adrianba]
...but in the end it needs to look for a pass or test completed result in a consistent way
15:19:52 [jgraham]
You can't just replace a test designed to work in that way with one designed to work in a different way
15:20:05 [adrianba]
plh: right now, kris you provided one?
15:20:12 [adrianba]
krisk: i wouldn't say a harness, more like a loader
15:20:21 [adrianba]
...could be extended to do more
15:20:29 [adrianba]
plh: do we need to do more? i suggest we don't need to
15:20:40 [adrianba]
...once we have plenty of tests we can focus on that if necessary
15:20:50 [adrianba]
gsnedders: once we have plenty of tests it will be harder to change
15:21:02 [adrianba]
...for example if someone gives lots of tests for a section of the spec
15:21:08 [jgraham]
FWIW my point of view is that we should write a harness and provide hooks for people to get the results out for whatever purpose they have
15:21:28 [jgraham]
e.g. customs regression-detection systems
15:21:34 [jgraham]
*custom
15:21:46 [adrianba]
plh: the reason i'm not in a hurry to develop another test harness is to avoid duplicating work
15:21:56 [adrianba]
...would rather delay discussion on test harness until june
15:22:00 [adrianba]
...see where we are then
15:22:16 [adrianba]
krisk: regardless of the harness we should define the consisten way of seeing a pass
15:22:24 [adrianba]
plh: yes, we should define assertEquals, etc
15:22:32 [adrianba]
krisk: i mean even like tag id
15:22:39 [adrianba]
gnsedders: why do we need to define that?
15:22:53 [adrianba]
krisk: a number of tests have different ways of saying the pass
15:22:57 [jgraham]
one assert != one passing test
15:23:05 [adrianba]
...at the moment mostly it is reading what is on the screen
15:23:26 [adrianba]
gsnedders: i sent e-mail an hour ago about how to programatically get this out
15:23:46 [gsnedders]
s/how to/getting this/
15:24:08 [krisk]
this post ?
15:24:10 [krisk]
http://lists.w3.org/Archives/Public/public-html-testsuite/2010May/0005.html
15:24:21 [gsnedders]
yeah
15:24:51 [adrianba]
krisk: when you call assertEquals, the implementation of that will go call something else
15:24:57 [adrianba]
...so you take the logging out of the test
15:25:06 [adrianba]
...then you can change assertEquals to do whatever you want
15:25:13 [jgraham]
So one approach is that you have a callback funtion that you call when the test has a result
15:25:16 [adrianba]
gsnedders: this is what most of the JS libraries do
15:25:33 [adrianba]
krisk: otherwise you have a consistent place - the harness knows to go look there
15:25:43 [adrianba]
gsnedders: the problem is that you have to keep polling the DOM
15:25:54 [adrianba]
...but a function means you can send it back as soon as you get the call
15:26:05 [adrianba]
...makes it easy to send back to regression tracking systems
15:26:11 [jgraham]
To be clear, a single assertEquals should not be considered a single result. One result may be the combination of several assertions
15:26:44 [adrianba]
krisk: should the harness be in control? what happens if the test blows up?
15:26:58 [adrianba]
gsnedders: the harness should catch all exceptions
15:27:27 [adrianba]
...for critical issues in a browser you won't be able to access the DOM either
15:27:32 [adrianba]
...i don't think there's a difference
15:28:01 [adrianba]
krisk: it sounds like we would want to change the DOM tests JS implementation of assertEquals to fit into a harness at some point
15:28:09 [adrianba]
gsnedders: yeah
15:28:18 [adrianba]
krisk: sounds like we have agreement
15:29:03 [krisk]
wiki -> http://www.w3.org/html/wg/wiki/Testing
15:29:03 [plh]
http://www.w3.org/html/wg/wiki/Main_Page
15:30:28 [adrianba]
TOPIC: Review Current Tests Posted To List For Approval
15:30:43 [adrianba]
krisk: we've covered this too
15:30:52 [adrianba]
TOPIC: Ask group for any upcoming tests to be approved/submitted
15:31:00 [adrianba]
krisk: is anyone thinking about submitting more tests?
15:31:29 [adrianba]
plh: one of the reasons I was asking this - do we have anything that says which areas we are looking for tests for
15:32:00 [adrianba]
...for example, progress events for video aren't implemented in some browsers - if we ask for video tests we might get tests for things that aren't stable
15:32:15 [adrianba]
...i remember we said that we wanted tests for window because that was more stable
15:32:19 [adrianba]
krisk: video would be one
15:32:26 [adrianba]
plh: yes, as long as you don't go too far
15:32:33 [adrianba]
...for example, testing the js play function is fine
15:33:24 [adrianba]
plh: i might start working on video tests, probably not in may
15:33:25 [jgraham]
Writing video tests is one way to ensure that we support async tets well :)
15:33:31 [jgraham]
*tests
15:34:02 [adrianba]
krisk: some people at microsoft have some tests that we'd like to submit soon too
15:34:14 [adrianba]
plh: i don't want to duplicate effort
15:34:24 [adrianba]
...if there are people who will submit video tests soonish
15:34:37 [adrianba]
gsnedders: i will when i have time
15:34:48 [adrianba]
...hopefully this month
15:35:17 [adrianba]
plh: video will be important - <video> gets the most buzz
15:35:35 [adrianba]
krisk: do we want to use <video> to test the async part of our requirements?
15:35:38 [adrianba]
plh: we could
15:35:42 [adrianba]
krisk: we should
15:35:49 [adrianba]
gsnedders: okay
15:36:04 [adrianba]
TOPIC: Any other business
15:36:21 [plh]
http://lists.w3.org/Archives/Public/public-html-testsuite/2010Apr/0017.html
15:36:28 [adrianba]
plh: sent a report last week about getting things moved to the W3C site
15:36:46 [adrianba]
...after discussing with system folks they would like to avoid putting thousands of tests into main site
15:36:53 [adrianba]
...running into issues with css2.1
15:37:05 [adrianba]
...will provide access to a web site by end of may
15:37:18 [adrianba]
...will have to different hosts, two different domains in fact
15:37:25 [adrianba]
...so that we can do cross-origin tests if needed
15:37:46 [adrianba]
...also looking at whether test harness runs on this server or if we need it on a third server
15:37:54 [plh]
tests.w3.org and tests.www.org
15:37:57 [krisk]
for example....
15:38:04 [krisk]
tests.w3.org/html5/
15:38:07 [adrianba]
krisk: i'd make one primary - i think one ending in w3.org is important
15:38:32 [adrianba]
plh: these sites will also host the css2.1
15:38:47 [adrianba]
...would prefer not to run php or whatever on the same site
15:38:53 [adrianba]
...prefer to run harness somewhere else
15:39:06 [adrianba]
...but it's not possible to differentiate which is the harness at the moment
15:39:12 [adrianba]
...still considering how to approach the problem
15:39:23 [adrianba]
...but if harness only uses js then not an issue
15:39:35 [adrianba]
...only an issue if we want to run server-side php or python
15:39:53 [adrianba]
gsnedders: we will want some tests eventually for, e.g. slow loading images
15:40:09 [adrianba]
plh: at some point we will need something on the server-side but not in the immediate future
15:40:23 [adrianba]
...so we should run with this idea by end of may and this is the main priority right now
15:40:27 [adrianba]
...expect more information next week
15:40:49 [adrianba]
...not going to get deployment right away because of that but will have a better solution by the end of the month
15:40:50 [adrianba]
krisk
15:40:56 [plh]
htmt5tests.w3.org
15:41:01 [adrianba]
krisk: i think this is a good starting point
15:41:33 [adrianba]
krisk: keep it simple in the short term seems good to me
15:41:42 [adrianba]
krisk: anything else?
15:42:13 [gsnedders]
q+ video format
15:42:13 [krisk]
When I push to HG now i get a permission denied error
15:42:57 [krisk]
I hit the error on 2 differnet systems
15:43:02 [krisk]
specific error
15:43:04 [krisk]
permission denied: .hg/store/lock
15:46:18 [adrianba]
ack next
15:46:49 [adrianba]
ack next
15:47:19 [adrianba]
gsnedders: for the video tests, we're going to have to use some video format
15:47:22 [adrianba]
plh: we can use two
15:47:30 [adrianba]
gsnedders: yes, we can use two
15:47:42 [plh]
http://www.w3.org/2008/12/dfxp-testsuite/web-framework/START.html
15:47:44 [adrianba]
plh: this is similar to what happened in the timed text working group
15:48:01 [adrianba]
...there is a video and it is available in multiple formats
15:48:07 [adrianba]
...it's not a big deal
15:48:39 [adrianba]
...we could use the same videos - they were done for the purpose of testing the timing
15:48:52 [adrianba]
...but it doesn't contain sound
15:48:58 [adrianba]
gsnedders: we have test videos too
15:49:14 [adrianba]
plh: yes, if people want to submit videos as long as they have the rights we can use them
15:49:29 [adrianba]
...and we can transform into whatever format we need to
15:49:41 [adrianba]
krisk: sounds like we have agreement that we'll have to support different formats
15:49:56 [adrianba]
...and plh will help with converting
15:50:08 [adrianba]
plh: we will run into some problems for example testing the src attribute
15:50:35 [adrianba]
...but we could have some JS hacks testing which the browser supports
15:50:54 [adrianba]
krisk: any other business?
15:52:04 [adrianba]
krisk: okay, meeting adjourned
15:52:10 [Zakim]
-gsnedders
15:52:11 [Zakim]
-Plh
15:52:12 [Zakim]
-[Microsoft]
15:52:14 [Zakim]
HTML_WG(HTMLT)11:00AM has ended
15:52:16 [Zakim]
Attendees were Plh, +04613479aaaa, gsnedders, adrianba, krisk
15:52:20 [adrianba]
rrsagent, make minutes
15:52:20 [RRSAgent]
I have made the request to generate http://www.w3.org/2010/05/04-htmlt-minutes.html adrianba
15:52:28 [adrianba]
rrsagent, make logs public
17:00:32 [plh]
plh has left #htmlt
17:54:32 [Zakim]
Zakim has left #htmlt
19:21:45 [krisk]
krisk has joined #HTMLT