00:43:47 <RRSAgent> RRSAgent has joined #testing
00:43:47 <RRSAgent> logging to https://www.w3.org/2019/09/17-testing-irc
00:44:01 <zcorpan> RRSAgent: this meeting spans midnight
00:44:09 <JohnChen> present+
00:44:15 <boaz> boaz has joined #testing
00:44:15 <Hexcles> present+
00:44:22 <boaz> present+
00:44:29 <AutomatedTester> present+ David Burns, Mozilla
00:44:36 <jorydotcom> present+ Jory Burson
00:44:38 <zcorpan> present+
00:44:42 <zcorpan> present- Mozilla
00:44:47 <boaz> present+ Boaz Sender, Bocoup
00:44:54 <JohnJansen> present+
00:45:01 <Lan> Lan has joined #testing
00:45:01 <zcorpan> present- Bocoup
00:45:05 <jorydotcom> jugglinmike are you dialed in?
00:45:07 <MikeSmith> present+
00:45:10 <NavidZ_> present+ NavidZ
00:45:11 <Hexcles> RRSAgent, draft minutes
00:45:11 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
00:45:13 <foolip> jugglinmike: you can try joining now
00:45:18 <Lan> present+
00:45:28 <MikeSmith> RRSAgent, make logs public
00:45:40 <JohnJansen> RRSAgent, make minutes
00:45:40 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
00:46:05 <JohnJansen> RRSAgent, Title: Browser Testing and Tools, TPAC 2019
00:46:05 <RRSAgent> I'm logging. I don't understand 'Title: Browser Testing and Tools, TPAC 2019', JohnJansen.  Try /msg RRSAgent help
00:46:15 <MikeSmith> Present+ JoryBurson
00:46:25 <MikeSmith> Present- Burson
00:46:32 <MikeSmith> Present- Jory
00:46:32 <JohnJansen> Topic: Test Automation
00:46:42 <MikeSmith> RRSAgent, make minutes
00:46:42 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html MikeSmith
00:46:50 <JohnJansen> Title: Browser Testing and Tools, TPAC 2019
00:47:05 <foolip> jgraham: talked with WebXR people yesterday, they're seemingly the first to define a test-only API which has multiple implementations
00:47:08 <MikeSmith> RRSAgent, make minutes
00:47:08 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html MikeSmith
00:47:13 <AutomatedTester> scribenick: foolip
00:47:36 <foolip> jgraham: there might also be tests that break for the wrong reason, like assuming that the mojo stuff is there
00:48:02 <foolip> jgraham: our intent, depending on foolip et al, is to make the mojo part transparent so that you don't have to explictly included
00:48:18 <foolip> jgraham: but maybe some metadata somewhere to say for which directories that injection needs to happen
00:48:43 <foolip> jgraham: in the end, it'd work as if you set a pref and get a test-only API
00:49:11 <JohnJansen> Meeting: Web-Platform-Tests TPAC 2019
00:49:23 <foolip> jgraham: that probably covers it for mojo
00:49:37 <reillyg> present+
00:49:47 <JohnJansen> RRSAgent, make minutes
00:49:47 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
00:49:50 <odejesush> present*
00:50:20 <foolip> reillyg: want to say sorry, my work was the first to use mojo, and other have copied that. some haven't hidden their use of mojo as much
00:50:34 <foolip> reillyg: question of how such tests should fail if the browser doesn't support the API or mojo
00:51:03 <foolip> reillyg: if we can get infra improvements to get a guide for how this should work, then that's the right way forward
00:51:29 <foolip> reillyg: more long term, APIs like this should probably be webdriver extensions, but open questions about capabilities of it
00:51:44 <foolip> reillyg: but as long as we have general agreement about how to structure to not cause issues, that's fine
00:52:00 <MikeSmith> Present+ odejesush
00:52:07 <ato> Present+
00:52:14 <foolip> jgraham: if we can get to the point where we don't need an explicit mojo include, we'd just lint to check that you don't
00:52:34 <JohnJansen> RRSAgent, Title: Browser Testing and Tools, TPAC 2019
00:52:34 <RRSAgent> I'm logging. I don't understand 'Title: Browser Testing and Tools, TPAC 2019', JohnJansen.  Try /msg RRSAgent help
00:52:42 <ato> Meeting: Web Platform Tests, Day 2, TPAC 2019
00:52:45 <foolip> reillyg: the pattern you can see in wpt isn't the way it should be
00:53:01 <foolip> reillyg: the most contentious was scripts just included mojo using <script>
00:53:09 <ato> RRSAgent: make minutes
00:53:09 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
00:53:32 <foolip> reillyg: but for scripts that detect if mojo is available, that then loads mojo stubs, is that acceptable, or should impl and files be moved to elsewhere?
00:53:42 <foolip> q?
00:53:45 <foolip> q+ boaz
00:53:47 <boaz> q+
00:54:02 <JohnJansen> q?
00:54:04 <wanming> wanming has joined #testing
00:54:15 <foolip> jgraham: it should be possible to make it almost completely transparent, so that you can just use the test API
00:54:25 <Zakim> Zakim has joined #testing
00:54:29 <foolip> jgraham: in the short term the pattern of check if mojo is available seems reasonable
00:54:31 <zcorpan> q+ boaz
00:55:01 <foolip> jgraham: a pattern that we can point to as "this is what you should do" then that's a good step on the way
00:55:24 <reillyg> q+
00:55:24 <foolip> Hexcles: sounds about right
00:55:50 <foolip> Hexcles: general principle is that /gen/ shouldn't be checked into wpt, and we're looking at ways at archiving and releases those somewhere, along with Chromium
00:55:57 <ato> RRSAgent: make minutes
00:55:57 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
00:56:03 <foolip> Hexcles: as for impl of test APIs, impl itself can be in wpt I think
00:56:03 <BitBot> (14wpt) [PR] lukebjerring 03merged 13#19106 into 07master: Update interfaces/geometry.idl - https://git.io/JeOBX
00:56:05 <jorydotcom> JohnJansen did you also have a question
00:56:08 <MikeSmith> q?
00:56:15 <jorydotcom> ack boaz
00:56:15 <MikeSmith> ack boaz
00:56:23 <foolip> boaz: hi Mike
00:56:25 <zcorpan> q+
00:56:29 <ato> RRSAgent: This meeting spans midnight
00:56:37 <zcorpan> q?
00:56:41 <MikeSmith> ack reillyg
00:56:43 <boaz> q?
00:56:51 <plh> plh has joined #testing
00:57:17 <foolip> reillyg: only concern is I want to avoid it being too magical, figuring out how and why they got loaded was a bit of a journey
00:57:24 <foolip> reillyg: needs to be clearly at the top of a test
00:57:50 <yigu> yigu has joined #testing
00:58:09 <foolip> jgraham: in other browsers that don't do the same thing, it wouldn't be accurate
00:58:24 <foolip> reillyg: but everyone would have it implemented somehow, and pointing out how it works would be good
00:58:33 <foolip> jgraham: maybe a meta flag for the manifest
00:58:41 <foolip> jgraham: for gecko it'd be behind a pref
00:58:57 <foolip> reillyg: if I'm developing in gecko, I'd like to know what pref I have to enable
00:59:02 <foolip> q?
00:59:30 <jugglinmike> q+
00:59:31 <foolip> jgraham: we could do a <meta name="something"> to communicate
00:59:40 <ella> ella has joined #testing
00:59:53 <Hexcles> ack zcorpan
01:00:04 <reillyg> ack jugglinmike
01:00:26 <foolip> jugglinmike: last year part of the discussion was whether APIs would be exposed to web devs, a compelling reason to use webdriver
01:00:37 <foolip> jugglinmike: if in Chromium it's done by injecting code that won't happen
01:00:43 <foolip> jugglinmike: wonder if that's still a goal
01:00:56 <reillyg> q+
01:01:06 <foolip> jugglinmike: can we reach a state where we just run the browser with a flag?
01:01:22 <foolip> jgraham: agreement that where we can use webdriver that is better
01:01:29 <karl> karl has joined #testing
01:01:38 <foolip> jgraham: but some of these APIs can add significant binary bloat, increasing download size
01:02:14 <foolip> jgraham: adding to webdriver is always good, historically we have cases tested internally that should have been tested using webdriver
01:02:39 <foolip> jgraham: but there will always be cases where it's not possible or expedient to figure out the webdriver API for a specific thing
01:02:42 <foolip> q+
01:03:03 <foolip> jgraham: probably makes sense to have some test-only APIs in some cases long term
01:03:31 <foolip> reillyg: the use of mojo is out of expedience and development speed, adding webdriver surface is a lot of work
01:03:34 <boaz> q+
01:03:54 <foolip> reillyg: first cut using mojo will be easier. would like to see webdriver for some of these things, but value isn't there for all of them
01:04:05 <james> james has joined #testing
01:04:18 <foolip> reillyg: for webusb, what I need to test in depth is very different to what a web developer needs to test their code
01:04:34 <foolip> reillyg: I want to mock the underlying OS, they want do mock the web platform
01:05:07 <reillyg> s/mock the web platform/mock user interaction with permissions UI/
01:05:10 <foolip> jgraham: for webxr, in the long term probably want some automation solution, but hard to get that done quickly
01:05:29 <foolip> q?
01:05:36 <foolip> q- reillyg
01:05:40 <xiaoqian> xiaoqian has joined #testing
01:05:43 <Hexcles> ack foolip
01:06:13 <foolip> foolip: what guidance should we give to people who want to know what path to take: webdriver of test-only API?
01:06:32 <foolip> jgraham: I think, prefer webdriver, and if you're doing test-only API it needs to have a spec
01:06:49 <foolip> jgraham: it's not an invitation to expose internal guts that can't be implemented by anyone else
01:07:10 <foolip> jgraham: haven't had cases of multi-vendor interest until now, webxr being the first.
01:07:23 <zcorpan> ato: we can use the same https://docs.google.com/spreadsheets/d/1cqPK6ze2OCLsho4twJHNLZUPktfejIiiDlMwv0TaZBg/edit?usp=sharing
01:07:38 <foolip> jgraham: when we do get more interest, I'm sure we'll get feedback on test-only APIs accidentally relying on internal details
01:07:58 <Hexcles> ack boaz
01:07:59 <foolip> jgraham: you should look to get as much feedback on your test-only API as the rest of the spec
01:08:30 <foolip> boaz: in Chrome, how do you decide whether to pick webdriver or test-only API?
01:08:40 <foolip> foolip: we don't have guidance and don't know, that's why I ask :)
01:09:33 <foolip> reillyg: writing the test-only API was informative for crystalizing the behavior I wanted, recommend doing that
01:09:36 <foolip> q?
01:09:41 <jugglinmike> q
01:09:55 <jugglinmike> rather
01:09:56 <jugglinmike> q+
01:10:56 <foolip> jgraham: it would be *nice* to know we don't depend on insufficiently defined API
01:10:59 <Hexcles> s/rather//
01:11:08 <foolip> jgraham: when the second impl comes along turns out you have to make a lot of changes
01:11:21 <JohnJansen> q?
01:11:24 <foolip> jgraham: I imagine we'll have the same problem with test-only APIs
01:11:34 <foolip> boaz: anything we can do to avoid that?
01:11:34 <ato> s/Web-Platform-Tests TPAC 2019/Web Platform Tests, Day 2, TPAC 2019/
01:11:44 <foolip> boaz: like not contributing tests that can only run in one browser?
01:11:55 <zcorpan> q?
01:11:56 <MikeSmith> q?
01:12:03 <foolip> jgraham: think that's difficult, deeper question is how valuable are tests for features with only one impl?
01:12:04 <Hexcles> ack jugglinmike
01:12:04 <MikeSmith> ack jugglinmike
01:12:10 <reillyg> q+
01:12:17 <foolip> jugglinmike: that's the real interesting thing
01:12:31 <foolip> jugglinmike: whether webdriver or test-only best discussed with [scribe not sure]
01:12:48 <reillyg> s/[scribe not sure]/Browser Testing and Tools WG/
01:13:00 <foolip> jugglinmike: I'm not sure how much to push for/against test-only APIs
01:13:16 <foolip> jugglinmike: should we require when we add a new dir in wpt that the spec says something about automation?
01:13:27 <ato> q+
01:13:28 <foolip> jugglinmike: that may be too drastic, but would be good with more clarity for reviewers
01:13:52 <CalebRouleau> CalebRouleau has joined #testing
01:13:54 <foolip> jgraham: people probably agree that it's OK to complain about tests using non-std APIs
01:14:13 <CalebRouleau> present+
01:14:16 <foolip> jgraham: put people determined to write tests for their experimental feature, hard to stop them from doing that
01:14:23 <foolip> jgraham: difficult to do that gatekeeping
01:14:29 <boaz> q?
01:14:36 <boaz> q+
01:14:39 <foolip> jgraham: if there are cases that have a good spec for the web-exposed part but not for the testing we should push back
01:14:51 <Hexcles> ack reillyg
01:15:11 <foolip> reillyg: I'd be sad to this group reject tests for features that aren't standardized yet
01:15:42 <MikeSmith> q?
01:15:48 <foolip> jgraham: by "standardized" I mean just "spec exists", as opposed to just an explainer. this isn't about the standards track, it's about making an attempt to make a standard
01:16:24 <foolip> reillyg: in early stages of development you'll write some code, some tests, some spec, seems reasonable to have tests before there's a formal spec
01:16:35 <foolip> reillyg: whether those are in wpt or kept private, is a reasonable question
01:16:49 <foolip> q?
01:17:06 <jorydotcom> jorydotcom has joined #testing
01:17:13 <foolip> reillyg: I would like to see more linting, testing that we aren't depending on random things
01:17:28 <foolip> reillyg: some level of linting to check you aren't assuming things that aren't well-defined
01:17:34 <zcorpan> q?
01:17:52 <foolip> foolip: tentative tests are a thing
01:17:59 <foolip> jgraham: but not tentative directories
01:18:19 <foolip> ato: the webdriver group has engaged with other groups, like permissions
01:18:30 <foolip> ato: returning problem is lack of motivation for implementation
01:18:57 <foolip> ato: webdriver covers an enormous part of the browser, so people on webdriver don't have expertise to implement things specific to say webxr
01:19:12 <foolip> ato: should encourage tests to think hard about testing, but some things are very hard with current webdriver protocol
01:19:22 <foolip> ato: some things also might not make sense to test using webdriver
01:19:49 <foolip> ato: jugglinmike mentioned what would be useful for web devs also, part of our responsibility to think about that
01:19:49 <zcorpan> ack ato
01:19:54 <zcorpan> ack boaz
01:20:24 <foolip> boaz: agree that writing tests early is great, and that the material should be shared, last thing we want to do is to keep that from happening
01:20:34 <foolip> boaz: for some of these experiments, would another place make sense?
01:20:38 <foolip> q+
01:20:52 <Hexcles> q+
01:20:58 <foolip> boaz: for tests that depend on things that are proprietay
01:21:19 <foolip> jgraham: two cases. if you depend on proprietary APIs don't upstream.
01:21:35 <foolip> jgraham: ... if you're depending on something you have no intent to standardize
01:21:44 <rniwa> rniwa has joined #testing
01:21:46 <foolip> jgraham: but for something like std-toast the tentative test mechanism works
01:22:02 <foolip> boaz: but isn't there material that can't be run in all browser?
01:22:29 <foolip> jgraham: unclear if there are tests that will never work even if they implement the feature. that's the line: would this be testable if the browser implemented the spec as written?
01:22:32 <foolip> q?
01:22:36 <foolip> q-
01:23:43 <foolip> jgraham: there have been cases where changes make the tests wrong and not per spec, browser-specific by accident
01:24:02 <foolip> Hexcles: a bit concerned about adding yet another place for staging
01:24:10 <foolip> Hexcles: in chromium we already have wpt_internal
01:24:20 <jorydotcom> ack Hexcles
01:24:21 <marcosc> marcosc has joined #testing
01:24:22 <foolip> Hexcles: this is for things that are very early stage or unlikely to be testable
01:24:37 <foolip> Hexcles: a level between that and wpt is too much to organize
01:24:45 <foolip> Hexcles: but tentative directories is reasonable
01:24:59 <foolip> Hexcles: like if you don't yet have a spec draft, mark the whole directory as tentative
01:25:02 <ato> Hexcles: ++
01:25:02 <jgraham> q+
01:25:19 <foolip> Hexcles: agree with jgraham that the bar is if it could be tested in other browsers, that's good enough for wpt
01:25:23 <zcorpan> q+
01:25:40 <foolip> Hexcles: need not require that it can already be tested in multiple impls
01:25:44 <foolip> https://cs.chromium.org/chromium/src/third_party/blink/web_tests/wpt_internal/
01:25:58 <foolip> jgraham: we have the same thing, a directory for things we can't export for some reason
01:26:12 <foolip> q?
01:26:23 <JohnJansen> q?
01:26:28 <foolip> jgraham: there may also be a wpt.fyi feature request to search for tentative tests, other than in the test name
01:26:30 <Hexcles> ack jgraham
01:26:59 <foolip> jgraham: might be a wpt.fyi feature request to filter by tentative
01:27:01 <JohnJansen> q+
01:27:04 <foolip> `!tentatve` should work
01:27:05 <jgraham> ack zcorpan
01:27:21 <foolip> zcorpan: what's the effect of having tentative tests?
01:27:24 <foolip> jgraham: informative only
01:27:32 <foolip> zcorpan: so not different in wpt.fyi
01:27:53 <foolip> zcorpan: wonder what tentative directories would be for?
01:27:59 <foolip> q?
01:28:11 <foolip> JohnJansen: hello jugglinmike
01:28:21 <foolip> JohnJansen: a long time ago we had vendor folders in wpt
01:28:23 <Hexcles> ack JohnJansen
01:28:24 <foolip> jgraham: *laugh*
01:28:45 <foolip> JohnJansen: the idea itself was sound, an indication they weren't meant to run in other browsers
01:28:51 <ato> q+
01:28:58 <foolip> JohnJansen: sounds similar to tentative folder or another staging repository
01:29:19 <foolip> https://wpt.fyi/results/old-tests/submission/Microsoft
01:30:24 <zcorpan> q+
01:30:31 <foolip> JohnJansen: coming up with ideas here for how to store non-all-browser tests is irrelevant if we can't enforce that all tests follow that pattern
01:30:37 <boaz> q?
01:30:53 <foolip> jgraham: rule for wpt is that in theory all tests should run in all browsers
01:31:06 <JohnJansen> q?
01:31:11 <JohnJansen> +1
01:31:16 <foolip> jgraham: let's clear the queue and break
01:31:48 <NavidZ_> q+
01:32:11 <Hexcles> ack ato
01:32:14 <foolip> ato: I'm not aware of the history of vendor directories, but in principle sounds like a good idea to run other's tests
01:32:45 <foolip> ato: in Mozilla have tests in mozilla-central where we might want to reuse the wpt harness
01:32:53 <tidoust> tidoust has joined #testing
01:33:02 <foolip> ato: and turns out some of the wpt tooling is quite good
01:33:07 <foolip> q?
01:33:08 <JohnJansen> q?
01:33:21 <foolip> zcorpan: in response to JohnJansen, how to enforce?
01:33:30 <foolip> zcorpan: one way is lint checker, which jugglinmike already did for mojo tests
01:33:33 <JohnJansen> RRSAgent, make minutes
01:33:33 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
01:33:36 <Hexcles> ack zcorpan
01:33:44 <jgraham> Zakim: close the queue
01:33:49 <foolip> zcorpan: think jgraham reviewed that, so we have a lint already for mojo tests
01:33:54 <jgraham> close the queue
01:34:31 <foolip> zcorpan: we may need a policy written down
01:34:48 <foolip> foolip: we also have a lint for LayoutTests-specific APIs
01:35:10 <zcorpan> q?
01:35:10 <Hexcles> ack NavidZ_
01:35:15 <foolip> boaz: can we have an action item for that?
01:35:22 <foolip> foolip: you have edit access now :)
01:35:53 <foolip> NavidZ_: webdriver pointer actions, we have that now.
01:36:09 <foolip> NavidZ_: but some things depend on the browser or the platform, like disabling flinging
01:36:24 <foolip> NavidZ_: because I don't want the page to scroll along some random curve
01:36:49 <foolip> NavidZ_: second is platform-specific things like #ifdefs in the code
01:37:06 <foolip> NavidZ_: browsers match platform conventions, e.g. with how stylus works
01:37:24 <foolip> NavidZ_: context menus are also different on different platforms
01:37:33 <foolip> NavidZ_: should we expose those differences to tests?
01:38:05 <zcorpan> https://github.com/web-platform-tests/wpt/pull/18509 is the lint check PR
01:38:20 <shimazu> shimazu has joined #testing
01:38:24 <zcorpan> break till 11
01:38:32 <Hexcles> Hexcles has joined #testing
01:44:01 <reillyg> reillyg has joined #testing
01:44:35 <Orphis> Orphis has joined #testing
01:48:13 <odejesush> odejesush has joined #testing
01:58:02 <shimazu> shimazu has joined #testing
01:59:27 <spectranaut> spectranaut has joined #testing
02:01:18 <Hexcles> Hexcles has joined #testing
02:03:02 <zcorpan> zcorpan has joined #testing
02:03:12 <BitBot> (14wpt) [PR] chromium-wpt-export-bot requested 13#19107 merge into 07master: Port perspective and perspective-origin interpolation tests to wpt - https://git.io/JeORo
02:03:36 <james> james has joined #testing
02:04:12 <shimazu> shimazu has joined #testing
02:06:12 <BitBot> (14wpt) [PR] chromium-wpt-export-bot 03merged 13#19016 into 07master: Expand 'autofocus' attribute support to all of HTML and SVG elements. - https://git.io/JemyF
02:09:04 <AutomatedTester> scribenick: AutomatedTester
02:09:05 <kevers> kevers has joined #testing
02:09:12 <JohnJansen> https://github.com/web-platform-tests/rfcs/pull/24
02:09:15 <kevers> present+
02:09:15 <JohnJansen> present+
02:09:32 <JohnJansen> Topic: RFC: Test Editor
02:09:46 <foolip> q?
02:10:08 <AutomatedTester> foolip: this session is for the rfc for the test editor
02:10:12 <jorydotcom> jorydotcom has joined #testing
02:10:17 <foolip> q?
02:10:55 <fremy> fremy has joined #testing
02:11:01 <jgraham> scribe AutomatedTester
02:11:27 <marcosc> marcosc has joined #testing
02:11:30 <ato> Reminder, this is the current seat map: https://docs.google.com/spreadsheets/d/1cqPK6ze2OCLsho4twJHNLZUPktfejIiiDlMwv0TaZBg/edit#gid=0
02:11:39 <AutomatedTester> fremy: let's talk about the history of this rfc. Part of my work there was a need to quickly make wpt. This would be from reducing a webcompat issues
02:12:04 <AutomatedTester> ... one of my ideas was to use something like jsfiddle and would be quick and easy to make wpt
02:12:14 <AutomatedTester> ... there would just have the basics for the doc
02:12:30 <AutomatedTester> ... and then you can add simple assertions.
02:12:54 <AutomatedTester> fremy: people do not want to have to fork wpt/ github repo. They just want to quickly make a test.
02:13:05 <ato> RRSAgent: please make the minutes
02:13:05 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
02:13:35 <AutomatedTester> fremy: you can go from "i need to create a quick test" to "here is a PR" and make it seemless as possible
02:13:57 <ato> ScribeNick: AutomatedTester
02:14:04 <AutomatedTester> ... this is how it started and as part of CSSWG we have expanded it
02:14:40 <AutomatedTester> JohnJansen: there was a request from several people outside of MS to move it away from MS to wpt
02:15:02 <AutomatedTester> ... I have renamed the repo to editor
02:15:15 <AutomatedTester> ... and we have been getting a bunch of questions on how we can use it
02:15:21 <ato> q?
02:15:32 <foolip> q?
02:15:42 <AutomatedTester> ... the request then to see if we can move forward and what is required to close the rfc
02:16:06 <AutomatedTester> foolip: is there someone to willing to deploy and maintain it? including infra?
02:16:11 <CalebRouleau> CalebRouleau has joined #testing
02:16:21 <rniwa> rniwa has joined #testing
02:16:35 <AutomatedTester> JohnJansen: we have modified the azure instance to do all of this
02:16:36 <fremy> https://wptest.center
02:16:43 <foolip> q?
02:16:47 <AutomatedTester> foolip: is there someone to work on this?
02:16:54 <JohnJansen> https://github.com/web-platform-tests/editor
02:16:54 <AutomatedTester> JohnJansen: yes
02:17:10 <JohnJansen> https://wptest.center/#/new
02:18:00 <ato> Present + fremy
02:18:05 <AutomatedTester> fremy: what does it take to make this project useful for people at large? Workflow changes? Is there value for this group? How can we manage the contribution story
02:18:26 <AutomatedTester> ... I am happy to continue to contributing this and make the changes to make it work
02:18:39 <jorydotcom> q+
02:18:44 <AutomatedTester> ... but it would be good to make sure that there are other vendors willing to participate
02:18:53 <JohnJansen> q?
02:19:23 <AutomatedTester> jorydotcom: This moved out of MS github to wpt github.
02:19:37 <foolip> ack jorydotcom
02:19:39 <foolip> q?
02:19:43 <AutomatedTester> ... have we done all the due diligence on the move
02:19:49 <AutomatedTester> JohnJansen: yes, MS legal agreed
02:20:23 <AutomatedTester> JohnJansen: I have been using it to test reductions that I have made on web compat issues
02:20:55 <AutomatedTester> JohnJansen: we were in a state where reductions were more important and therefore no one was doing wpt for these issues
02:21:56 <AutomatedTester> jgraham: the people in this group are less likely to use this project and this might be the wrong group to figure out what is missing and what features could be missing
02:22:38 <foolip> q?
02:22:42 <AutomatedTester> ... having said that speaking to the webcompat or MDN people might mean the feature requests would come from people adjacent to this group
02:23:07 <MikeSmith> q?
02:23:23 <AutomatedTester> JohnJansen: the request in the RFC is to make editor available from this group for other people to use
02:23:37 <zghadyali> zghadyali has joined #testing
02:23:44 <Lan> Lan has joined #testing
02:24:16 <AutomatedTester> ... we could use take it and then see if in the year that is great then we support it or if it doesnt then just archive the repo
02:25:40 <AutomatedTester> fremy: the CSSWG like and we have changed the process to start changing a spec then it needed a test. We used this process to remove the friction in getting new tests. TabAtkins has used this in their workflow and seems happy so far
02:26:10 <AutomatedTester> ... we need to make tools to allow editors that they dont need to change their workflow to make more wpt
02:26:25 <BitBot> (14rfcs) [issue] boazsender opened 13#30: Create an RFC for what material is and is not allowed in WPT - https://git.io/JeOR5
02:26:26 <xiaoqian> xiaoqian has joined #testing
02:26:37 <AutomatedTester> ... so if we can make this more official then people are likely to use it more
02:27:16 <jgraham> q+
02:29:13 <foolip> q?
02:29:17 <foolip> ack jgraham
02:29:47 <AutomatedTester> jgraham: there is a community building thing here. people could use but there are "things" that need fixing. But if we can build a community and see how it goes. We can link to the docs but not say it is the prefered way
02:30:09 <AutomatedTester> foolip: is going to need someone to support it?
02:30:22 <AutomatedTester> fremy: is MS going to support it?
02:31:07 <AutomatedTester> JohnJansen: it depends on how much work but we can have a look to see how it goes. If MS fails in the contribution then we can re-evaluate it
02:31:36 <AutomatedTester> foolip: if it is going to be called a editor then it needs to fix tests are already in the repo
02:32:22 <AutomatedTester> fremy: this is currently being hosted on its own environment to manipulate it. E.g. slow network.
02:32:33 <AutomatedTester> foolip: so edit and then run on the CI.
02:32:37 <AutomatedTester> fremy: yes
02:33:08 <AutomatedTester> jgraham: if you are writing a test then you can use wpt-test.live to host it
02:33:37 <AutomatedTester> jgraham: you can reuse server side features this way
02:34:12 <AutomatedTester> foolip: unless you are changing the resources in the backend you can do a lot of tests now
02:35:13 <AutomatedTester> jgraham: [describes how the resources could work as an example and use the wpt]
02:35:43 <AutomatedTester> fremy: for a new test it is nice to have the test resouces (css, js) split and easy to use
02:35:56 <AutomatedTester> q?
02:36:54 <AutomatedTester> foolip: to resolve this RFC, we do have some questions that need answering
02:36:58 <TabAtkins> While there are lots of cool things the editor could do as well, I can day with confidence that its current functionality is extremely useful for writing tests for myself. It's surprisingly hard to do the test-commit dance by hand.
02:37:09 <AutomatedTester> ... like CoC.
02:37:28 <AutomatedTester> JohnJansen: there was a question about the reference to mongodb
02:37:53 <AutomatedTester> Hexcles: there is an issue with google people needing to have mongo and we cant
02:38:08 <plh> plh has joined #testing
02:38:30 <AutomatedTester> fremy: you could have it locally but you don't really use it. in azure we use a doc storage
02:38:55 <AutomatedTester> fremy: you just need the mongodb protocol
02:39:16 <AutomatedTester> Hexcles: is there other alternatives that use mongodb protocol
02:39:25 <AutomatedTester> fremy: yes
02:39:44 <AutomatedTester> foolip: so this could be abstracted away
02:40:05 <AutomatedTester> foolip: but it uses mongodb npm package?
02:40:16 <AutomatedTester> fremy: thats the compat issue
02:40:22 <AutomatedTester> jgraham: who implemented it?
02:41:02 <AutomatedTester> Hexcles: we can just double check with legal but we can't install the database
02:41:31 <AutomatedTester> foolip: there are details that we dont but we need to figure this out
02:42:14 <AutomatedTester> [discussion about what database needs to be installed locally]
02:43:13 <AutomatedTester> fremy: we dont need a database locally. We just needed something that was able to store documents and to make sure that it doesn't limit you to the MS cloud document storage
02:43:20 <xiaoqian> xiaoqian has joined #testing
02:44:11 <AutomatedTester> foolip: if you run the tests locally with no mongodb it just errors so we can talk to legal
02:44:21 <AutomatedTester> foolip: so it jkust needs a kv storage
02:44:29 <AutomatedTester> fremy: yes, its really simple
02:45:39 <AutomatedTester> fremy: the use of the database is for login and storing the tests during dev
02:45:50 <AutomatedTester> AutomatedTester: could we move to sqlite?
02:45:56 <AutomatedTester> jgraham: yes but the model is different
02:46:40 <AutomatedTester> fremy: for no database we could just write to files, thats how it used to work. It's really really simple
02:47:02 <foolip> q?
02:47:12 <AutomatedTester> foolip: there is the CLA question
02:47:28 <AutomatedTester> fremy: we should delete it
02:47:32 <AutomatedTester> JohnJansen: agreed
02:48:03 <AutomatedTester> JohnJansen: we can create individual issues on the repo and approve this rfc
02:48:22 <shimazu> shimazu has joined #testing
02:48:25 <AutomatedTester> foolip: we just to make sure that some things are done in order
02:48:35 <zcorpan> zcorpan has joined #testing
02:49:21 <AutomatedTester> foolip: things that need to happen. 1) change the wpt CoC 2) delete the cla 3) pick a URL
02:49:32 <AutomatedTester> 3) will not block the RFC
02:50:06 <AutomatedTester> foolip: is there a hostname we should use
02:50:35 <AutomatedTester> jgraham: we can have it under web-platform-tests.org
02:51:07 <AutomatedTester> foolip: so editor.web-platform-tests.org
02:52:00 <JohnJansen> rrsagent, make minutes
02:52:00 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
02:52:24 <AutomatedTester> fremy: MS is going continue to do hosting
02:52:42 <Boaz> Boaz has joined #Testing
02:52:44 <AutomatedTester> ... and then TabAtkins, Anton and I will continue to look at it
02:52:46 <Boaz> Present+ Boaz Sender, Bocoup
02:53:10 <AutomatedTester> foolip: is there going to be a big change to make it an editor and not just a reduction tool
02:53:14 <ato> RRSAgent: make minutes
02:53:14 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
02:53:45 <ato> Present- Bocoup
02:54:05 <ato> Present- boaz
02:54:10 <AutomatedTester> fremy: you would need to have a file list
02:54:19 <ato> RRSAgent: make minutes
02:54:19 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
02:54:29 <AutomatedTester> [technical discussion on editor and how that would work]
02:55:00 <AutomatedTester> ACTION JohnJansen to file issues for 1) change the wpt CoC 2) delete the cla
02:55:23 <ato> RRSAgent: Please make the minutes
02:55:23 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
02:55:25 <foolip> q?
02:55:27 <jgraham> q?
02:55:30 <AutomatedTester> ACTION JohnJansen to paste links to the minutes
02:56:19 <Boaz> http://web-platform-tests.live/
02:56:25 <AutomatedTester> lukebjerring: one thing to mention, there is an implicit flow and how it copies it to web-platform-tests.live
02:56:27 <Boaz> will be http://wpt.live
02:56:35 <ato> ACTION: JohnJansen to file issues for (1) change the WPT CoC (2) delete the CLA
02:57:31 <foolip> q?
02:57:31 <ato> s/ACTION JohnJansen to paste links to the minutes//
02:57:59 <AutomatedTester> ...the flow for http://web-platform-tests.live/ will allow resources to work for people
02:58:01 <ato> RRSAgent: Please draft the minutes
02:58:01 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
02:58:10 <AutomatedTester> fremy: I was unaware of this process
02:58:35 <AutomatedTester> ... we can discuss how this could work and make/fix features in the editor
02:58:48 <foolip> q?
03:01:00 <AutomatedTester> Topic: CoC
03:02:10 <zcorpan> github: https://github.com/web-platform-tests/rfcs/pull/17
03:02:19 <AutomatedTester> Boaz: there was a PR for the CoC that was opened a year ago. I and jorydotcom are happy to be points of contact
03:02:25 <marcosc> marcosc has joined #testing
03:02:32 <AutomatedTester> ... and we need to get consesus on the RFC
03:02:54 <foolip> q?
03:03:11 <AutomatedTester> ... and make wpt a good place to contribute
03:03:42 <foolip> q+
03:03:43 <foolip> q?
03:03:43 <AutomatedTester> jorydotcom: I spoke to gsnedders about some of the process. They had concerns about 2 people from the same company
03:04:31 <AutomatedTester> jgraham: and I have spoke to gsnedders about this and one of the concerns was that for a CoC enforcing group would need to reflect the diversity that we want and have a large diverse group
03:05:15 <foolip> q?
03:05:26 <AutomatedTester> ... we need to make sure that we cover the diversity groups covered and while the people offering are more diverse than this group but it doesnt reflect where we want to be
03:05:56 <AutomatedTester> jorydotcom: absent of the diversity we want the people on the group need to make sure that the relevant people have the relevant training
03:06:01 <foolip> q?
03:06:25 <AutomatedTester> Boaz: the right thing to do is to also add the process for people to voluteer on the CoC group
03:06:53 <AutomatedTester> foolip: is the RFC process for adding/removing people
03:07:06 <AutomatedTester> jgraham: that is unclear. We need to have that properly documented
03:07:34 <AutomatedTester> ... and not everything needs to be documented in the public
03:08:15 <AutomatedTester> ... e.g. if there are concerns for the moderating being in the team that doesnt need to be in the public sphere.
03:08:40 <AutomatedTester> plh: we have moderates that feel supported to jump in and make the relevant comments to protect the group
03:09:04 <AutomatedTester> plh: and it shouldnt really be from the core team
03:10:25 <AutomatedTester> jorydotcom: I can share the TC39 CoC how that works. People can email saying that would like to join and then the CoC team will discuss and then CoC team emails a private list for TC39 only to say who is being added
03:11:01 <AutomatedTester> jgraham: that process works for TC39 since they are closed membership where we dont have that process and tooling for that
03:11:59 <AutomatedTester> jgraham: we dont have a closed mailing list
03:12:06 <zcorpan> zcorpan has joined #testing
03:13:08 <AutomatedTester> jgraham: we would need to have a process that someone is nominated, self nominated is fine, and return feedback would come back. The announce needs to be public but the feedback is private
03:13:14 <MikeSmith> q+ to comment
03:13:20 <foolip> q?
03:13:28 <AutomatedTester> ack foolip
03:13:33 <foolip> ack foolip
03:13:34 <foolip> ack MikeSmith
03:13:34 <Zakim> MikeSmith, you wanted to comment
03:13:37 <Boaz> q+
03:13:37 <foolip> q?
03:13:51 <zcorpan> zcorpan has joined #testing
03:13:58 <AutomatedTester> MikeSmith: I want to ask if the moderates responding is private?
03:13:59 <zcorpan> q?
03:14:04 <foolip> ack Boaz
03:14:11 <AutomatedTester> Boaz: no this is for the process for joinging the moderates
03:14:12 <zcorpan> q+
03:14:22 <JohnJansen> RRSAgent, make minuts
03:14:22 <RRSAgent> I'm logging. I don't understand 'make minuts', JohnJansen.  Try /msg RRSAgent help
03:14:25 <JohnJansen> RRSAgent, make minutes
03:14:25 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
03:14:30 <AutomatedTester> Boaz: this sounds like we need to make the process and move it forward
03:14:59 <AutomatedTester> Boaz: since we are from the same company we would like to get more people to join so it is not just us
03:15:37 <AutomatedTester> jgraham: I can't speak for gsnedders as they have the concerns and they would need to give it
03:16:10 <AutomatedTester> jgraham: I think that if we dont have it perfect and iterate with the idea of where we want to be then it is a good start
03:16:20 <foolip> ack zcorpan
03:16:59 <spectranaut_> spectranaut_ has joined #testing
03:17:23 <AutomatedTester> zcorpan: I want to discuss something with the moderates. What is the consequence if someone violates the CoC. Ban seems harsh and warnings is not harsh enough. We need to know how this could work
03:18:51 <AutomatedTester> jorydotcom: There is a limited set of things that we can do as a group of volunteers. You need to build a ladder from "you broke our norms and here is some docs on what we agree and we want you in the community" to a full ban. In TC39 we do have temp bans for X months and keep that in a private file.
03:19:14 <AutomatedTester> zcorpan: that is the thing that we need to record a history for a specific person on each violation
03:19:31 <AutomatedTester> ... and what was done and said to that person
03:19:52 <AutomatedTester> Boaz: we can also explicitly add addedums on what we need to do
03:19:53 <zcorpan> zcorpan has joined #testing
03:19:55 <zcorpan> zcorpan has joined #testing
03:20:10 <AutomatedTester> jgraham: the Rust community has a moderation policy on top of the CoC
03:20:43 <AutomatedTester> ... and what the ladder looks like and describes what the valid responses can be
03:21:06 <jgraham> https://www.rust-lang.org/policies/code-of-conduct#moderation
03:21:09 <AutomatedTester> ... e.g. keep responses private if you have been moderated
03:21:41 <AutomatedTester> fremy: how would you appeal?
03:21:56 <Boaz> q+
03:23:06 <AutomatedTester> fremy: if the group is small then the appeals might cause issues
03:23:14 <AutomatedTester> Boaz: next steps?
03:23:30 <AutomatedTester> AutomatedTester: we need to get gsnedders opinion on this
03:23:52 <AutomatedTester> jgraham: I am not sure the RFC process is valid for this...
03:25:03 <AutomatedTester> jgraham: the consesus is that it is not perfect but it is a good start and we need iterate on this to make sure things arent blocked
03:25:42 <AutomatedTester> ACTION: create process for adding moderators via RFC
03:26:52 <Hexcles> Hexcles has joined #testing
03:26:53 <AutomatedTester> RRSAgent: make notes
03:26:53 <RRSAgent> I'm logging. I don't understand 'make notes', AutomatedTester.  Try /msg RRSAgent help
03:35:32 <plh> plh has joined #testing
04:02:04 <Hexcles> Hexcles has joined #testing
04:02:28 <rniwa> rniwa has joined #testing
04:02:53 <cybai> cybai has joined #testing
04:06:20 <zcorpan> zcorpan has joined #testing
04:06:31 <BitBot> (14wpt) [PR] chromium-wpt-export-bot requested 13#19108 merge into 07master: Move 'autofocus' IDL attribute to HTMLOrSVGElement - https://git.io/JeO0z
04:07:20 <denis> denis has joined #testing
04:10:00 <spectranaut_> presentation: https://bocoup.github.io/presentation-aria-and-webdriver/#/
04:10:01 <shimazu> shimazu has joined #testing
04:10:04 <wanming> wanming has joined #testing
04:10:33 <spectranaut_> whoops wrong channel ^
04:23:59 <shimazu> shimazu has joined #testing
04:25:49 <marcosc> marcosc has joined #testing
04:28:25 <Hexcles> Hexcles has joined #testing
04:31:01 <denis> denis has joined #testing
04:34:51 <shimazu> shimazu has joined #testing
04:36:00 <shimazu> shimazu has joined #testing
04:36:49 <tidoust> tidoust has joined #testing
04:38:58 <zcorpan> zcorpan has joined #testing
04:40:34 <cb> thanks for sharing anyway !
04:41:49 <shimazu> shimazu has joined #testing
04:46:16 <shimazu> shimazu has joined #testing
04:49:35 <jorydotcom> jorydotcom has joined #testing
04:52:25 <zcorpan> zcorpan has joined #testing
04:53:13 <BitBot> (14wpt) [PR] moz-wptsync-bot 03merged 13#19093 into 07master: [Gecko Bug 1488530] Add more tests for clip-path:path(). - https://git.io/JeOch
04:54:24 <marcosc> marcosc has joined #testing
04:54:54 <Hexcles> Hexcles has joined #testing
04:55:07 <marcosc> marcosc has joined #testing
04:55:44 <BitBot> (14wpt) [PR] chromium-wpt-export-bot requested 13#19109 merge into 07master: Add expression representation to InterpolableLength - https://git.io/JeO0A
04:55:50 <Hexcles> RRSAgent, make minutes
04:55:50 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
04:57:05 <plh> plh has joined #testing
04:57:36 <zcorpan_> zcorpan_ has joined #testing
05:06:15 <marcosc> marcosc has joined #testing
05:06:49 <foolip> Alright, we should start the wpt.fyi session now
05:07:27 <ato> We should just start.
05:08:18 <AutomatedTester> RRSAgent: make minutes
05:08:18 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html AutomatedTester
05:08:47 <foolip> q?
05:08:52 <foolip> q- botie
05:08:55 <foolip> q- Boaz
05:09:19 <foolip> scribenick: lukebjerring
05:09:20 <AutomatedTester> scribenick lukebjerring
05:09:27 <JohnJansen> Topic: WPT.FYI features, etc
05:09:33 <zcorpan> zcorpan has joined #testing
05:09:39 <kevers> kevers has joined #testing
05:09:53 <Hexcles> RRSAgent, who's the scriber?
05:09:53 <RRSAgent> I'm logging.  Sorry, nothing found for 'who's the scriber'
05:09:55 <diemol> diemol has joined #testing
05:10:04 <JohnJansen> RRSAgent, scribe?
05:10:04 <RRSAgent> I'm logging.  Sorry, nothing found for 'scribe'
05:10:06 <ato> RRSAgent: Please draft the minutes
05:10:06 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
05:10:28 <foolip> scribenick foolip
05:10:34 <ricea> ricea has joined #testing
05:10:41 <foolip> lukebjerring: overview of the triage metadata project:
05:10:47 <ato> ScribeNick: foolip
05:10:53 <foolip> lukebjerring: we've created https://github.com/web-platform-tests/wpt-metadata parallel to wpt
05:11:03 <JohnJansen> TOPIC: WPT.FYI features, etc
05:11:18 <foolip> lukebjerring: it's out-of-tree metadata about the results, which wpt.fyi consumes to let people triage their searches
05:11:31 <foolip> lukebjerring: there's a `link:something` atom that you can filter by
05:11:59 <foolip> lukebjerring: work also underway to create UI for adding links when you spot the failures
05:12:25 <foolip> lukebjerring: had some feedback which made us revise, new plan is to use GitHub login and have cookie to do stuff on your behalf
05:12:40 <CalebRouleau> CalebRouleau has joined #testing
05:12:42 <foolip> foolip: why is it important to do anything on the user's behalf?
05:13:18 <foolip> jgraham: don't necessarily need to use GitHub API call with user's credentials, but it is a requirement that we know who the user is
05:13:31 <foolip> jgraham: wpt.fyi wants to validate with GitHub who you are
05:13:53 <foolip> lukebjerring: feature requests in CSS working group...
05:14:02 <foolip> lukebjerring: they'd like to be able to search by flags in a test
05:14:23 <foolip> lukebjerring: I've filed https://github.com/web-platform-tests/wpt.fyi/issues/1491 for labeling of results
05:14:48 <foolip> lukebjerring: then a bot could extract the flags from the wpt repo's tests
05:14:54 <foolip> lukebjerring: then search by label on wpt.fyi
05:15:11 <foolip> jgraham: similar to what we do for manifest
05:15:18 <denis> denis has joined #testing
05:15:33 <foolip> jgraham: one way would be to put it into the manifest and update from that, but we'd worry about bloating it
05:15:45 <foolip> lukebjerring: question of whether we need a separate repo
05:16:00 <foolip> lukebjerring: main reason is we expect a lot of bloat that's only for wpt.fyi
05:16:03 <foolip> q?
05:16:31 <foolip> lukebjerring: also have requests to filter by test type, and another for `<link rel="help">` links
05:16:56 <foolip> jgraham: historically the only information in the manifest is what you need to run the tests
05:17:10 <foolip> jgraham: now 18 MB or so, could easy grow much bigger
05:17:43 <foolip> lukebjerring: another kind of metadata is flakiness
05:18:50 <foolip> lukebjerring: we want a few different ways to add flaky metadata to the repo, and have that information be used for filtering in wpt.fyi
05:19:19 <foolip> AutomatedTester: does flakiness apply to all browsers?
05:19:30 <foolip> lukebjerring: it's like the triage links, you can specify which browser
05:20:31 <AutomatedTester> scribenick lukebjerring
05:20:44 <plh> plh has joined #testing
05:20:53 <rniwa> rniwa has joined #testing
05:21:36 <lukebjerring> lukebjerring: other suggestions welcome
05:21:47 <lukebjerring> jgraham: labels should be generic enough to cover most stuff
05:21:56 <ato> RRSAgent: Please draft the minutes
05:21:56 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
05:22:12 <ato> ScribeNick: lukebjerring
05:22:17 <lukebjerring> jgraham: wondering what the tradeoff for keeping it separate to wpt repo justified
05:22:43 <lukebjerring> ... could have a single job that has the right creds and only worry about the one repo
05:22:55 <lukebjerring> foolip: it's not too bad to use tokens
05:23:11 <lukebjerring> jgraham: GitHub actions give you a token more securely
05:23:26 <ato> s/scribenick lukebjerring//
05:23:30 <lukebjerring> foolip: there is an API you can call to trigger an action anyway
05:23:46 <lukebjerring> jgraham: one repo may end up being the right trade-off
05:24:01 <lukebjerring> foolip: would have to drop the files on import/export
05:24:26 <lukebjerring> jgraham: could just leave it there. info might be useful. you could atomically add a test and its metadata at the same time
05:24:52 <shimazu> shimazu has joined #testing
05:25:09 <lukebjerring> foolip: taskcluster jobs will just have to be restarted, which wouldn't be the case for one repo
05:26:14 <lukebjerring> lukebjerring: also an issue with file renames, could catch it straight away
05:27:10 <lukebjerring> jgraham: we could add lint rules etc
05:27:41 <lukebjerring> lukebjerring: it's not clear whether we would break consumers of the wpt META.yml files
05:28:13 <lukebjerring> foolip: would we nest another file tree? or re-use the existing META.yml files
05:29:43 <lukebjerring> lukebjerring: one upside of separation is grabbing the whole repo as an archive without any bloat
05:29:54 <JohnJansen> RRSAgent, make minutes
05:29:54 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
05:30:12 <lukebjerring> foolip: deriving labels from the tests themselves, having bots that update the metadata, etc... could get messy
05:31:18 <lukebjerring> jgraham: initially we were only considering human-generated stuff. now we're trying to crawl data from the tests and exposing it
05:31:21 <karl> karl has joined #testing
05:31:37 <lukebjerring> foolip: an extended manifest with derived info jgraham suggested could be an option here
05:32:12 <lukebjerring> jgraham: --extended manifest could just include extra data. if you need it you use it, if you don't, you don't
05:32:45 <lukebjerring> ... also potentially gives you a complete list of tests that should have been run, which will help with validation of reports
05:34:41 <lukebjerring> ... sounds like if we're extracting from the tests, we shouldn't have it in the META.yml files. that'd be... difficult / potentially divergent
05:34:55 <lukebjerring> ... larger version of the manifest is the better option
05:35:08 <lukebjerring> JohnJansen: what would be the extra info?
05:35:31 <lukebjerring> jgraham: spec URL(s), some parsed meta tag info
05:35:45 <eeeps> eeeps has joined #testing
05:36:05 <lukebjerring> ... reason its not done is just the sheer bloat. performance is also an issue, but can be solved with some incremental updates
05:36:36 <lukebjerring> foolip: manifest is huge, but most people don't update it, they download the latest one
05:37:23 <lukebjerring> jgraham: Windows struggles because of the amount of io
05:37:29 <BitBot> (14wpt) [PR] chromium-wpt-export-bot 03merged 13#19108 into 07master: Move 'autofocus' IDL attribute to HTMLOrSVGElement - https://git.io/JeO0z
05:38:24 <lukebjerring> jgraham: we'll just silently produce two copies of the manifest, wpt.fyi will be the only extended manifest consumer.
05:39:28 <lukebjerring> ... sounds like this addresses the needs.
05:40:16 <lukebjerring> foolip: so having the meta in wpt would solve the issues ? are we agreeing that we should converge the two?
05:40:41 <lukebjerring> lukebjerring: yep. and reuse the existing META.yml files, no need for parallel dir
05:41:07 <lukebjerring> foolip: deleting irrelevant META now becomes a bit more of an ask? lots of spam
05:42:51 <lukebjerring> Hexcles: We can have a GitHub action to archive out all the META.yml in isolation. maybe even the extended manifest in the same repo.
05:45:43 <lukebjerring> foolip: there's an item here for triaging regressions in major releases
05:46:11 <lukebjerring> lukebjerring: what's the feature request here? helper for building version-comparison query/diff?
05:46:56 <lukebjerring> Hexcles: difficulty is that WPT is changing over time, as well as the browsers
05:47:10 <lukebjerring> jgraham: just compare stable and beta ongoingly.
05:47:30 <lukebjerring> ... could come up with a deliberate overlap for older stable versions for a while.
05:47:53 <lukebjerring> Hexcles: we have different flags for beta. what we need is apples to apples comparison
05:48:13 <BitBot> (14wpt) [PR] fred-wang requested 13#19110 merge into 07master: Add MathML tests to check that legacy mstyle attributes are ignored. - https://git.io/JeOEw
05:48:22 <lukebjerring> jgraham: We've tried to get beta and stable as close to vanilla as possible
05:48:39 <zcorpan_> zcorpan_ has joined #testing
05:49:01 <lukebjerring> ... so far we haven't had anyone requesting this
05:49:20 <lukebjerring> ... should the bugs be triaged exactly the same? I think probably the should.
05:49:59 <shimazu> shimazu has joined #testing
05:50:04 <lukebjerring> foolip: browser specific failures are probably more useful, but the beta comparison is still good.
05:51:34 <lukebjerring> lukebjerring: let's talk about manual results submission
05:52:01 <lukebjerring> ... hard part is aggregating conflicting submissions (which was a feature request)
05:52:30 <lukebjerring> jgraham: wpt.fyi is very run-centric. submitting one result at a time is not
05:54:03 <tidoust> tidoust has joined #testing
05:56:15 <lukebjerring> JohnJansen: the model that they're using is running the test, saying pass/fail, maybe wait a long time, run another test, and then aggregate it all next to the automatically collected results
05:56:19 <plh> plh has joined #testing
05:57:00 <lukebjerring> jgraham: we can have this convo with them, but, you'll probably find that lots of manual data munging to get it all into one view that doesn't aggregate with the automatic data, they won't use it, they'll use the existing tool
05:57:37 <marcosc> marcosc has joined #testing
05:57:52 <lukebjerring> JohnJansen: they used to have a locked test suite (locked to a SHA) and they would not change the rev of the suite
05:58:07 <lukebjerring> Hexcles: so they locked multiple components to different SHAs?
05:58:29 <lukebjerring> JohnJansen: yes because they're interested in whether it most recently passed
05:59:11 <lukebjerring> jgraham: If you can take a SHA, collapse by product (single column), that would fix the way wpt.fyi treats runs separately
06:02:43 <BitBot> (14wpt) [PR] aluochromium requested 13#19111 merge into 07master: Configure chrome_android and android_webview to use server_host env option. - https://git.io/JeOED
06:04:15 <lukebjerring> JohnJansen: how would collapsing different runs in the UI handle conflicting results?
06:04:22 <lukebjerring> lukebjerring: most recent wins?
06:04:45 <lukebjerring> JohnJansen: I doubt they'd be OK with that, they'd want most authoritative wins
06:04:54 <lukebjerring> ... they actually have a voting UI
06:06:03 <Hexcles> Hexcles has joined #testing
06:06:06 <BitBot> (14wpt) [PR] fred-wang requested 13#19112 merge into 07master: Add MathML test to check that menclose@notation="radical" is not supp… - https://git.io/JeOEQ
06:07:53 <fantasai> fantasai has joined #testing
06:07:57 <fantasai> yo
06:14:19 <denis> denis has joined #testing
06:18:03 <BitBot> (14wpt) [PR] chromium-wpt-export-bot 03merged 13#19082 into 07master: [webnfc] Add tests for NFCPushOptions.ignoreRead - https://git.io/JeOL8
06:18:31 <xiaoqian> xiaoqian has joined #testing
06:19:23 <rniwa> rniwa has joined #testing
06:32:21 <tidoust> tidoust has joined #testing
06:33:25 <Hexcles> Hexcles has joined #testing
06:33:31 <JohnJansen> yo
06:34:36 <shimazu> shimazu has joined #testing
06:34:45 <ato> RRSAgent: Please draft the minutes
06:34:45 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
06:39:10 <gsnedders> RRSAgent: make the minutes
06:39:10 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html gsnedders
06:39:19 <AutomatedTester> scribenick AutomatedTester
06:40:11 <ato> ScribeNick: AutomatedTester
06:40:17 <ato> s/scribenick AutomatedTester//
06:40:36 <AutomatedTester> Hexcles: introduction: We have chromium team asking for support for QUIC. The problem is python2 doesnt have cryto issues
06:40:55 <JohnJansen> s/cryto/crypto
06:40:58 <AutomatedTester> Hexcles: we want to see if there a way that we can use a Go implementation
06:41:11 <AutomatedTester> jgraham: what is the difference for the H2?
06:41:30 <AutomatedTester> Victor: what does it have in it?
06:41:46 <AutomatedTester> jgraham: [explains details of H2 support]
06:42:06 <AutomatedTester> Victor: the QUIC and H2 are very different due to tls support
06:42:45 <AutomatedTester> Victor if you look at the QUIC impl matrix are there are 20 on there and 10 well done
06:43:06 <AutomatedTester> Victor [describes different example impl]
06:43:32 <zcorpan> zcorpan has joined #testing
06:44:04 <BitBot> (14wpt) [PR] fred-wang requested 13#19113 merge into 07master: Add MathML reftest to check that the mode attribute has no effect. - https://git.io/JeOu4
06:44:18 <zcorpan> zcorpan has joined #testing
06:44:39 <AutomatedTester> jgraham: 1 question: what are the use cases for testing are and what API are you thinking?
06:45:06 <AutomatedTester> ... traditionally we have been testing the interaction between protocol and features but not the protocol directly
06:46:04 <AutomatedTester> ... What would the test look like?
06:47:04 <ricea> ricea has joined #testing
06:47:22 <BitBot> (14wpt) [PR] fred-wang 03merged 13#19110 into 07master: Add MathML tests to check that legacy mstyle attributes are ignored. - https://git.io/JeOEw
06:47:25 <AutomatedTester> yhirano_: we have a python script that ...(websocket mention)
06:47:39 <BitBot> (14wpt) [PR] fred-wang 03merged 13#19112 into 07master: Add MathML test to check that menclose@notation="radical" is not supp… - https://git.io/JeOEQ
06:48:32 <AutomatedTester> jgraham: if something that needs to be in the API, what would you do it in
06:48:40 <AutomatedTester> Victor: that would be in Go or Rust
06:49:12 <AutomatedTester> jgraham: we currently dont have and compiled code step atm, we can add it but its not there at the moment
06:49:26 <AutomatedTester> jgraham: there is some impl complexity that needs to be supported
06:49:30 <gsnedders> https://wicg.github.io/web-transport/ is the specification in question, right?
06:49:31 <BitBot> (14wpt) [PR] fred-wang 03merged 13#19113 into 07master: Add MathML reftest to check that the mode attribute has no effect. - https://git.io/JeOu4
06:49:56 <shimazu> shimazu has joined #testing
06:50:14 <AutomatedTester> Hexcles: could the QUIC team give us a prebuilt binary and compiling tests
06:50:58 <AutomatedTester> ... and be separate repo for tests
06:51:08 <AutomatedTester> jgraham: we need to have ideally in a single repo
06:51:28 <AutomatedTester> yhirano_: we would have to have the handlers somewhere that can be updated regularly
06:51:50 <AutomatedTester> victor: is the requirement that no compilation or it must be python
06:52:01 <AutomatedTester> jgraham: there is no precedent for compiled code in wpt
06:53:51 <AutomatedTester> Hexcles: but the language doesnt matter here if it is rust or go
06:54:19 <AutomatedTester> jgraham: the complexity is that Firefox CI does Rust not Go
06:54:46 <AutomatedTester> ... and google will probably have the opposite problem
06:55:29 <AutomatedTester> ricea: [describes how we could have python over the go and connections to wptserve]
06:56:05 <AutomatedTester> ricea: or we could do the handlers in JS but it would be horrible
06:58:16 <AutomatedTester> ricea: my concern is that if we ship it we are limiting who can contribute to fix it while if we ship the build script then more people could add things. if the compiled step then having a handler in a non-compiuled language makes sense
06:58:51 <AutomatedTester> gsnedders: my concerns if we want webkit more involved they have pushed back if there has been runtime dependencies
06:59:07 <Hexcles> i/What would the test look like?/Victor: tests are for WebTransport, which is similar to WebSocket but in QUIC
07:01:00 <AutomatedTester> jgraham: we could have a process for compiling and then uploading to github for upstream, the browser version would be very different
07:02:04 <rniwa> rniwa has joined #testing
07:02:27 <AutomatedTester> jgraham: understanding the constraints for everyone would be simplify things.
07:03:33 <AutomatedTester> youennf: we have multiple servers why should we have it
07:03:43 <AutomatedTester> ... but at the end of the day the value of tests will win
07:04:39 <ricea> ricea has joined #testing
07:06:02 <zcorpan> zcorpan has joined #testing
07:06:30 <marcosc> marcosc has joined #testing
07:06:58 <Hexcles> scibenick: Hexcles
07:07:08 <Hexcles> ScribeNick: Hexcles
07:07:36 <Hexcles> Topic: ergonomics of test writing
07:07:45 <Hexcles> RRSAgent, update minutes
07:07:45 <RRSAgent> I'm logging. I don't understand 'update minutes', Hexcles.  Try /msg RRSAgent help
07:07:51 <Hexcles> RRSAgent, make minutes
07:07:51 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
07:08:10 <JohnJansen> TOPIC: Ergonomics
07:08:11 <Hexcles> reillyg: I've found it difficult to debug WPT
07:08:23 <JohnJansen> Topic: ergonomics of test writing
07:08:50 <Hexcles> foolilp: is it the error message or wpt.fyi that's confusing?
07:09:03 <Hexcles> reillyg: e.g. the "expand" button on wpt.fyi
07:09:11 <Hexcles> lukebjerring: [explains the UI]
07:09:19 <jorydotcom> jorydotcom has joined #testing
07:09:41 <Hexcles> jgraham: what is the ideal state?
07:10:51 <Hexcles> reillyg: summary of all the tests failed per browser (referring to Blink layout test viewer: expected, actual, diff)
07:11:12 <shimazu> shimazu has joined #testing
07:11:43 <Hexcles> ... make it as obvious as possible -- a link to failing tests, especially in PR
07:12:13 <Hexcles> lukebjerring: wpt.fyi GitHub Checks are not great (see discussion yesterday)
07:13:21 <Hexcles> jgraham: the "diff" on wpt.fyi is different from the diff in Blink CI -- we don't have baselines
07:13:30 <Ms2ger> Ms2ger has joined #testing
07:13:58 <Hexcles> lukebjerring: perhaps expand the results by default
07:15:00 <Hexcles> reillyg: want a summary page of all browsers instead of five different links
07:15:54 <Hexcles> lukebjerring: probably better to build a new view for the PR use case
07:17:28 <Hexcles> jgraham: collapse everything into one single check
07:18:17 <xiaoqian> xiaoqian has joined #testing
07:18:36 <Hexcles> lukebjerring: open issue: attach a list of run IDs to a check run instead of using e.g. "chrome@SHA" to avoid a race condition and to be able to show pending status
07:19:50 <ato> RRSAgent: Please draft the minutes.
07:19:50 <RRSAgent> I'm logging. I don't understand 'Please draft the minutes.', ato.  Try /msg RRSAgent help
07:19:52 <ato> RRSAgent: Please draft the minutes
07:19:52 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
07:20:00 <Hexcles> shimazu asked about sub test time out (one subtest time out makes the whole test time out)
07:21:00 <Hexcles> jgraham: promise_test is especially problematic because they run sequentially
07:21:21 <Hexcles> ... if a sub test times out the subsequent tests won't run
07:22:49 <Hexcles> lukebjerring: "difficult to determine subtest timeout" isn't a great argument. Being able to set subtest timeout is better and more meaningful than harness timoeout
07:23:08 <reillyg> q+
07:24:09 <Hexcles> jgraham: allowing authors to define subtest timeout is error-prone and flaky (e.g. a virtual environment can be paused)
07:24:48 <Hexcles> lukebjerring: but step_timeout takes multiplier into account; we can do the same for subtest timeout
07:25:20 <Hexcles> ... or we can set "short" "long" etc. timeout instead of concrete numbers
07:25:43 <Hexcles> q?
07:25:52 <Hexcles> ack reillyg
07:26:14 <Hexcles> reillyg: slow test is usually a test with too many subtests
07:26:25 <Hexcles> ... shall we prefer splitting those
07:26:52 <Hexcles> ... can the harness monitor the progress of subtests?
07:27:27 <Hexcles> jgraham: it sounds like a good design and possible, but we don't do that now (harness sends a JSON blob at the end)
07:28:43 <Hexcles> ... it doesn't solve your promise_test case?
07:28:44 <Hexcles> reillyg: combine the two: the harness monitors the promise chain
07:29:30 <Hexcles> jgraham: good idea
07:29:39 <zcorpan> zcorpan has joined #testing
07:30:22 <Hexcles> lukebjerring: e.g. similar to Travi (timeout if no output in 10m)
07:30:45 <Hexcles> jgraham: there's a lot of effort; testdriver.js & wptrunner need to be re-engineered
07:31:20 <Hexcles> ... testdriver.js already has a queue
07:32:41 <Hexcles> reillyg: improve the communication between harness and wptrunner. the test can declare the total number of subtests
07:32:49 <Hexcles> jgraham: there are more complicated cases
07:33:04 <Hexcles> foolip: for now we can split tests
07:33:23 <Hexcles> shimazu: yes we are doing that
07:33:46 <yhirano_> yhirano_ has left #testing
07:34:10 <Hexcles> shimazu: one more thing: service worker un/registration is expensive, which becomes a big overhead if we split promise tests
07:34:20 <xiaoqian> xiaoqian has joined #testing
07:35:08 <Hexcles> ... we use a promise test to do the setup
07:35:16 <Hexcles> foolip: this is the same as IDL harness test
07:35:38 <Hexcles> lukebjerring: this is not a great pattern
07:36:17 <Hexcles> foolip: we don't have an async setup function
07:36:28 <Hexcles> ... [explains how this breaks async_test]
07:37:02 <Hexcles> jgraham: yeah the lifecycle is tricky
07:37:32 <Hexcles> ... if we say "if you use async_setup you can only use promise_test" then we might be fine
07:38:44 <Hexcles> jgraham: we can have promise_setup!
07:39:24 <Hexcles> foolip: and emit errors if authors mix wrong setup/tests
07:40:20 <Hexcles> ... write an RFC!
07:41:00 <marcosc> marcosc has joined #testing
07:41:22 <Hexcles>  lukebjerring: I imagine it's very common to have lots of promise tests stuffed in a test file to avoid the setup overhead
07:42:18 <jorydotcom> jorydotcom has joined #testing
07:42:46 <Hexcles> foolip: can we just call promise_setup setup? if it returns a thenable then it's a setup for promise_test
07:43:19 <gsnedders> jgraham: everyone loves things that are polymorphic on their return type, right?
07:43:30 <Hexcles> jgraham: put this as an option in the RFC
07:44:18 <Hexcles> lukebjerring: I actually sent an RFC earlier for explicit naming / against polymorphic parameters
07:45:26 <Hexcles> ACTION jgraham file an issue to keep running as long as they’re producing subtest results
07:45:39 <Hexcles> ACTION foolip: RFC for async setup() (or promise_setup())
07:45:48 <Hexcles>   ACTION jgraham: file an issue to keep running as long as they’re producing subtest results
07:45:56 <JohnJansen> RRSAgent, make minutes
07:45:56 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
07:46:26 <Hexcles> foolip: switching topic to single page tests
07:46:41 <Hexcles> ... [explains single page tests]
07:46:50 <kevers> kevers has joined #testing
07:47:04 <Hexcles> ... it was added earlier to reduce boilerplate (similar to mochi tests)
07:47:28 <Hexcles> ... useful for async tests (saves a layer of wrapping)
07:47:39 <Hexcles> ... it's become problematic because of accidental opt-in
07:48:34 <Hexcles> ... ~600 tests that fail strangely because they accidentally trigger single page mode (e.g. they trigger an exception before starting any test)
07:49:20 <Hexcles> jgraham and foolip debating what result type is / and should be in this case
07:49:50 <Hexcles> foolip: [explains again when single page mode is triggered -- see RFC / docs]
07:50:13 <Hexcles> https://github.com/web-platform-tests/rfcs/pull/28
07:50:22 <Hexcles> lukebjerring: what's the motivation?
07:50:54 <Hexcles> jgraham: not for async tests, but e.g. something that responds to multiple events, each of which is a callback that needs to be wrapped in t.step
07:51:42 <Hexcles> ... if you don't wrap the callbacks, it ends up as a harness error
07:52:22 <Hexcles> ... so the real use case is to avoid wrapping every single callback
07:53:05 <Hexcles> foolip: my proposal is an explicit opt-in
07:53:17 <Hexcles> ... the real issue is difficulty to write correct async tests
07:53:32 <reillyg> q+ odejesush
07:53:34 <Hexcles> ... if we solve that we can get rid of single page tests altogether
07:54:10 <Hexcles> jgraham: I'm historically opposed to adding explicit opt-in to a feature which aims to avoid boilerplate
07:55:05 <Hexcles> lukebjerring: there are so few of single-page tests (~130), so adding a one-line opt-in isn't too bad
07:55:53 <Hexcles> jgraham: if there are only 130 of these -- although it seems to me this is a useful feature -- people are not using the feature much so it doesn't really matter?
07:55:56 <Hexcles> q?
07:55:59 <foolip> q+
07:56:11 <AutomatedTester> ack odejesush
07:56:53 <shimazu> shimazu has joined #testing
07:57:31 <Hexcles> odejesush: another issue: sometimes forget an event listener (e.g. for bluetooth disconnection)
07:58:40 <Hexcles> foolip: is the problem just forgetting to write an await
07:59:30 <Hexcles> reillyg: perhaps a linter?
07:59:36 <marcosc> marcosc has joined #testing
07:59:38 <Hexcles> ... rewrite all in TypeScript!
07:59:41 <Hexcles> (laugh)
07:59:44 <foolip> q?
07:59:50 <Hexcles> ack foolip
08:02:12 <Hexcles> foolip: [back to single-page tests] can we require setup.. explict done?
08:02:41 <Hexcles> jgraham: that sounds like a bad opt-in
08:04:14 <Hexcles> foolip: to fix 9000(?) tests, we can either add the t.step wrapper or turn them into single-page tests
08:04:38 <Hexcles> ... 9000 tests are the number of tests that have only one subtest
08:05:23 <shimazu> shimazu has joined #testing
08:05:27 <Hexcles> ... they are not necessarily broken
08:06:37 <JohnJansen> rrsagent, make minutes
08:06:37 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html JohnJansen
08:06:37 <Hexcles> ... but if there's a harness error in such case we could fix it by adding the t.step wrapper or turning them into single-page tests
08:06:48 <Hexcles> Topic: 2020 priorities
08:07:12 <Hexcles> https://docs.google.com/document/u/1/d/1gie7LFb6cAUfabY86MYuWM7m7ux_FaKhDkLdpz0zWkg/edit?usp=sharing
08:07:22 <Hexcles> foolip: last year we wrote lots of things
08:07:35 <Hexcles> ... but nobody looked at it over the year
08:07:45 <Hexcles> ... though we managed to get lots of them done
08:08:47 <MikeSmith> denis: https://www.w3.org/2019/09/17-testing-minutes.html has not been getting (re)generated for most of the day
08:09:51 <Hexcles> jgraham: this is not going to be something that has teeth, but something to align on
08:09:58 <MikeSmith> s/denis: https:\/\/www.w3.org\/2019\/09\/17-testing-minutes.html has not been getting (re)generated for most of the day/
08:10:08 <Hexcles> lukebjerring: let's enumerate stuff that's important
08:11:02 <Hexcles> Hexcles: can we start by going over action items?
08:11:57 <lajava> lajava has joined #testing
08:12:04 <Hexcles> jgraham: they are concrete things to do but not goals
08:12:21 <Hexcles> ... need to find the underlying theme
08:12:55 <zcorpan> zcorpan has joined #testing
08:12:58 <MikeSmith> actions?
08:14:16 <Hexcles> [please refer to https://docs.google.com/document/d/1gie7LFb6cAUfabY86MYuWM7m7ux_FaKhDkLdpz0zWkg/edit ]
08:14:19 <boaz> boaz has joined #testing
08:15:33 <Hexcles> [people are discussing wording of the goals]
08:15:40 <Hexcles> RRSAgent, make minutes
08:15:40 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
08:30:01 <zcorpan> zcorpan has joined #testing
08:30:31 <zcorpan> zcorpan has joined #testing
08:31:42 <zcorpan> q+
08:34:19 <zcorpan> ack zcorpan
08:35:07 <zcorpan> zcorpan has joined #testing
08:38:15 <lgrahl> lgrahl has joined #testing
08:41:54 <boaz> present+
08:42:45 <ato> RRSAgent: make minutes
08:42:45 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html ato
08:43:12 <ato> Present- Boaz_Sender
08:43:33 <tidoust> tidoust has joined #testing
08:45:20 <marcosc> marcosc has joined #testing
08:45:28 <rego> rego has joined #testing
08:46:40 <marcosc> marcosc has joined #testing
08:47:45 <foolip> q?
08:51:18 <BitBot> (14wpt) [PR] Honry 04closed 13#11600: [sensors] Bring setUpdateSensorReadingFunction from sensor mojo inter… - https://git.io/JeOzb
08:51:44 <denis> RRSAgent: make minutes
08:51:44 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html denis
08:53:48 <zcorpan> zcorpan has joined #testing
08:56:08 <marcosc> marcosc has joined #testing
08:58:03 <marcosc_> marcosc_ has joined #testing
09:05:28 <BitBot> (14wpt) [issue] Hexcles opened 13#19114: Add a QUIC server to test WebTransport - https://git.io/JeOzj
09:14:31 <zcorpan> https://bocoup.github.io/wpt-disabled-tests-report/
09:17:32 <Hexcles> s/TOPIC: Ergonomics//
09:17:38 <Hexcles> s/Topic: ergonomics of test writing//
09:18:00 <Hexcles> i/scribenick AutomatedTester/Topic: QUIC
09:18:16 <Hexcles> RRSAgent, make minutes
09:18:16 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
09:19:19 <Hexcles> i/ScribeNick: AutomatedTester/Topic: QUIC
09:19:21 <Hexcles> RRSAgent, make minutes
09:19:21 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
09:19:28 <denis> denis has joined #testing
09:23:58 <Hexcles> s|scibenick: Hexcles|https://github.com/web-platform-tests/wpt/issues/19114
09:24:07 <Hexcles> RRSAgent, make minutes
09:24:07 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/17-testing-minutes.html Hexcles
09:32:24 <Hexcles> Hexcles has joined #testing
09:49:23 <BitBot> (14wpt) [PR] chromium-wpt-export-bot requested 13#19115 merge into 07master: [LayoutNG] Correct overflow inside anonymous blocks inside %-insets. - https://git.io/JeOgM
09:51:00 <BitBot> (14wpt) [PR] Honry requested 13#19116 merge into 07master: Fixed lint error in #18886 - https://git.io/JeOgS
09:53:53 <plinss_> plinss_ has joined #testing
09:57:39 <BitBot> (14wpt) [PR] CYBAI requested 13#19117 merge into 07master: Fix ordering of script elements in `the-script-element/module/module-vs-script-*.html` tests - https://git.io/JeOgp
10:58:38 <marcosc> marcosc has joined #testing
11:08:04 <wanming> wanming has joined #testing
11:18:13 <BitBot> (14wpt) [PR] fred-wang requested 13#19118 merge into 07master: Add tests to check that legacy maction/semantics elements are treated… - https://git.io/JeOa3
11:38:03 <BitBot> (14wpt) [PR] chromium-wpt-export-bot requested 13#19119 merge into 07master: Block Xr.DeviceChange event without FeaturePolicy and add tests - https://git.io/JeOaS
11:38:28 <BitBot> (14wpt) [PR] chromium-wpt-export-bot requested 13#19120 merge into 07master: Split skip-not-fully-active.html into two tests - https://git.io/JeOa9
11:44:57 <cybai> cybai has joined #testing
11:46:33 <BitBot> (14wpt) [PR] fred-wang 03merged 13#19118 into 07master: Add tests to check that legacy maction/semantics elements are treated… - https://git.io/JeOa3
11:55:06 <marcosc> marcosc has joined #testing
11:55:36 <eeeps> eeeps has joined #testing
11:56:26 <marcosc_> marcosc_ has joined #testing
11:57:32 <marcosc> marcosc has joined #testing
11:58:44 <marcosc_> marcosc_ has joined #testing
12:02:17 <jorydotcom> jorydotcom has joined #testing
12:02:43 <birtles> birtles has joined #testing
12:06:13 <rniwa> rniwa has joined #testing
12:06:13 <marcosc> marcosc has joined #testing
12:21:50 <marcosc> marcosc has joined #testing
12:30:53 <xiaoqian> xiaoqian has joined #testing
12:34:36 <denis> denis has joined #testing
12:44:24 <BitBot> (14wpt) [PR] jdm 03merged 13#19117 into 07master: Fix ordering of script elements in `the-script-element/module/module-vs-script-*.html` tests - https://git.io/JeOgp
12:59:49 <karl> karl has joined #testing
13:04:03 <Zakim> Zakim has left #testing
13:06:35 <BitBot> BitBot has joined #testing
13:12:33 <BitBot> BitBot has joined #testing
13:22:32 <ato> RRSAgent: stop