14:28:29 <RRSAgent> RRSAgent has joined #annotation
14:28:29 <RRSAgent> logging to http://www.w3.org/2016/09/16-annotation-irc
14:28:31 <trackbot> RRSAgent, make logs public
14:28:31 <Zakim> Zakim has joined #annotation
14:28:33 <trackbot> Zakim, this will be 2666
14:28:33 <Zakim> ok, trackbot
14:28:34 <trackbot> Meeting: Web Annotation Working Group Teleconference
14:28:34 <trackbot> Date: 16 September 2016
14:28:37 <ivan> Agenda: http://www.w3.org/mid/033001d20f58$8f91ee50$aeb5caf0$@illinois.edu
14:28:57 <ivan> ivan has changed the topic to: Agenda for 2016-09-16: http://www.w3.org/mid/033001d20f58$8f91ee50$aeb5caf0$@illinois.edu
14:29:06 <ivan> Chair: Tim
14:29:10 <ivan> Regrets: Rob
14:59:42 <ivan> Present+
14:59:43 <ShaneM> present+ ShaneM
14:59:52 <TimCole> TimCole has joined #annotation
15:02:32 <tbdinesh> tbdinesh has joined #annotation
15:02:44 <TimCole> Present+ Tim_Cole
15:02:57 <tbdinesh> Present+ TB_Dinesh
15:04:21 <timeless> timeless has joined #annotation
15:04:26 <bigbluehat> bigbluehat has joined #annotation
15:05:15 <takeshi> takeshi has joined #annotation
15:05:29 <ShaneM> Scribe: ShaneM
15:05:49 <uskudarli> uskudarli has joined #annotation
15:06:53 <PaoloCiccarese> PaoloCiccarese has joined #annotation
15:07:19 <dwhly> Present+ DAN_WHALEY
15:07:20 <ShaneM> Topic: M<inutes
15:07:21 <TimCole> PROPOSED RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/09/09-annotation-minutes.html
15:07:24 <ShaneM> TOpic: Minutes
15:08:07 <ShaneM> RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/09/09-annotation-minutes.html
15:08:10 <TimCole> Topic: Announcements
15:08:19 <takeshi> Present+ Takeshi_Kanai
15:08:41 <TimCole> See https://lists.w3.org/Archives/Public/public-annotation/2016Sep/0058.html
15:08:45 <ShaneM> Regrets+ Benjamin
15:09:04 <ShaneM> Charter extended through February.
15:10:06 <TimCole> Topic: Issue Updates
15:10:16 <TimCole> None
15:10:22 <ShaneM> *cruckets*
15:10:28 <ShaneM> s/cruckets/crickets/
15:10:31 <TimCole> Topic: Testing Updates
15:10:43 <ShaneM> Topic: Testing
15:11:27 <ShaneM> There is an issue with the link into w3c-test.org with a deep path doesn't seem to work right
15:12:44 <ShaneM> TimCole: there might be an issue in the future when there are other test collections
15:12:50 <ShaneM> ShaneM: it works for me as well
15:14:25 <ShaneM> dwhly: I will try to sort it out
15:14:41 <ShaneM> TimCole: Janina mentioned that refreshing the window index.html sometimes didnt work.
15:14:57 <PaoloCiccarese> Present+ Paolo_Ciccarese
15:15:38 <ShaneM> TimCole: Nick reported the timing of the text box means pasting before the page populates will cause the informaiton int eh window to get cleared out.
15:15:43 <ShaneM> ShaneM: That doesnt surprise me
15:16:05 <ShaneM> TimCole: We did consolidate the tests.  My impression from everyone is that is better.
15:17:02 <aaronpk> aaronpk has joined #annotation
15:17:17 <ivan> q+
15:17:21 <ShaneM> ShaneM: Dont change the anmes of any tests in the future.
15:17:24 <TimCole> ack ivan
15:17:36 <ShaneM> TimCole: I think it is order sensitive too.   I will not change names nor order.
15:17:41 <tbdinesh> update: test links seem to work ok now. both. maybe its to do with server load?
15:18:03 <ShaneM> ivan: At the end when we are finished we will need to reconcile things.  We have different columns for the same implementation now at different times.
15:18:18 <ShaneM> tbdinesh: oh, that's possible.
15:19:01 <ShaneM> ShaneM: The various columns represent tests run with different input.
15:19:38 <ShaneM> TimCole: If you have an implementation that generates different kinds of annotations... for example Janina's can link images, annotation, or transcribe... you end up with different annotations that use different features of the model.
15:20:04 <ShaneM> ... so in order to see the complete set of features, you need to test multiple annotations and then collapse the columns together to show the various features supported.
15:20:14 <ShaneM> ... I still think that we should collapse the columns together.
15:20:30 <ShaneM> ivan: I didnt know that.  I am not really sure how we do that at the end.
15:20:50 <ShaneM> TimCole: I think that means if there are any from the same implementation, you OR them together.
15:21:02 <ShaneM> ... you may not want to do that with the mandatory tests.
15:21:24 <ShaneM> ... you might want to eliminate columns with failed mandatory tests.
15:22:24 <ShaneM> ivan: from the CR point of view... which tests the spec.  If an implementation doesn't pass a required feature, I would expect essentially the three columns to be identical for a correct implementation.
15:23:01 <ShaneM> TimCole: It might be that an implementation hasn't implemented a new kind of selector.  We have an assertion that checks if you have SpecificResource in your annotation and it has a selector it is one of the six of seven types defined in the model.
15:23:13 <ShaneM> ... if I define a new sleector type, one of my annotations may not pass that test.
15:23:42 <ShaneM> ... have I failed from a requirements standpoint?  That's something we would need to interpret.  If we leave it a MUST test we need to decide if we can ignore that kind of failure or not.
15:23:56 <ShaneM> ivan: if it is an extension then it cannot be a must
15:24:05 <ShaneM> TimCole: We have a section that defines how you extend the model.
15:24:14 <ShaneM> ivan: if it is not a normative section then it is not a must
15:24:34 <ShaneM> ivan: so in this case the extension is not something we want to test.  It feels esoteric.
15:26:00 <ShaneM> ... for MUST tests all the columns should have pass and fail.  So we can throw out the aberrant results if some fail a mandatory test as long as some tests PASS the mandatory tests for the same implementation.
15:26:32 <ShaneM> TimCole: We should talk to Rob about this.  We we try to make the sections about extensions normative then we need to update the SHOULDS to MUSTS
15:27:19 <ShaneM> (some discussion about extending the vocabulary...)
15:27:38 <ShaneM> TimCole: let's not finish this here...
15:27:55 <ShaneM> ivan: no.  it is important.  We need to be sure that we have not made editorial errors.
15:28:08 <ShaneM> ... for example I see that the JSON-LD Frame is normative and it should not be.
15:29:00 <ShaneM> ivan: the boilerplate for the spec says appendices are normative unless they explicitly say they are informative
15:30:25 <ShaneM> TimCole: Discussion of optional "fails".  Should we report them at all?
15:31:08 <ShaneM> ivan: I am a little bit messed up... what happens right now?
15:31:18 <ShaneM> TimCole: for example we have 36 checks related to agents.
15:31:35 <ShaneM> agents is optional.  You SHOULD have an annotationCreator but it is not required
15:31:52 <ShaneM> ... if you don't have one, then there are a bunch of related tests that will fail
15:32:30 <ShaneM> ... if you don't report those you will get 8 yellow boxes.  If you report the SHOULDs you will get some red and some yellow.
15:33:04 <ShaneM> ivan: what are our options?
15:33:25 <ShaneM> ShaneM: We don't really have any control.  There is pass, or fail, or no data (yellow)
15:33:37 <ShaneM> ivan: Since it is not a MUST it is not really a fail.
15:34:09 <ShaneM> TimCole: So how are we looking at the may requirements?  If no one implements text direction as a result of testing, does that mean we should mark it at risk?
15:34:16 <ShaneM> ivan: that seems crazy to me.
15:34:30 <ShaneM> TimCole: It is not a SHOULD nor a MUST.  it is in because people think it might be useful.
15:34:38 <ShaneM> ... there might not be anyone who uses it.
15:35:11 <ShaneM> ShaneM: can we actually mark things at risk now?
15:35:15 <ShaneM> ivan: No, we cannot.
15:35:42 <ShaneM> ivan: If a SHOULD is not implemented, then it can be yellow...  but I feel a little uneasy about it.
15:35:54 <ShaneM> ... it is not FAIL in that it doesn't meet the specification.
15:37:23 <ShaneM> ShaneM: THe features need to be called out.
15:37:43 <ShaneM> TimCole: The SHOULDs seem to map to features.
15:38:02 <ShaneM> ... MAYs are not really features.
15:38:22 <ShaneM> q+ to ask about what we do to recreate the results?
15:38:58 <TimCole> ack ShaneM
15:38:58 <Zakim> ShaneM, you wanted to ask about what we do to recreate the results?
15:39:06 <ShaneM> ivan: the problem is that the reporting mechanism is too strict.
15:39:15 <ShaneM> ShaneM: we can change it... but how?
15:39:51 <ShaneM> ivan: On the report... if there is a SHOULD or MAY and if an implementation doesn't do it then I would like to see that as a different entry that means not implemented.  Something that indicates it is not an error.
15:40:03 <ShaneM> ... the FAILS should be only appearing for the MUST features.
15:40:57 <ShaneM> ... for optional features there should be a way to indicate that an implementation supports or does not support an optional feature.
15:41:53 <ShaneM> ShaneM: What happens if an optional feature is supported but does not pass the tests?
15:41:57 <ShaneM> ivan: then that is an error.
15:42:03 <ShaneM> TimCole: We have that in the mandatory tests now.
15:42:18 <ShaneM> TimCole: Reviews what are "features" in the data model
15:43:01 <ShaneM> ... no options are listed as features.
15:43:10 <ShaneM> ivan: then they should never me listed as a fail.
15:44:25 <ShaneM> TimCole: They only thing fuzzy one agent class as related to an annotation.  With predicates creator and generator.
15:44:39 <ShaneM> ... they are currently listed as SHOULDs
15:45:27 <ShaneM> ivan: I believe the agentClass is an option and has requirements
15:45:39 <ShaneM> TimCole: In actuality the agents do not have this requirement
15:45:47 <ShaneM> ivan: then those two entries do not mean anything
15:46:25 <ShaneM> ... hard to discuss without Rob, but there doesn't seem anothing relevant here to exit criteria
15:46:33 <ShaneM> TimCole: so in terms of the tests, we have a couple options.
15:47:04 <ShaneM> ... we could delete the tests.  We could NOT report them failing.  But we could still gather the data (in case there is any debate later).  Or we can leave them as they are and they show red.
15:47:08 <ShaneM> q+ to make a proposal
15:47:21 <ShaneM> ivan: we sort of left it open.  Is there a logic we can follow?
15:47:27 <TimCole> ack ShaneM
15:47:27 <Zakim> ShaneM, you wanted to make a proposal
15:48:23 <TimCole> Proposal: Shane has a version of the test library that will not report when an optional assertion fails
15:49:09 <TimCole> ShaneM: If an assertion is a should or may and it fails, the test does not report a result
15:49:21 <TimCole> ... the effect is a yellow box in the final report
15:50:14 <TimCole> ... suppose there is an assertion that the sky is pink. No implementation passes, so no row will appear for this assertion
15:50:46 <TimCole> ... it's yellow for all others if at least one implementation uses and all others don't
15:53:09 <ShaneM> TimCole: I have a reference implementation that supports every optional
15:53:19 <ShaneM> ivan: I don't mind that being in the report.  It is a real implementation.
15:54:00 <ShaneM> ... I dont like it to look like a FAIL on a SHOULD is the same as a FAIL on a MUST
15:54:07 <ShaneM> ShaneM: Isn't it okay to have that indicated int he name
15:54:18 <ShaneM> ivan: No - it isn't.  People don't look closely.
15:54:32 <ivan> q+ on something else
15:54:36 <ShaneM> TimCole: If we have the implemenmtation that ignores shoulds and mays, let's try it.
15:55:26 <ShaneM> TimCole: For example in one implementation there are three real fails.  I hope they choose to fix those before the end of CR.  If fixed they would have green for everything mandatory and some optionals
15:55:32 <ShaneM> ... they would have yellow for everything else.
15:55:33 <TimCole> ack ivan
15:55:33 <Zakim> ivan, you wanted to comment on something else
15:56:17 <ShaneM> ivan: Ralph and I were looking at the reports.  A minor comment.  We use in the text "Reference Implementation".  We have not defined it.  We should use a different term.
15:56:22 <ShaneM> TimCole: I didn't realize that
15:56:48 <ShaneM> ShaneM: I did that - I called it the RI.
15:57:26 <ShaneM> TimCole: No meeting next week because of TPAC
15:57:55 <ShaneM> TimCole: meeting the week after that.  Need to prioritize how to reach out to OA implementors that illustrate changes to the current model.
15:58:10 <ShaneM> .. changes of key names and a few other things.  We have not collated those all in one place.
15:58:15 <ShaneM> ivan: wait... I thought that
15:58:33 <ShaneM> ... in the model document there is appendix F.3.  It details the changes.
15:58:37 <ShaneM> TimCole: You're right.  Good!
15:58:56 <ShaneM> ivan: Might not be anything in the vocab but I think this is done.  We may need more explanation.
15:59:02 <ShaneM> TimCole: I will talk to Rob about it for next week.
16:00:11 <ShaneM> rrsagent, make logs public
16:00:20 <ShaneM> rrsagent, make minutes
16:00:20 <RRSAgent> I have made the request to generate http://www.w3.org/2016/09/16-annotation-minutes.html ShaneM
16:00:33 <ShaneM> Next meeting in two weeks
16:01:12 <ShaneM> rrsagent, make minutes
16:01:12 <RRSAgent> I have made the request to generate http://www.w3.org/2016/09/16-annotation-minutes.html ShaneM
16:03:38 <ivan> rrsagent, draft minutes
16:03:38 <RRSAgent> I have made the request to generate http://www.w3.org/2016/09/16-annotation-minutes.html ivan
16:05:45 <ivan> ok
16:06:56 <csarven> Who will be at TPAC next week?
16:07:06 <ShaneM> I will
16:07:56 <tantek> tantek has joined #annotation
16:14:33 <ivan> me too
16:17:44 <ShaneM> csarven: are you attending?
16:21:21 <csarven> Yea! I'd really like to poke some of you on my implementation.
16:21:51 <csarven> TBH, I feel a bit foreign to the tests right now. Need some handholding.
16:22:11 <csarven> Implementation as a Sender/Consumer
16:32:32 <csarven> Is there a test annotationService that I can POST to, GET from?
17:26:23 <tilgovi> tilgovi has joined #annotation
17:45:13 <ShaneM> csarven: sorry I was away
17:45:43 <ShaneM> We have one in the WPT repo that you can use to play with
17:45:58 <ShaneM> there are no TESTS for senders / consumers, since there are no requirements on them per se.
18:05:12 <shepazu> shepazu has joined #annotation
18:09:03 <shepazu_> shepazu_ has joined #annotation
18:15:14 <Zakim> Zakim has left #annotation
19:12:15 <tilgovi> tilgovi has joined #annotation
20:40:59 <tilgovi> tilgovi has joined #annotation
21:06:41 <KevinMarks> KevinMarks has joined #annotation
21:25:33 <tilgovi> tilgovi has joined #annotation
21:38:54 <tilgovi_> tilgovi_ has joined #annotation
23:36:45 <ben_thatmust> ben_thatmust has joined #annotation