<scribe> scribenick: zcorpan
[presentation]
cyril: container format used
heavily for video, now proposed for images
... HEIF
... lots of images on a web page, on average 30 per page
... small in resolution
... for icons
... constraints cna be very different compared to video
... images need to be decoded very fast
... footprint is a problem
... small footprint for video decoder
... gpu for video
... goal is to try to understand the consequences for browsers,
content providers, users
... overlap and differences
... not everyone is familiar, want to give more details
... ISOBMFF
... "boxes"
... MSE-type video "moov" "moof"
... for images "meta" box
... freely available
... link to spec
... more recent spec defines hwo to use the "meta" box to store
props for th eimage
... new type of track, "pict" for animations
... how to store thumbnails
... something channels
... codec agnostic
... annexes how to store HEVC, AVC, JPEG, but can use any
codec
<markw> ISO BMFF: https://standards.iso.org/ittf/PubliclyAvailableStandards/c068960_ISO_IEC_14496-12_2015.zip
pal: freely available? Rf?
cyril: the spec is freely
available.
... in terms of royalties, the spec has been available for 20
years...
pal: ISO patent database...
cyril: doesn't mean naything
<markw> MPEG Image File Format: https://standards.iso.org/ittf/PubliclyAvailableStandards/c066067_ISO_IEC_23008-12_2017.zip
cyril: if someone impl it, and
has been sued, that's relevant
... large spec with lots of tech, not all of it is useful
... MIAF
... meta box is required
... for specifying base image
... even if you have a sequence of images
... e.g. middle image is representative, used for
thumbnail
... media profiles
... brands, special profiles
... progressively loaded
... burst of images
... spec is not freely downloadable
... can pay to get it
... AVIF
... hosted on github
... freely available
... is based on MIAF
... uses AV1 codec
... transparency
... two media profiles
... baseline, advanced
... main and high
... reference software
... C impl, portable
... encode and decode
... including alpha
... the support, HEIF-based images, in browsers is ...
currently limited
... does apple support HEIF?
OS supports it but not web-exposed
JakeA: you can encode but not decode in Safari macOS
cyril: other browsers, dunno?
mstange: firefox doesn't support
cyril: chrome bug...
annevk: we're looking into
it
... open issues with AVIF
cyril: open questions... is it reasonbale to assume that if a video codec is supported, will the image codec be too?
annevk: once we support both, sure
LOL
cyril: don't want to use multiple decoders
??: path for gecko, between image and video totally different
??: but can change
mstange: in the past the answer
has been no. for webp and webm
... we don't want the web platform to rely on (something) image
formats
yoav: it may be technically possible but there are other considerations
cyril: would media capabilities ?
mstange: i don't see why not
yoav: accept headers...
ChrisLilley: accept were expected to have that role, but failed
yoav: ok but sounds like a reasonable use case
markw: possible that the device
can support both
... which should you choose
yoav: nowehere near that spec but sounds reasonable use case
cyril: AVIF format, supported in
video
... how likely is it to get uniform support
... delta to achieve parsers
??: common demuxing and decoding video and images
<Yves> https://tools.ietf.org/id/draft-ietf-httpbis-variants-00.html is relevant to the 'accept' header discussion
annevk: for image loading there's
some byte sniffing
... some challenge
... don't want to add new sniffing
... strict mime types for new formats
yoav: strict mime types seems reasonable
annevk: more than reasoanble
ChrisLilley: do you say ??????
??: only to determine the container, not the codec
zcorpan: for top level navigation, sniffing to figure out type of document
cyril: the way HEIF define mime
types
... one for static images
... one for image sequences
??: discussion a while back with w3c on media whether you listen to the server mime type
??: can only determine by sniffing, not by what the server says
yoav: servers lie
... but if we add new stuff, we can break it
??: for firefox/safari (some format) , need sniffing
??: issue of trust
??: already don't trust mime type
yoav: we should trust and
verify
... if they lie, nothing runs
... people should figure it out and fix it
annevk: we require the mime type to be correct, for webvtt, javascript modules...
??: video in particular
??: what makes major difference is the container
annevk: video element does a request
??: media source, you specify what you're going to be fed
annevk: you pass into an API
??: when you create your SourceBuffer
jya: (missed)
yoav: the server mime type is irrelevant
JakeA: sniffing is fine if you have full access to (something?)
annevk: sniffing i'm talking
about if you navigate to a random url
... or if we load cross-origin resources
... safe-listing media resources
... others we'd block
... i'd like to restrict the sniffing. not keep adding
cyril: the first bytes of the
file contains the type of file
... the mime type tells you that
jya: video src = bla
yoav: server sends a mime
type
... it needs to correspond to the content
... otherwise we need to add to the sniffing
JakeA: if ... CORS, would just work
annevk: maybe
cyril: specific mime type
... issue is "why 2 mime types"
... either you change the spec, if the static mime type is
given but the content is actually a sequence
... rendered as the static
... or get rid of the second mime type
ChrisLilley: given it has
htumbnail
... show static image
... may want to do that deliberately
yoav: are there scenarios where
static will be supported but sequence not?
... if so we need 2 mime types
cyril: when i talk to media folks at google, they don't know
mstange: animations
jya: does the user want to be annoyed with sound
yoav: can be scenarios where people will have multiple ... dunno
cyril: no requirement in the spec to treat the mime types differently
annevk: problem is the mime type has no impact on processing model, will end up the same
yoav: <picture>,
preload
... if i specify a mime type that is not supported
... it will be skipped
JakeA: if you specify jpeg but return png, it works
yoav: if you support only one, you want Accept to reflect that (hopefully), and type="" should work
annevk: why would you switch between those two mime types?
yoav: may want to fall back to other format
cyril: let's say you intend to
support an api
... would you support only static?
annevk: moz would support both
yoav: problem with apng
ChrisLilley: PNG group said "you should register video/png" but they didn't want to do that
yoav: question: does anyone want to ship one but not the other?
ChrisLilley: use case for static or animated?
yoav: for goth type attr and
accept header
... if UA supports the format for both
... same signal
... no way for the web dev to distinguish based on mime
type
cyril: third bullet point
... consistent support between <img> and
<video>
... will we see ... will the overlap , unified model,
convergance between the two elements?
yoav: wouldn't expect it
cyril: difference is subtle
yoav: there are multiple image
formats that have animation, gif, webp
... conv with our media team
... needs convincing that the complexity is worth it
... for video codec in <img>
... if you have data
... byte-savings e.g.
... i can cummunicate it
... ability to sandbox
cyril: safari team?
smfr: if you think about...
yes
... when we display an mp4 file
... use hardward decoder
... decode one frame at a time
... we don't need to keep them all around
... with animated gif
... may not want to decode each frame every loop
... savings
JakeA: comparison is <img src=video> vs <video muted>
smfr: can't use video as background image
yoav: problem of not everyone can
control and modify their markup
... i argued for that internally
ChrisLilley: ...lost?
yoav: well yes
... animated webp is assumed to be enough
... for background-image
... lack of control for markup...
... otherwise <video loop muted> would work
smfr: arguing power and memory...
yoav: so... based on gpu decoding?
smfr: yes
yoav: argument was that images
and animated gifs tend to come in lots vs video is only
one
... you could have multiple animated gifs
... i will convey your skeptisism ericc/ ^
Richard: gpu or cpu, does it
matter?
... <img> tag, or if they merge
mstange: video where img is
supported...
... img is supported in many places
smfr: we support video everywhere
cyril: (missed)
yoav: bring data
annevk: why do you want that?
cyril: when only some browser
support
... same format, same codec
jya: video format was just so
much better at compressing compared to gif
... you see on reddit, 100MB gif
... is it relevant
... reason is it compresses better
... now images compress well, is it relevant?
annevk: also special cases
jya: would you like your images to be (missed)
cyril: depends on format?
... can follow image pipeline
jya: webp, use image
decoder
... different objective
... want to be able to pause
... can't do that with video
... interrupts page loading
... purely technical
cyril: can still provide same decoding features
jya: ok if you have video that is gpu decoded, may not have same features in cpu
smfr: fundamental differences
between img and video
... color profiles different
... not sure those would converge
... might be a while before those converge
... netflix is interested in mixing the two
cyril: yes
jya: if it were easy, everyone would have done it
cyril: some have done it
jya: one format
cyril: adjurn
This is scribe.perl Revision: 1.154 of Date: 2018/09/25 16:35:56 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Succeeded: s/YY/markw/ Succeeded: s/YY/markw/ Succeeded: s/??/jya/ Succeeded: s/webp/apng/ Succeeded: s/smfr/ericc/ ^/ Present: ChrisLilley zcorpan smfr gkatsev JakeA mstange MasakazuKitahara domfarolino markw yoav bdekoz bkardell_ Found ScribeNick: zcorpan Inferring Scribes: zcorpan WARNING: No "Topic:" lines found. WARNING: No meeting chair found! You should specify the meeting chair like this: <dbooth> Chair: dbooth WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: No "Topic: ..." lines found! Resulting HTML may have an empty (invalid) <ol>...</ol>. Explanation: "Topic: ..." lines are used to indicate the start of new discussion topics or agenda items, such as: <dbooth> Topic: Review of Amy's report WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]