A sample picture implementation
Hi all.
As a result of a weird lunchtime hacking idea, I’ve created a crude working example of the proposed picture element markup and behavior (a polyfill for a non-existent feature, how weird…).
This isn’t meant to be used in a production setting right now; it just seemed like it might be helpful to have something tangeable to work with while mapping out a preferred markup pattern. That said, I think it behaves roughly the same as existing client-only workarounds, so if it ends up proving helpful, I’ll try and maintain it as discussions progress so that it could potentially be used as a polyfill when-or-if a picture element lands in a browser or a candidate recommendation.
Example URL
http://scottjehl.github.com/picturefill/
Like the proposed picture element, you’ll need a media-query supporting browser to see anything more than fallback content, so IE9+, etc.
Source code
https://github.com/scottjehl/picturefill
Markup: https://github.com/scottjehl/picturefill/blob/master/index.html
JS: https://github.com/scottjehl/picturefill/blob/master/picturefill.js
Some notes:
- The JS uses the matchMedia API (with polyfill included in the external folder) to find matches within the source elements, keeping the last match it finds in source order.
- Per Mat’s example markup, the last element in the picture element is the fallback img element, and for the sake of this script, the matching src is simply plugged into that element.
- This script does nothing to prevent the fallback image from downloading before it is overridden by another potential match, and I’m not sure it needs to, since that’s an advantage a native implementation will bring.
- The script is configured to run when the DOM is ready, and on window resize (which tends to catch orientationchange as well).
—
Please let me know if this is at all detrimental to the planning stages. Here’s hoping it provides us something to play with and see what sort of markup will make the most sense in the end.
Thanks,
Scott Jehl
Thanks Scott. Some feedback:
The stacking of `min-width: 400px`, `min-width: 800px`, etc., should probably behave like it would in CSS. I was expecting all the images to be downloaded at higher viewport dimensions – as all stylesheets would be if using the examples `media` values – but only the larger one was.
When I resize the viewport, smaller images are downloaded. And when I resize to wider viewports, the larger images are re-downloaded. So I can end up with dozens of requests.
Hey Nicolas. Yeah, great points.
First pass, I wasn’t sure how best to handle all of that, so this is nice to discuss.
For the rules, I was thinking along the lines of CSS behavior too, but maybe in a different way than you were. I was thinking that if we considered the picture to only allow one valid source at a time, sortof like an element’s background-image value, then the last valid src in the source order would be requested, and ideally, none of the others that it overrides would be requested.
On resize, again like media queries, it’s re-running the logic, and I am not at all sure if that’s desirable for folks here, but it’s mapping to what I think I’d expect to happen with CSS at least, where there’s also potential for many requests after resize. Do you think it should only apply during initial load?
There’s probably precedent for both of these questions with the video element and its relation to source media attributes. Maybe we can make sure this maps to that, if it makes sense for picture.
Thanks!
What does `video` do with `source` and `media` like this? It would be confusing if any potential new media element did things differently to `video`.
About the resizing. I don’t think you’d ever want to redownload an image that has already been downloaded. But, ignoring that bug, this early script is a great way to show one way how the end-user experience might turn out for a new element like this. Thanks.
Yeah, I’m tending to agree with what you’re thinking about resize. Pending what others say, I’m not sure picture source elements should update at all after initial load…
One scenario where it could be useful is if the different sizes use contextual crops (rather than simple resizes). In that case, you’d want the proper size to flip in on resize, I think.
The Nokia browser site used to be a nice example of that, but it doesn’t seem to have different crops anymore.. hmm.
I think they’re still doing that in some places. For example, the top image on http://browser.nokia.com/meego-browser.html.
I would personally expect the ‘picture’ element to swap sources on resize. However, considering ‘source’ is it’s own element in the spec, the behavior should be consistent regardless of which media element it is contained in. So, if ‘video’ doesn’t reassess the source, ‘picture’ probably shouldn’t either.
I certainly think images should be reassessed after a window resize:
This matches the way CSS media queries work; e.g. which media queries are active is reassessed when resizing.
Resizing is a bit of an edge case (not many people actively do it), so I don’t see it as a serious performance hit.
It keeps you from getting into awkward situations. For example, you may want to change things like aspect ratios between breakpoints — that banner image isn’t so great at mobile sizes. If your mobile styles are assuming, say, a square image, not reassessing the image could lead to layout breakage.
Good point, but orientation changes are assumed to be more common, and usually result in a resize too (unless the device is square? 🙂 ).
Point taken, but I still think the other 2 points hold.
I’m leaning back towards yes on resize + reassess 🙂
Nice work 🙂
“This script does nothing to prevent the fallback image from downloading before it is overridden by another potential match, and I’m not sure it needs to, since that’s an advantage a native implementation will bring.”
I class this firmly as a disadvantage, not an advantage. One of the goals for responsive images is efficiency, and downloading an image that won’t be used is inefficient, both in bandwidth, HTTP calls, and device memory. That’s why the existing JS solutions are all no-go’s.
Hey Matt!
Oh, agreed. It’s totally a disadvantage. That’s why we’re trying to get a native implementation, right?
If you wanted to make this sort of script production-ready/responsible, tying it together with your script or even a quick inline cookie setting in the head could at least make that fallback request re-route to a blank gif or something less painful.
The current markup is at least unlikely to punish a handheld, since the fallback and the first source element are equal sources.
But yeah, you’re totally correct on production value of this. It’s not intended for that at this point.
Thanks!
It seems like the question of bandwidth continues to be a limiting issue, and I’ve seen a couple people batting around the notion of
@media (bandwidth >= x)– not necessarily in those terms, but that basic concept. I don’t know the browser side of it as much, but it seems like that should be doable in general, right? That would allow for a sensible native implementation of this: combining screen resolution with bandwidth would allow you to prioritize that way, and the same thing would supportandnicely. The only potential downside I see is that bandwidth might change mid-streaming of the content. Someone much more familiar with browser attributes/limitations than me might be better equipped to comment…That should read “the same thing would support video and audio nicely”; I wrapped them in
codetags, but may have done so incorrectly.Hey Chris,
Bandwidth is a complicated issue that I’m not sure the picture element is poised to address on its own, but perhaps other apis and media queries will get there one day, and the proposed markup for picture should be able to tap into whatever comes with that.
Hmm… yeah, sorry; that comment probably belongs in a different context. I’ve got it on the brain at the moment.
More on topic: Presumably you’re still going to have the ability to save the image (*right-click→save”); how would a native implementation handle allowing the user to save the highest resolution image? Would it default to that with a save action? I am wondering if video and audio already have solutions for that which picture could adopt, so I’ll start by going and reading the specs.
Okay, I went and read through the specs for audio and video. It looks like those specs assume that the only time an author is going to want a direct link to those elements – that is, the only time a user is going to be attempting to save those elements – is when the user-agent player is not working, i.e. the fallback case. That makes some sense in the context of those particular elements, I suppose, although I think an argument could probably be made for easier ability to save those elements to local storage for later retrieval.
Pictures are a bit different, though, at least from my point of view. I often want to at least see the highest resolution of an image, even if I wouldn’t necessarily want it loaded on my page. Now, in theory a content provider could wrap a
<picture>element in a link to the highest resolution form of the image – that’s already fairly common practice in cases where a thumbnail is being displayed. Should we just assume that<picture>will ultimately be supplying “thumbnail”-like content and wrapping in links if we want the users to have access to that image at a higher resolution? That seems an eminently reasonable approach to me, but I’m curious if people who are thinking much more and harder about this than me already have thoughts on this issue.Awesome work in any case.
Ahhh, I think I mis-interpreted what you said originally, sorry! I thought you were saying you weren’t sure *the final solution* needed to avoid the duplicate requests. You’re actually saying that the final solution would solve that problem so the ‘bad behaviour’ of the test isn’t an issue?
Doh.
You put it far more clearly. So, what you said! 🙂
I’m not sure what a finalized picture element polyfill will entail once the element is actually supported in some places, but I’d imagine the first step would be to return immediately after determining the element is actually supported, and provide a way to easily tie-in server-side workarounds for the double-request problem. It’d be nice if the most basic polyfill, like that above, didn’t care much about that part. At any rate, when it gets to that stage, I’d love your help in planning how that piece can work.
I’m up for helping where I can 🙂 It’s a shame my JS knowledge is poor, and my PHP rusty (Adaptive Images was the first time I’d done any real PHP in 5yrs or more!)
Yeah, as a proof of concept polyfill what you’ve got seems good. Be interesting to see how versatile it can be made.
@Scott Jehl looks like we spent our President’s Day weekend the same way: github.com/gordonbrander/Picturize :).
One thing the script I hacked together has on offer is a Modernizr test, followed by a fast exit if the hypothetical
picturetag is supported. Feel free to lift/repurpose.Hey Gordon! Hah, it appears we did 🙂
Great work. At a glance, it seems these do similar things, which is great to see. As I mentioned above and you did also, a feature test for picture support is missing in the version I’d posted. However, I think there’s some value in a non-jQuery dependent version too, and the matchMedia polyfill nudges quite a few media-query-supporting browsers as well.
Might I invite you in on the picturefill repo so we can get the element test integrated up top in the script? If it needs modernizr, that could be okay, though a quick element test without it might do just as well for now.
I’ve added you as a committer, so let me know if you’re interested 🙂
Thanks and again, awesome work!
Totally agreed re: jQuery — a plain ol’ JS approach is superior.
I’ll send a pull request your way with those changes later tonight.
Can’t wait to see it Gordon! Thanks for joining forces on this 🙂
Sent the pull request your way https://github.com/scottjehl/picturefill/pull/3. I’ll let you do the merge if it all looks fine and dandy to you.
Gordon, fantastic work, man. I will review this today.
I would definitely expect/assume them to be reassessed on re-size at first thought. Although it seems more HTTP requests are a bit scary at the moment to do the switch. I know this might sound a bit silly, but If only there was a way to make something like this to serve up images for devices/resolution:
@mobile {..} or
@mobile and (min-resolution:96dpi) and (max-resolution: 100dpi) {..}
@desktop and (min-width:960px) {}
@print {..}
Indeed, there are media queries that exist already for that purpose, and you can already use them with this polyfill for browsers that understand them. For example, this line sends higher-resolution icons to HD (or “retina”) screens in jQuery Mobile: https://github.com/jquery/jquery-mobile/blob/master/css/themes/default/jquery.mobile.theme.css#L880
A great start/solution for background imagery, but I see its still back to a needed solution to accomplish this for embedded media much like the
<video>and<audio>tag already discussed. hmmmmm….more sleep , then coffee. Maybe we should stop embedding images in markup altogether?heh. well, if that’s suitable for you, then this https://twitter.com/scottjehl/status/169117397838602240
🙂
Awesome! just popped it up on the opera mobile emulator and what a difference. The image was so elastic.
Side note: I wish resolution detection was more like I referenced in my initial comment rather than
device–pixel–ratiocurrently implemented. It just seems easier to reference and understand the exact resolution over the ratio usage.Can I play the devil’s advocate and suggest a “reassess” attribute on the picture element.
This would give developers an opt-in to the “reload on resize” question on an element-by-element basis.
Pingback: <picture> for existing browsers. | Responsive Images Community Group
Pingback: Currying Favour with Responsive Web Developers | abitgone+
Pingback: Polyfilling picture without the overhead | Responsive Images Community Group
@Brian – I think that’s a good idea. I’d like to see an attribute on the picture element, something like reload=”true|false” or some better wording where you could specify if images in the picture element should be reassessed on resize as opposed to reassessing all of them.
Pingback: The Responsive Image Problem | Mouse Potato
Pingback: Picturefill | TRICKFLASH
Pingback: 迷路的猫少年 » Blog Archive » html5需要个picture标签?
Pingback: The Web Needs to Get Ready for the High-Resolution Future | Paramark Inc.
How would <picture> handle image maps? different sources may very well rearrange the contents of an image, image-maps would then have to be changed accordingly.
Very good point that until now hasn’t been discussed. I think it has to be the same as with the image tag. We need a new spec for ?
This again questions if we should have a solution like a “MQ-template” as Matt proposed.
It would actually be much easier to support image maps with this sort of implementation.
Currently, on the image, you just assign which image map to use.
With Scott’s polyfill, and the proposed implementation, we would simply move that to the source tag, which fixes almost ANY problem related to image maps.
Granted, this now means you would have to have an image map for each size you show at, so perhaps there could be some improvements with the imagemap accepting percentages (or some other measurement).
I always saw image maps as a kind of hack anyway, and they’re much less needed now that you can just directly position anchors over an area where you want them to appear.
Maybe the answer is that image maps are dead?
image maps are pixel-based, rather than percentages, right? That seems like less of a problem with the picture element and more of a problem with image maps not working in fluid layouts.
You might also try and reproduce an image map technique with positioned anchors in CSS, where you’d likely have much better luck with fluid and responsive layouts.
I’m not sure that this is a problem picture needs to solve though…
Agreed. I think anchors in CSS are the better solution anyway.
Pingback: Responsive Web Design – Interface - Blog of Web Communications at the University of Missouri