A sample picture implementation

Scott Jehl | Posted on: February 21, 2012

Hi all.

As a result of a weird lunchtime hacking idea, I’ve created a crude working example of the proposed picture element markup and behavior (a polyfill for a non-existent feature, how weird…).

This isn’t meant to be used in a production setting right now; it just seemed like it might be helpful to have something tangeable to work with while mapping out a preferred markup pattern. That said, I think it behaves roughly the same as existing client-only workarounds, so if it ends up proving helpful, I’ll try and maintain it as discussions progress so that it could potentially be used as a polyfill when-or-if a picture element lands in a browser or a candidate recommendation.

Example URL

http://scottjehl.github.com/picturefill/

Like the proposed picture element, you’ll need a media-query supporting browser to see anything more than fallback content, so IE9+, etc.

Source code

https://github.com/scottjehl/picturefill

Markup: https://github.com/scottjehl/picturefill/blob/master/index.html

JS: https://github.com/scottjehl/picturefill/blob/master/picturefill.js

Some notes:

The JS uses the matchMedia API (with polyfill included in the external folder) to find matches within the source elements, keeping the last match it finds in source order.
Per Mat’s example markup, the last element in the picture element is the fallback img element, and for the sake of this script, the matching src is simply plugged into that element.
This script does nothing to prevent the fallback image from downloading before it is overridden by another potential match, and I’m not sure it needs to, since that’s an advantage a native implementation will bring.
The script is configured to run when the DOM is ready, and on window resize (which tends to catch orientationchange as well).

—

Please let me know if this is at all detrimental to the planning stages. Here’s hoping it provides us something to play with and see what sort of markup will make the most sense in the end.

Thanks,

Scott Jehl

47 Responses to A sample picture implementation

Nicolas Gallagher ✪ February 21, 2012 at 12:27 pm

Thanks Scott. Some feedback:

The stacking of `min-width: 400px`, `min-width: 800px`, etc., should probably behave like it would in CSS. I was expecting all the images to be downloaded at higher viewport dimensions – as all stylesheets would be if using the examples `media` values – but only the larger one was.

When I resize the viewport, smaller images are downloaded. And when I resize to wider viewports, the larger images are re-downloaded. So I can end up with dozens of requests.

- Scott Jehl ✪ February 21, 2012 at 12:41 pm
  
  Hey Nicolas. Yeah, great points.
  
  First pass, I wasn’t sure how best to handle all of that, so this is nice to discuss.
  
  For the rules, I was thinking along the lines of CSS behavior too, but maybe in a different way than you were. I was thinking that if we considered the picture to only allow one valid source at a time, sortof like an element’s background-image value, then the last valid src in the source order would be requested, and ideally, none of the others that it overrides would be requested.
  
  On resize, again like media queries, it’s re-running the logic, and I am not at all sure if that’s desirable for folks here, but it’s mapping to what I think I’d expect to happen with CSS at least, where there’s also potential for many requests after resize. Do you think it should only apply during initial load?
  
  There’s probably precedent for both of these questions with the video element and its relation to source media attributes. Maybe we can make sure this maps to that, if it makes sense for picture.
  
  Thanks!
  
  - Nicolas Gallagher ✪ February 21, 2012 at 1:49 pm
    
    What does `video` do with `source` and `media` like this? It would be confusing if any potential new media element did things differently to `video`.
    
    About the resizing. I don’t think you’d ever want to redownload an image that has already been downloaded. But, ignoring that bug, this early script is a great way to show one way how the end-user experience might turn out for a new element like this. Thanks.
    
    - Scott Jehl ✪ February 21, 2012 at 1:53 pm
      
      Yeah, I’m tending to agree with what you’re thinking about resize. Pending what others say, I’m not sure picture source elements should update at all after initial load…
      
    - Scott Jehl ✪ February 21, 2012 at 1:58 pm
      
      One scenario where it could be useful is if the different sizes use contextual crops (rather than simple resizes). In that case, you’d want the proper size to flip in on resize, I think.
      
      The Nokia browser site used to be a nice example of that, but it doesn’t seem to have different crops anymore.. hmm.
      
      - Tim Kadlec ✪ February 22, 2012 at 2:47 am
        
        I think they’re still doing that in some places. For example, the top image on http://browser.nokia.com/meego-browser.html.
        
        I would personally expect the ‘picture’ element to swap sources on resize. However, considering ‘source’ is it’s own element in the spec, the behavior should be consistent regardless of which media element it is contained in. So, if ‘video’ doesn’t reassess the source, ‘picture’ probably shouldn’t either.
    - Gordon Brander ✪ February 22, 2012 at 2:40 am
      
      I certainly think images should be reassessed after a window resize:
      
      This matches the way CSS media queries work; e.g. which media queries are active is reassessed when resizing.
      Resizing is a bit of an edge case (not many people actively do it), so I don’t see it as a serious performance hit.
      It keeps you from getting into awkward situations. For example, you may want to change things like aspect ratios between breakpoints — that banner image isn’t so great at mobile sizes. If your mobile styles are assuming, say, a square image, not reassessing the image could lead to layout breakage.
      
      - Scott Jehl ✪ February 22, 2012 at 2:50 am
        
        Good point, but orientation changes are assumed to be more common, and usually result in a resize too (unless the device is square? 🙂 ).
      - Gordon Brander ✪ February 22, 2012 at 3:10 am
        
        Point taken, but I still think the other 2 points hold.
      - Scott Jehl ✪ February 22, 2012 at 4:04 am
        
        I’m leaning back towards yes on resize + reassess 🙂
Matt Wilcox ✪ February 21, 2012 at 12:46 pm

Nice work 🙂

“This script does nothing to prevent the fallback image from downloading before it is overridden by another potential match, and I’m not sure it needs to, since that’s an advantage a native implementation will bring.”

I class this firmly as a disadvantage, not an advantage. One of the goals for responsive images is efficiency, and downloading an image that won’t be used is inefficient, both in bandwidth, HTTP calls, and device memory. That’s why the existing JS solutions are all no-go’s.

Scott Jehl ✪ February 21, 2012 at 12:58 pm

Hey Matt!

Oh, agreed. It’s totally a disadvantage. That’s why we’re trying to get a native implementation, right?

If you wanted to make this sort of script production-ready/responsible, tying it together with your script or even a quick inline cookie setting in the head could at least make that fallback request re-route to a blank gif or something less painful.

The current markup is at least unlikely to punish a handheld, since the fallback and the first source element are equal sources.

But yeah, you’re totally correct on production value of this. It’s not intended for that at this point.

Thanks!

- Chris ✪ February 21, 2012 at 1:55 pm
  
  It seems like the question of bandwidth continues to be a limiting issue, and I’ve seen a couple people batting around the notion of @media (bandwidth >= x) – not necessarily in those terms, but that basic concept. I don’t know the browser side of it as much, but it seems like that should be doable in general, right? That would allow for a sensible native implementation of this: combining screen resolution with bandwidth would allow you to prioritize that way, and the same thing would support and nicely. The only potential downside I see is that bandwidth might change mid-streaming of the content. Someone much more familiar with browser attributes/limitations than me might be better equipped to comment…
  
  - Chris Krycho ✪ February 21, 2012 at 1:58 pm
    
    That should read “the same thing would support video and audio nicely”; I wrapped them in code tags, but may have done so incorrectly.
    
  - Scott Jehl ✪ February 21, 2012 at 2:00 pm
    
    Hey Chris,
    Bandwidth is a complicated issue that I’m not sure the picture element is poised to address on its own, but perhaps other apis and media queries will get there one day, and the proposed markup for picture should be able to tap into whatever comes with that.
    
    - Chris Krycho ✪ February 21, 2012 at 2:09 pm
      
      Hmm… yeah, sorry; that comment probably belongs in a different context. I’ve got it on the brain at the moment.
      
      More on topic: Presumably you’re still going to have the ability to save the image (*right-click→save”); how would a native implementation handle allowing the user to save the highest resolution image? Would it default to that with a save action? I am wondering if video and audio already have solutions for that which picture could adopt, so I’ll start by going and reading the specs.
      
      - Chris Krycho ✪ February 21, 2012 at 4:05 pm
        
        Okay, I went and read through the specs for audio and video. It looks like those specs assume that the only time an author is going to want a direct link to those elements – that is, the only time a user is going to be attempting to save those elements – is when the user-agent player is not working, i.e. the fallback case. That makes some sense in the context of those particular elements, I suppose, although I think an argument could probably be made for easier ability to save those elements to local storage for later retrieval.
        
        Pictures are a bit different, though, at least from my point of view. I often want to at least see the highest resolution of an image, even if I wouldn’t necessarily want it loaded on my page. Now, in theory a content provider could wrap a <picture> element in a link to the highest resolution form of the image – that’s already fairly common practice in cases where a thumbnail is being displayed. Should we just assume that <picture> will ultimately be supplying “thumbnail”-like content and wrapping in links if we want the users to have access to that image at a higher resolution? That seems an eminently reasonable approach to me, but I’m curious if people who are thinking much more and harder about this than me already have thoughts on this issue.
        
        Awesome work in any case.
- Matt Wilcox ✪ February 21, 2012 at 2:32 pm
  
  Ahhh, I think I mis-interpreted what you said originally, sorry! I thought you were saying you weren’t sure *the final solution* needed to avoid the duplicate requests. You’re actually saying that the final solution would solve that problem so the ‘bad behaviour’ of the test isn’t an issue?
  
  Doh.
  
  - Scott Jehl ✪ February 21, 2012 at 2:43 pm
    
    You put it far more clearly. So, what you said! 🙂
    
    I’m not sure what a finalized picture element polyfill will entail once the element is actually supported in some places, but I’d imagine the first step would be to return immediately after determining the element is actually supported, and provide a way to easily tie-in server-side workarounds for the double-request problem. It’d be nice if the most basic polyfill, like that above, didn’t care much about that part. At any rate, when it gets to that stage, I’d love your help in planning how that piece can work.
    
    - Matt Wilcox ✪ February 21, 2012 at 3:41 pm
      
      I’m up for helping where I can 🙂 It’s a shame my JS knowledge is poor, and my PHP rusty (Adaptive Images was the first time I’d done any real PHP in 5yrs or more!)
      
      Yeah, as a proof of concept polyfill what you’ve got seems good. Be interesting to see how versatile it can be made.
      
Gordon Brander ✪ February 21, 2012 at 5:34 pm

@Scott Jehl looks like we spent our President’s Day weekend the same way: github.com/gordonbrander/Picturize :).

One thing the script I hacked together has on offer is a Modernizr test, followed by a fast exit if the hypothetical picture tag is supported. Feel free to lift/repurpose.

Scott Jehl ✪ February 22, 2012 at 1:18 am

Hey Gordon! Hah, it appears we did 🙂

Great work. At a glance, it seems these do similar things, which is great to see. As I mentioned above and you did also, a feature test for picture support is missing in the version I’d posted. However, I think there’s some value in a non-jQuery dependent version too, and the matchMedia polyfill nudges quite a few media-query-supporting browsers as well.

Might I invite you in on the picturefill repo so we can get the element test integrated up top in the script? If it needs modernizr, that could be okay, though a quick element test without it might do just as well for now.

I’ve added you as a committer, so let me know if you’re interested 🙂

Thanks and again, awesome work!

- Gordon Brander ✪ February 22, 2012 at 1:42 am
  
  Totally agreed re: jQuery — a plain ol’ JS approach is superior.
  
  I’ll send a pull request your way with those changes later tonight.
  
  - Scott Jehl ✪ February 22, 2012 at 2:05 am
    
    Can’t wait to see it Gordon! Thanks for joining forces on this 🙂
    
  - Gordon Brander ✪ February 22, 2012 at 3:31 am
    
    Sent the pull request your way https://github.com/scottjehl/picturefill/pull/3. I’ll let you do the merge if it all looks fine and dandy to you.
    
    - Scott Jehl ✪ February 22, 2012 at 3:42 am
      
      Gordon, fantastic work, man. I will review this today.
      
Gray Ghost Visuals ✪ February 22, 2012 at 2:45 am

I would definitely expect/assume them to be reassessed on re-size at first thought. Although it seems more HTTP requests are a bit scary at the moment to do the switch. I know this might sound a bit silly, but If only there was a way to make something like this to serve up images for devices/resolution:

@mobile {..} or @mobile and (min-resolution:96dpi) and (max-resolution: 100dpi) {..}
@desktop and (min-width:960px) {}
@print {..}

- Scott Jehl ✪ February 22, 2012 at 3:02 am
  
  Indeed, there are media queries that exist already for that purpose, and you can already use them with this polyfill for browsers that understand them. For example, this line sends higher-resolution icons to HD (or “retina”) screens in jQuery Mobile: https://github.com/jquery/jquery-mobile/blob/master/css/themes/default/jquery.mobile.theme.css#L880
  
  - grayghostvisuals ✪ February 22, 2012 at 3:36 am
    
    A great start/solution for background imagery, but I see its still back to a needed solution to accomplish this for embedded media much like the <video> and <audio> tag already discussed. hmmmmm….more sleep , then coffee. Maybe we should stop embedding images in markup altogether?
    
    - Scott Jehl ✪ February 22, 2012 at 3:41 am
      
      heh. well, if that’s suitable for you, then this https://twitter.com/scottjehl/status/169117397838602240
      
      🙂
      
      - grayghostvisuals ✪ February 22, 2012 at 4:00 am
        
        Awesome! just popped it up on the opera mobile emulator and what a difference. The image was so elastic.
grayghostvisuals ✪ February 22, 2012 at 3:53 am

Side note: I wish resolution detection was more like I referenced in my initial comment rather than device–pixel–ratio currently implemented. It just seems easier to reference and understand the exact resolution over the ratio usage.

Brian McAllister ✪ February 22, 2012 at 7:54 am

Can I play the devil’s advocate and suggest a “reassess” attribute on the picture element.

This would give developers an opt-in to the “reload on resize” question on an element-by-element basis.

Pingback: <picture> for existing browsers. | Responsive Images Community Group
Pingback: Currying Favour with Responsive Web Developers | abitgone+
Pingback: Polyfilling picture without the overhead | Responsive Images Community Group
Brett Jankord ✪ March 15, 2012 at 11:03 pm

@Brian – I think that’s a good idea. I’d like to see an attribute on the picture element, something like reload=”true|false” or some better wording where you could specify if images in the picture element should be reassessed on resize as opposed to reassessing all of them.

Pingback: The Responsive Image Problem | Mouse Potato
Pingback: Picturefill | TRICKFLASH
Pingback: 迷路的猫少年 » Blog Archive » html5需要个picture标签?
Pingback: The Web Needs to Get Ready for the High-Resolution Future | Paramark Inc.
Rodney Rehm ✪ March 28, 2012 at 9:19 am

How would <picture> handle image maps? different sources may very well rearrange the contents of an image, image-maps would then have to be changed accordingly.

- Anselm Hannemann ✪ March 28, 2012 at 10:59 am
  
  Very good point that until now hasn’t been discussed. I think it has to be the same as with the image tag. We need a new spec for ?
  This again questions if we should have a solution like a “MQ-template” as Matt proposed.
  
- Jacob Mather ✪ March 28, 2012 at 1:59 pm
  
  It would actually be much easier to support image maps with this sort of implementation.
  
  Currently, on the image, you just assign which image map to use.
  
  With Scott’s polyfill, and the proposed implementation, we would simply move that to the source tag, which fixes almost ANY problem related to image maps.
  
  Granted, this now means you would have to have an image map for each size you show at, so perhaps there could be some improvements with the imagemap accepting percentages (or some other measurement).
  
  I always saw image maps as a kind of hack anyway, and they’re much less needed now that you can just directly position anchors over an area where you want them to appear.
  
  Maybe the answer is that image maps are dead?
  
Scott Jehl ✪ March 28, 2012 at 11:17 am

image maps are pixel-based, rather than percentages, right? That seems like less of a problem with the picture element and more of a problem with image maps not working in fluid layouts.

You might also try and reproduce an image map technique with positioned anchors in CSS, where you’d likely have much better luck with fluid and responsive layouts.

I’m not sure that this is a problem picture needs to solve though…

- Jacob Mather ✪ March 28, 2012 at 2:05 pm
  
  Agreed. I think anchors in CSS are the better solution anyway.
  
Pingback: Responsive Web Design – Interface - Blog of Web Communications at the University of Missouri

Community & Business Groups

A sample picture implementation

47 Responses to A sample picture implementation

Leave a Reply Cancel reply

Archives

Categories