Application Foundations for the Open Web Platform

Bringing HTML5 to the status of W3C Recommendation (in October 2014) is a defining moment in the development of the Open Web Platform (OWP), a set of technologies for developing distributed applications with the greatest interoperability in history. This year is also the 25th anniversary of the Web and 20th anniversary of W3C, making this an even more meaningful time to engage with the community about the Next Big Thing for the Web Platform.

My starting point for this discussion is that, now that HTML5 is done, W3C should focus on strengthening the parts of the Open Web Platform that developers most urgently need for success. I call this push for developers "Application Foundations."

This is a new formulation, shaped in part by discussion at the September Extensible Web Summit in Berlin, as well as discussions within the W3C staff. I am planning further discussion at W3C's TPAC 2014 meeting at the end of the month, and I welcome your feedback to this post and in the months ahead.

While this formulation is new, most of the work is not new. Rather, this represents a new way of looking at the considerable work that is already in the Web community, and placing some structure around the considerable work in front of us.

The Focus on Developers

The OWP is widely deployed, improving in function every day, and transforming industry after industry. According to a survey earlier this year, 42% of developers are using HTML5, CSS, and JavaScript when building applications. The promise of the Open Web Platform is to lower the cost of developing powerful applications to reach the most people, on any device.

As popular as the OWP is, it is still too challenging for developers to create some types of Web applications. Lack of broad interoperability for some features complicates development. Lack of standard features in the platform drives developers to create hybrid applications, implying a larger mix of tools, libraries, and interoperability issues. There is more work to meet growing expectations around privacy, security, and accessibility.

There are many ways to focus on developers. Many W3C activities outside of standards development are geared toward enabling developers, including tools (validator), documentation (Web Platform Docs), training (W3DevCampus, W3Conf), participation (Community Groups, draft Webizen program).

The question I want to get at in this post, however, relates to our open standards agenda: are we building the platform that developers need? How can we find out?

That is where the Application Foundations come in. They give us a way to think about the Open Web Platform that will make it easier for the W3C community to converge on the top priorities for developers.

What are Application Foundations?

Platforms mature predictably in the following way: at a given time, some capabilities are core and "applications" rely on the core. Invariably, there comes a time when certain features are so pervasively used as services by other applications, the "next generation" of the platform must subsume some of those features (via libraries, platform services, daemons, etc.).

Operating systems provide a familiar example. Typically, an operating system kernel provides the key lower layer functions that a computer needs for its programs (aka applications): program execution, memory management, support for devices, etc. In early versions of many operating systems, there are also higher layer functions (such as networking, security, GUIs, etc.). Often these functions have some manifestation in the kernel, but also some manifestation in applications. Over time, given experience with the higher layer functions, people recognize that some must mature into major subsystems (aka foundations) that are above the kernel, leveraged by many applications. Modular design of these subsystems allows experts in different areas (security, communications protocols, and so on) to deliver solutions that will best serve all the other parts of the platform.

We see this pattern with the Open Web Platform as well. There was a time that video would have been viewed as an application of the Web, but in HTML5, video has unquestionably been absorbed into the core infrastructure (e.g., via the HTML <video> element). An apt metaphor is to call the programmable Open Web Platform of today the first generation operating system of the Web. In the past couple of years, important subsystems have already started to emerge, and in this post I propose a taxonomy of eight Application Foundations to focus our discussion on the next generation:

• Security and Privacy
• Core Web Design and Development
• Device Interaction
• Application Lifecycle
• Media and Real-Time Communications
• Performance and Tuning
• Usability and Accessibility
• Services

Each Foundation represents collection of services and capabilities that should be available for all applications. For example, the Security and Privacy Foundation includes capabilities such as crypto, multi-factor authentication, and resource integrity.

We expect each Foundation to evolve independently, driven by experts in that topic. We also know that there will be interconnections, such as Security implications of Device Interactions, or Accessibility considerations of streaming Media.

Below I will begin to enumerate the capabilities we have associated with each Foundation, both long-standing and new or planned work that will substantially advance the capability of the OWP.

In our internal discussions there was quick consensus on the usefulness of an Application Foundations paradigm. There was also passionate debate about taxonomy itself. Did we come up with one that will speak to developers? Did we neglect some important piece of functionality? Should this or that second-level item be a top-level category or vice versa? To help structure the broader debate to come, I'd like to provide some background for the choices proposed here.

Principles for Thinking about these Foundations

Bearing in mind that we are looking for a best fit to structure discussion, not a perfect dissection, here are some key principles for thinking about these Foundations:

• Although this exercise is motivated by the desire to meet developer needs, we sought labels that would be meaningful for end users as well. We looked for terms we thought would speak to those audiences about both desirable qualities of the platform and current pain points that we need to address.
• These topics were derived by looking at W3C's current priorities and discussions about upcoming work. W3C's agenda is in constant motion, and this taxonomy will only be useful so long as it aligns with priorities. But the river bed shapes the river and vice versa.
• Because the focus is on current developer pain points, we do not attempt to fit all of W3C's work into the eight categories. We are committed to our entire agenda, but this particular exercise is limited in scope. For the same reason, we do not attempt to represent all completed work in this categorization. While we might want to see how broadly we could apply this taxonomy, our priority project is to enable developers today and tomorrow.
• Because the focus is on W3C's agenda, these Foundations do not attempt to represent all things one we think of as being important to the Web, HTTP and JavaScript being two notable examples. Moreover, many key IETF standards (HTTP, URL, IPv6) might more properly be defined as part of the kernel - rather than a higher level Foundation.

Putting the Foundations to Use

Although this framework is meant initially only as a communications vehicle —a way of describing the substantial work we have to do to enhance the OWP— we may find other uses later. Once fleshed out and road-tested, for example, the W3C Technical Architecture Group (TAG) might use this model for architectural discussions about the Open Web Platform.

Ultimately, with such a framework, it becomes easier to identify what is missing from the platform, because we will think more cohesively about its key components. And where there are similar capabilities (e.g. different functions that show up in the same Foundation), it will make it easier to identify where synergies can simplify or improve the platform.

By definition, Foundations are common subsystems useful not only for "horizontal applications", but also for a variety of industries such as digital publishing, automotive, or entertainment. In a separate exercise we plan to work with those industries to create a view of the Foundations specific to what they need from the Open Web Platform.

So let's get started. In each paragraph below, I outline why we think this area deserves to be a Foundation. I list some absolutely critical problems the community is currently addressing. This will help motivate why each Foundation was chosen, and the technology development required to give rise to the next generation Web.

Application Foundations

Security and Privacy

The Web is an indispensable infrastructure for sharing and for commerce. As we have created the OWP, we have become increasingly sensitive to making this a secure infrastructure. See, for example, our 2013 "Montevideo" statement calling for greater Internet security.

The vulnerabilities have become increasingly prominent. They vary in range and style. There are vulnerabilities that result from criminal exploitation of security holes for financial gain. There are numerous situations where information and communications that was intended to be private has found its way into unauthorized hands.

From a pure technical point of view, there is a tremendous amount of security research and there is knowledge on how to make an infrastructure secure. Many security advances are available in devices that are connected to the Web today. Nonetheless, security exposures abound: because it is too difficult for applications to leverage the security that is available; because new security techniques are not yet in place; and because users are not encouraged to rely on strong security.

We do not expect all developers to be security experts, so we must make it easier to use the security mechanisms of operating systems. The Crypto API provides access to some of those services from within JavaScript, and is already being deployed. This trend will be extended as platforms add stronger security such as multi-factor authentication, biometrics, smartcards, all discussed at our September Workshop on Authentication, Hardware Tokens and Beyond. We also need to add a more comprehensive Identity Management system which discourages weak passwords.

To strengthen this Foundation, we are working closely with a number of organizations, including the IETF, FIDO Alliance, and Smartcard Alliance.

Core Web Design and Development

Developers use many widely deployed front end technologies for structure, style, layout, animations, graphics, interactivity, and typography of pages and apps. HTML5 brought native support for audio and video, canvas, and more. Dozens of CSS modules are used for advanced layout, transformations, transitions, filters, writing modes, and more. SVG is now widely supported for scalable graphics and animations, and WOFF is beautifying the Web and making it easier to read.

Still, the work is not complete. Much new work will be driven by the adoption of the Web on a broader set of devices, including mobile phones, tablets and e-book readers, televisions, and automobiles. The responsive design paradigm helps us think about how we need to enhance HTML, CSS, and other APIs to enable presentation across this wider set of devices.

One exciting direction for this Foundation is Web Components, which will make it easier for developers to carry out common tasks with reusable templates and code modules, all leveraging standards under the hood.

Another area of anticipated of work will be driven by a more complete integration of digital publishing into the Web. In the past, advanced styling and layout for magazines has remained an area where special purpose systems were required. In this Foundation, we will ensure that we have the primitives for advanced styling and layout so that all publishing can be done interoperably on all Web devices.

Our Test the Web Forward activity, though relevant across the Foundations, has been particularly influential for Core Web Design and Development, and we invite the community to contribute to that active testing effort.

Device interaction

Closely related to the Core Foundation is the Device Interaction Foundation, which describes the ways that devices are used to control or provide data to applications. New Web APIs are proposed weekly to give access to all of the features offered by supporting devices. For mobile phones, APIs exist or are in development for access to camera, microphone, orientation, GPS, vibration, ambient light, pointer lock, screen orientation, battery status, touch events, bluetooth, NFC, and more.

The next generation of Web attached devices will introduce new challenges. For instance, the Automotive and Web Platform Business Group is developing APIs to access information about vehicle speed, throttle position, interior lights, horn, and other car data that could help improve driving safety and convenience. We anticipate some of that work will advance to the standards track. In general, wearables, personal medical equipment devices, home energy management devices, and the Internet of Things will drive developer demand for data in applications, and for Web abstractions to simplify what will be new complexity in underlying networks. To achieve that simplicity for developers, the TAG, Device APIs Working Group, Web Apps Working Group, and Systems Applications Working Group all have a role to play in capturing good practices for API design.

Application Lifecycle

The proliferation of environments —both mobile and non-mobile— in which an application may run has created new challenges for developers to satisfy user expectations. People expect their apps to be useful even when there is no network ("offline"), to do the right thing when the network returns ("sync"), to take into account location-specific information ("geofencing"), to be easy to launch on their device ("manifest"), to respond to notifications (from the local device or remote server), and so on. The Application Lifecycle Foundation deals with the range of context changes that may affect an application. For example, developers have made clear that that AppCache fails to meet important offline use cases. so we must come up with a superior solution.

The emerging approach ("Workers") for addressing many these lifecycle requirements involves spawning important tasks as asynchronous processes outside of an application. For instance, a Worker can be used to manage a cache and update it according to network availability or receive server-sent notifications, even when an application is not actively running. Enhancing these Foundations these will enable developers to create superior user experiences.

Media and Real-Time Communications

A number of communications protocols and related APIs continue to serve developers well, from HTTP to XMLHttpRequest to Web Sockets. But to meet the growing demand for real-time communications and streaming media, we must add new capabilities, the focus of this Foundation.

The promise of WebRTC is to make every single connected device with a Web browser a potential communications end point. This turns the browser into a one-stop solution for voice, video, chat, and screen sharing. A sample use case driving interest in real-time in the browser is enabling "click-to-call" solutions for improved customer service. WebRTC has the potential to bring augmented reality to the Web and create a brand new class of user experiences - an exciting direction for this Foundation.

For audio and video, developers will have a variety of tools to manipulate media streams, edit audio input, and send output to multiple screens ("second screen"). This last capability is of particular interest to the entertainment industry. For example, in the US, a majority of people have a second device nearby while watching television, allowing for new interactive experiences such as social interactions or online commerce.

Performance and Tuning

Open Web Platform functionality has moved steadily to the client side, which creates a variety of new challenges related to security, application lifecycle management, but especially performance. JavaScript engines have improved dramatically in the past few years. But for high-end games, media streams, and even some simple interactions like scrolling, we still have much to do so that developers can monitor application performance and code in ways that make the best use of resources. This is the focus of our Performance and Tuning Foundation.

Today we are working on APIs for performance profiling such as navigation timing and resource hints. In various discussions and Workshops, people have asked for a number of enhancements: for understanding load times, enabling automatic collection of performance data, providing hints to the server for content adaptation, improving performance diagnostics, managing memory and garbage collection, preserving frame rates, using the network efficiently, and much more.

The responsive design paradigm mentioned in the Core Web Design and Development Foundation also plays a role in the world of performance: we can make better use of the network and processing power if we can take into account screen size and other device characteristics.

Usability and Accessibility

The richness of the Open Web Platform has raised new challenges for some users. It is great to be able to create an app that runs on every device, but is it easy to use or klunky? It's great to offer streaming media, but do developers have the standards to include captions to make the media accessible?

Designers have pioneered a number of approaches (responsive, mobile first), that can improve accessibility and usability, and W3C's Web Accessibility Initiative has developed some standards (such as WCAG2 and WAI-ARIA) to enable developers to build accessible applications. But we have more work to do to make it easier to design user interfaces that scale to a wide array of devices and assistive technologies. We have confidence that designers and developers will come up with creative new ways to use standards for new contexts. For example, the vibration API used by some mobile applications might offer new ways to communicate safely with drivers through the steering wheel in some automotive apps, and could also be used to create more accessible experiences for people with certain types of disabilities.

Less than one third of current Web users speak English as their native language and that proportion will continue to decrease as the Web reaches more and more communities of limited English proficiency. If the Web is to live up to the "World Wide" portion of its name, it must support the needs of world-wide users at a basic level as they engage with content in the various languages. The W3C Internationalization Activity pursues this goal in various ways, including coordination with other organizations, creation of educational materials, coordination on the work of other W3C groups, and technical work itself on various topics.

Services

Earlier I mentioned the pattern of widely used applications migrating "closer to the core." While this is true for all the Foundations, it is especially clear in the Services Foundation, where today we are exploring the four most likely candidates for future inclusion.

The first is Web payments. Payments have been with us for decades, and e-commerce is thriving, predicted to reach $1.471 trillion this year, an increase of nearly 20% from last year. But usability issues, security issues, and lack of open standard APIs are slowing innovation around digital wallets and other ways to benefit payments on the Web. W3C is poised to launch a group to study the current gaps in Web technology for payments. The Payments group will recommend new work to fill those gaps, some of which will have an impact on other Foundations (e.g., Usability, Security and Privacy). Because a successful integration of payments into the Web requires extensive cooperation, the group will also liaise with other organizations in the payments industry that are using Web technology to foster alignment and interoperability on a global scale.

The second is annotations. People annotate the Web in many ways, commenting on photos or videos, when reading e-books, and when supporting social media posts. But there is no standard infrastructure for annotations. Comments are siloed in someone else's blog system, or controlled by the publisher of an e-book. Our vision is that annotations on the Web should be more Web-like: linkable, sharable, discoverable, and decentralized. We need a standard annotation services layer.

The third is the Social Web. Consumer facing social Web services, combined with "bring your own device (BYOD)" and remote work policies in enterprise, have driven businesses to turn increasingly to social applications as a way to achieve scalable information integration. Businesses are now looking for open standards for status updates (e.g., Activity Streams) and other social data. These same standards will give users greater control over their own data and thus create new opportunities in the Security and Privacy Foundation as well.

The fourth is the Web of Data. The Semantic Web and Linked Data Platform already provide enhanced capabilities for publishing and linking data. These services have been used to enhance search engines and to address industry use cases in health care and life sciences, government, and elsewhere. But we know that more is necessary for developers to make use of the troves of data currently available. One upcoming activity will be to collect ontologies of how linked data should be organized for different applications (notably for search).

Conclusion

Web technology continues to expand by leaps and bounds. The core capability is growing, the application to industry is growing, and we continually find new devices for web technology and new use cases. To be able to focus on this expansion we need modular design, and a principle in modular design is to be able to clearly and succinctly talk about categories of function. Hopefully this post begins a healthy discussion about the framework for the Open Web Platform going forward.

As part of that discussion will continue to develop a new Application Foundations Web page and invite feedback via our public Application Foundations wiki.

Acknowledgments

I acknowledge extensive discussions within the W3C Team, but especially with Phil Archer, Robin Berjon, Dominique Hazaël-Massieux, Ivan Herman, Ian Jacobs, Philippe Le Hégaret, Dave Raggett, Wendy Seltzer, and Mike Smith. At the Extensible Web Summit, I received great input from Axel Rauschmayer, Benoit Marchant, and Alan Stearns.