With 1,134 commits in the last week, development certainly is getting back on track after the holidays. This week’s highlights include some plans for improving the CSSOM, implementation plans for CSS Variables and the landing of Interactive Validation within WebKit. I won’t address H.264. Enough people have done that.
If you’re interested in updates related to HTML, Anne van Kesteren published an item on recent changes happening around the WHATWG. The article mostly covers discussions which took place on the whatwg@whatwg.org mailing list, and certainly is an interesting read!
Tab Atkins, member of the CSS Working Group and the Google Chrome team, has published the slides of a presentation he hosted last Wednesday. It talks about how the CSSOM, the interface between JavaScript and CSS, basically sucks. The presentation demonstrates four relatively new concepts:
CSS Variables
By declaring a variable through the @var rule, its type and the value, anything from colors to lengths, rotations and functions can be defined and will be available to all CSS used in the document. Support for local variables will be available as well, through the @local at-rule.
@var petersh-paragraph-size length 12px;
@var petersh-paragraph-color color rgba(0,0,0,0);
p { . color: var(petersh-paragraph-color); . font-size: var(petersh-paragraph-size);
}
CSS Mixins
Mixins come down to being able to extend any rule with certain, common properties. They take parameters similarly to CSS Variables, including the ability to have default values for them.
CSS Nesting
Nesting in CSS, especially when used without classes or IDs, can be a pain. The @this at-rule will be introduced for usage in CSS rules. The selector you supply behind it will apply only to descendants of elements matched by the current rule. That clears the way for structures like this:
CSS Modules
No, not the CSS Modules which make up CSS Level 3. CSS Modules basically are namespaces for CSS Variables and Mixins. Mixins and variables can still be referred to by prepending their name with the namespace’s name, or by including the @use at-rule to have the parser include all of them.
While the idea is fine, I personally think this shouldn’t be part of the specification. The use-case is easily obtainable by advocating authors to prefix their variable names, like “petersh-header-color” instead of “header-color”. Two types of namespacing, sharing similar syntax, is just going to cause confusion.
While the current code is only available on a few machines in Sydney, Google is planning to publish an initial specification this quarter, and implement the changes before the end of the year. Considering neither these plans for CSS Variables nor the CSS Mixins and Nesting have been shared on the www-style mailing list yet, it’ll be interesting to see how it will be received by other vendors. Related discussion from a few years ago has shown that it’s a fairly controversial issue, with many opinions being far apart.
Last Thursday, Kent Tamura landed support for the Interactive Validation UI using the new Shadow DOMmodel. With most bugs out of the way, messages and an –albeit internal– API being available for Chromium to enable the feature, it won’t be long before it becomes available in the nightlies.
In other specification-related news, the scroll event will now be fired asynchronously. The WebKitCSSMatrix object in the DOM now acceptsnone as a valid keyword, and using the toDataURL method for a canvas on Chromium will now composite onto a black surface when the destination format does not support an alpha channel.
Other changes which happened last week:
Tony Gentilcore now is a WebKit reviewer, congratulations!
Adam Barth is working on a proposal for embedding a security policy in an HTML document.
He also announced to have finished moving WebKit’s source files to a separate directory.
The about:sync page has been updated to use the new DOM UI displaying system.
The media player from WebKit’s GTK port now features a slider to change the volume.
Simon Fraser wrote an excellent article on his CSS3 Gradients implementation.
Support for dropping data to WebKit2 has been implemented for Mac OS systems.
Password synchronization has been enabled by default for Google Chrome.
Some issues with Safari’s strict mode implementations have been fixed.
The security model used chrome://-URLs has been updated, as described in this message.
Development is slowly getting back on track with the 851 commits which were done last week. This week’s highlights include the availability of a per element JavaScript full screen API, a prefixless Navigation Timing implementation and the addition of support for repeating CSS gradients.
With both Microsoft and Google strongly pushing the Navigation Timing specification, both vendors finished most of their implementation and a Last Call specification will be published tomorrow. Because of this, WebKit has dropped the vendor prefix and Microsoft is very likely to do the same. Opera and Mozilla have expressedconcerns, but seem supportive as well. It’ll certainly become an interesting Last Call phase.
As for other specification-related changes, text emphasis marks won’t appear over characters that have ruby annotations anymore. Support for repeating gradients has been added as well, completing the set of gradient related CSS properties. Negated selectors will be given the specificity of the selector they’re negating, and parsing of the argument for the :nth-child CSS selector has been updated and now accepts whitespace, but rejects invalid values.
...
var elem = document.getElementById("my-element");
elem.onwebkitfullscreenchange = function () {
console.log ("We went fullscreen!");
};
elem.webkitRequestFullScreen ();
..
After several months of writing and updating patches, Jer Noble landed the final part of his work on supporting per-element full screen last Friday. While the implementation is only available in WebKit nightlies on Mac OS systems so far, I’d expect a Windows nightly rather soon and Chromium is interested as well.
WebKit’s implementation follows Mozilla’s proposal with only a few exceptions: instead of accepting flags for requestFullScreen, an additional method called requestFullScreenWithKeys has been made available. The following three methods have been added for the Document and all Elements in the DOM:
void webkitRequestFullScreen();
void webkitRequestFullScreenWithKeys();
void webkitCancelFullScreen(); // only on Document
Furthermore, a new attribute called allowFullScreen has been introduced for iframe elements, the fullScreenChange event has been added and three CSS pseudo-classes have been added, namely :full-screen, :full-screen-doc and :full-screen-root-with-target. Mind that each of these additions has been prefixed with “webkit” or “-webkit-“, depending on the context.
Want to try the feature yourself? All you need is a Mac OS X system on which you can run WebKit nightlies. Download and install the latest one and open up Terminal. Within it, execute the following command to enable the feature:
After (re)starting the nightly, the exposed API will actually launch an element in full screen. Click on the icon to the left of this paragraph to display this very article in full screen. Using it to display a single element in full screen can be done too, but still is a bit unstable at the time of writing.
Other changes which occurred last week:
The disabled property on stylesheet elements will now disable the sheet altogether.
Adam Barth moved the WebCore directory, containing most of WebKit’s code, to Source.
A memory sampler has been added to WebKit, primarily for usage in Safari on Mac OS X.
The onbeforeunload event is now available for framesets as well.
SGML-style comments within <style> elements will now be ignored when using XHTML.
Tabbed options (also known as WebUI) have been enabled by default in Chromium.
Per an update in the HTML5 specification, framesets will now be allowed after hidden inputs.
The Web Audio API received some more work for the Chromium port.
Mihai Parparita now is a WebKit reviewer, congratulations!
The PNG compression settings for Chromium have been optimized for performance.
Inline script execution will now block on pending stylesheet loads, as other browsers do.
The O3D Plugin API will now work with Opera 11 as well.
Web Inspector has been enabled for all internal chrome:// pages.
The icon for active background extensions over the wrench menu in Chrome has been updated.
The position of Notifications can now be changed via a notification’s option menu.
This week might bring four times faster JavaScript to Samsung’s EFL port, support for the object-fit CSS property and, hopefully, much more patches from the Web Inspector team. Have a good week!
Happy New Year! Even though there were only 354 commits in total, there certainly were some interesting subjects to write about. Highlights include cleaner gradient syntax for WebKit, support for the :in-range and :out-of-range CSS Selectors and background extensions enabled by default.
There are two things which I’ll change with the weekly updates, starting at this article. Firstly, I’m no longer forcing external links to open up in new tabs/windows: clicking them will navigate away from the article. Furthermore, I have added tags indicating the latest Chromium version at the time of publishing, making it easier to determine what changed in a major Chrome release: version 7, 8, 9 and the latest: Chrome 10.
Following Tab Atkins’ promise that the gradient part of the CSS3 Images specification won’t change anymore, Simon Fraser was quick to jump in and implement the gradient syntax in WebKit. This is good news for the cutting-edge developer, as WebKit’s own proprietary syntax won’t be nessesary anymore in the near feature.
So far, two patches for supporting the new gradient syntax have landed. The first one implements the syntax for non-repeating linear gradients, including support for non-percent based and negative lengths. The second patch adds support for radial gradients through the -webkit-radial-gradient property, which includes shapes and sizing for the ellipse. Mind that only circular ellipses are supported for now.
Hopefully the new CSS3 Images module will be published as a Working Draft soon, as Tab Atkins is intending to request it. The new draft introduces the ability to use elements as backgrounds, as well as a more convenient way of cross fading background images.
The Web Inspector team didn’t seem to care about the New Year either and landed various patches. The protocol has been cleaned up and was enriched with Network, DOM Storage and Database domains. Disabled style properties get enabled when they’re edited and pasting a full property declaration will now split it up in a property and value automatically.
Other changes in this week include:
Support for HTML5’s :in-range and :out-of-range CSS selectors has been added.
Viewing the source of a pinned/app window won’t clone the tab’s state anymore.
The JSON.parse method in JavaScriptCore now properly exists as a function taking two parameters.
With the addition of the Surface3D class, the Pepper3D API is now complete, but not fully functional yet.
JavaScriptCore now recognizes vertical tabs and form feeds as valid JSON whitespace characters.
Accidental white-space for Chromium OS usernames will now be removed.
Sandboxed iframes no longer have access to the top.history methods.
Periods in Google Apps account names won’t be ignored anymore in Chromium OS.
WebKit’s client-based GeoLocation has been enabled for Chromium.
Background extensions have been enabled for all platforms, except for Chromium OS.
Submitting forms using the GET method to the current location as its action won’t reload the page.
Where my first day of work in 2011 has already passed, yours might just be beginning. See you next week!
With the commit count being up 52 percent compared to last week, totaling 1,184 commits, activity has been a lot higher than two weeks ago. Highlights include Google Chrome 8, tomorrow’s announcement and lots of updates to standard support.
Exactly six weeks and over 8,472 commits after the release of Google Chrome 7, Google has updated the Stable channel to the 8th version of their browser. This release brings support for Element.classList, asynchronous script loading, support for the steps timing-function for transitions, the about:flags page and hyperlink auditing, enabled by default.
As a quick peek-ahead towards a feature for Google Chrome that’s currently scheduled for version 9, which would be around mid January: Google Cloud Printing. The idea is simple: connect your printer with GCP and you’ll be able to print to it from any computer or smart phone, regardless of where you are. Probably unrelated: an event will be happening in San Francisco tomorrow morning sharing some exciting news about Google Chrome. While I have a fair idea what it will be about, there is little point in morespeculation.
WebKit has taken a huge leap forward in terms of support for various standards. ArrayBuffers may now be transmitted using XMLHttpRequest’s send-method, a form’s elements property now includes fieldset and keygen elements, getBoundingClientRect won’t truncate the coordinates to integers anymore and focussed <area> elements will no longer use the image’s focus-styles.
An <input type=color> will no longer accept named colors and the incremental property is now available on the DOM of all input elements. The toDataURL method of a canvas’ context in Chromium can now export the image as a JPEG, the marquee element’s properties have been updated per the HTML5 specification (besides the events) and, while they don’t have visual effects yet, support for the four text-emphasis properties from the CSS3 Text module was added.
Finally, support for both lower and upper Armenian list style types has been implemented and initial versions of the HTML5 <details> and <summary> elements have been added by Luiz Agostini. Even though they aren’t interactive yet, work has started!
Other interesting changes which occurred last week:
Option elements will not be bold anymore for certain Chromium versions.
The about:flags page will now show a drop-down for items where multiple values are possible.
Hyperlink auditing (<a ping>) may now be disabled via the about:flags page.
No, of course I didn’t forget Web Inspector. The “revert to revision” system has been implemented for the revision-system in the Resources panel. The display name of an object will now equal its constructor rather than its type, copy(node) in the Console works again, the cookies tab for a Network resource will only show for Chromium, and, finally, the Inspector protocol has been cleaned up.
As already mentioned earlier in the post, the Google Chrome team scheduled an announcement for tomorrow. Be sure to keep an eye out for news and updates, I’ll certainly devote some tweets to the announcement!
Last week the projects received another 1,126 commits, 615 for Chromium and 511 for WebKit. Some highlights include updated Terms of Service and the removal of the CSS Variables code.
To start off with improved standards support, the stepUp and stepDown API methods for certain input elements have been updated to match the specification in case step-mismatching occurs. The unloadEventStart property has been added for the Web Timing implementation.
In scope of the Web Audio API, event hooks were added for JavaScript Audio Nodes, as well as custom bindings for AudioNode. More work has been completed on vertical text as well: support for vertical text in ruby was enhanced and fonts with no vertical metrics will now synthesize baselines.
WebKit’s implementation of CSS Variables has been removed. Even though it has been disabled for ages and the implementation never shipped in stable builds, I’m quite a fan of variables. It just makes creating and, more importantly, maintaining large websites a lot easier. Nevertheless, since Google really wants the feature to be implemented, be sure to keep an eye out for this bug if you’re interested as well.
Implementation is one thing, but getting them standardized and available in other engines won’t be easy. With an essay like this around from a member of the CSS Working Group and a lot of discussion in general, reaching a consensus on any syntax, behavior or draft won’t be simple.
Other updates which occurred last week:
Google Chrome’s Terms of Service have been updated, adding additional terms for Enterprise use.
The style property on attribute nodes has been removed, except for Objective-C bindings.
Internally Web Inspector’s Storage Panel has been renamed to Resources Panel as well.
CSS files included for Chromium’s Web Inspector will be concatenated.
WebKit now really passes the Acid 3 test, as two bugs were cancelling each other out.
Synchronization of resizing the renderer for accelerated compositing no longer is Darwin-specific.
Downloads in Chromium may now be restarted if they previously have been cancelled.
Asynchronous file writers will now be available for Web Workers.
The fullscreen UA-stylesheet will now only be injected when the document is in fullscreen mode.
Multiple background images on an element won’t cause repeated repaints anymore.
More message-functions have been added in preparation of full support for interactive validation.
Navigating on websites with dark backgrounds used to result in white flashes – it won’t anymore.
The focus ring on image maps has been updated to take zooming into account.
The spatial navigation’s node selection algorithm has been updated quite significantly.
The network-error pages will now only contain gradients if the user’s display supports high color depths.
DirectX diagnostics will now be gathered asynchronously on about:gpuinfo due to performance reasons.
A search box has been added to the DOM UI, but it doesn’t actually do anything yet.
The Omnibox Extension API has been moved out of its experimental status.
The “Edit Bookmark” dialogs on Chromium for Windows are now resizable as well. Great change really.
And that’s all for this week. If you happen to be a member of Fronteers, the annual general meeting will be held on the 24th of November. Furthermore, I’ll be giving a presentation about the Audio APIs on the 13th of December in Groningen, the Netherlands. If you speak Dutch and would like to attend, feel free to sign up!
Although, statistically seen, it has been quite a regular week with 1,096 commits, 494 for WebKit and 602 for Chromium, there have been some very nice changes and announcements over the past seven days.
As for ongoing work, Dave Hyatt added support for selections on vertical text, and made sure that repaint invalidation now works with vertical lines as well. Dan Bernstein committed basic support for multiple writing modes in tables, including support for collapsing borders.
Meanwhile, Chris Rogers has been busy landing parts of the Audio API in WebKit. The RealtimeAnalyser (and its Node) landed about a week ago, just like the ConvolverNode and the AudioBufferSourceNode. A class to pass on the active buffers to the onaudioprocess event, named the AudioProcessingEvents Interface, has been committed as well. With more patches in the queue, progress on the API steadily continues.
Several announcements were done during one of the keynotes at Adobe’s MAX conference last month, some of which illustrated Adobe’s interest in HTML5. Mark Anders introduced EDGE, giving web developers an interface similar to Adobe Flash for creating HTML-based animations on their websites. Furthermore, it was also announced that Adobe will be contributing an animation framework to jQuery, as well as proposing and contributing changes to WebKit.
Last week Paul Gubbay demoed one of the prototypes the team at Adobe has been working at, showing text-wrapping around arbitrary shapes. It actually consists of two parts: a JavaScript framework, which is likely to work with jQuery, allowing the user to move the text and image around, as well as handling text reflows. The WebKit-side of the demonstration exists of a CSS property in which a polygon gets defined. This polygon defines a region to which text should be clipped, or from which text should be excluded.
Adobe is working together with Google to accelerate the process of landing the changes in WebKit, and thus making them available in a browser release. While the CSS property’s syntax is expected to evolve following community feedback, Adobe certainly intends to propose the feature to the CSS Working Group.
In terms of standards compliance, getComputedStyle received an update allowing you to retrieve all backgrounds instead of just the first one. The window.name property will now return an empty string for unnamed windows and frames and, in preparation of landing the actual interactive validation UI, a framework for showing the messages has been added as well.
Kenichi Ishibashi added support for the HTML5 <output> element, which is intended to represent the result of a calculation of two or more other form fields. After a way too long period of time, Erik Arvidsson landed an adjusted version of my patch to support the unprefixed box-sizing CSS property. Finally, since the IETF now seems to consider prefixed HTTP headers harmful, the “X-Purpose” header has been renamed to “Purpose”.
More updates which occurred last week:
The integrated PDF viewer has landed in Chrome’s Beta channel.
A minor update was done to the EULA of Chrome OS, clarifying the concept and implications of themes.
All the poor folks with e-mail addresses ending with @google.com have hard-coded screen locking.
Many thanks to Paul Gubbay and Alexandru Costin from Adobe for answering my questions. Also, this page contains a clear lead about what’s (hopefully!) to expect for next week 😉
With 632 commits to the WebKit repository, and 608 towards the Chromium one -totalling 1240 commits-, it was a busy week. Safari seems to be gearing up for a new minor release, and Google pushed Chrome 7 to users participating in the beta-channel. Google also published a page explaining the differences between extensions, Packaged Applications and Hosted Applications.
Nikolas Zimmermann has done some amazing work on WebKit’s SVG implementation: almost all SVG Text layout-code has been rewritten. Because of this change, text in SVG files already consumes much less memory and performs better than it used to do. By splitting the layout process for texts into three phases rather than a single one, future patches can add various forms of caching. This will improve the rendering performance even more.
Following the WebM project, which provides a free and open-source video codec, Google has announced WebP: an image format based on VP8’s intra-frame techniques. According to Google’s announcement, using WebP will reduce the size of your images by an average of 39%, compared to today’s image formats.
In reality, I’m not so sure. Jason Garrett-Glaser, an x264 developer, concluded that the quality is poor, and that Google’s timing for announcing the format is odd. Jacob Miller, obviously being less biased, concluded that the compression schema indeed outperforms JPEG, but that WebP isn’t ready for real-world usage yet.
I absolutely agree that the timing surrounding this announcement is weird. There are some important features not (yet) available in WebP which could prove to be decisive in the success of the image format. I’m mainly talking about the limited file size it supports (a maximum of 16383 by 16383 pixels), no support for storing lossless images and no transparency (nor translucency?). For a future-proof image format, Google could also have looked at supporting other color-spaces (possibly even non-RGB, like the CIE XYZone). It makes me think like the announcement was a bit rushed, especially due to a sentence starting with “we plan to add”…
In my opinion, one of the primary things lacking for web development was a convenient way to modify the classes which applied to an element. While jQuery offered some excellent methods to do so, a proper native way wasn’t available. For that reason HTML5 introduces the “classList” property, which provices such an interface to each element on your page. While support for the property was added to Firefox well over a year ago, Erik Arvidsson added support to WebKit last Monday!
The <input type=number> element now supports theme-drawing on Windows for the spin buttons.
Speech input has now been enabled by default for all platforms within Google Chrome.
Starting next Thursday, Fronteers 2010 will be taking place at Pathé Tuschinski in Amsterdam. With speakers like Håkon Wium Lie, Christian Heilmann and Jeremy Keith it’s bound to become a success :). Finally, thanks to Finnur Thorarinsson for informing me about the issue, the Chromium Command-Line Flag RSS Feed will now properly include added arguments. They previously were included as if they got removed.
With just over a thousand commits in the last seven days, the majority of which were pushed towards the WebKit repository, activity seems to be slightly down in comparison with the previous weeks. Nevertheless, last week brought some interesting changes: changes to the script element, CSS timing functions and Hardware Acceleration for the masses.
On Wednesday Google made an announcement which was quite hard to miss: Google Instant. Google Search anticipates on what you’re going to search for and starts displaying results while you’re still typing. Chrome now features a similar possibility named match preview, although I think Chrome Instant sounds more appropriate. You can enable it by supplying the –enable-match-preview flag when launching Chromium, but keep in mind that the implementation still is rather rough.
Since the new HTML5 Parser and Tree Builder in WebKit kept timed script execution in mind, Tony Gentilcore was able to land support for <script async> only a few days after he added support for the defer attribute. To re-iterate, using the defer-attribute defers executing the script to after parsing the page has been completed. The async-attribute enables asynchronous execution of the script as soon as it’s available, therefore not blocking the parser.
Following the discussions of the face-to-face meeting of the CSS Working Group three weeks ago, Apple’s Dean Jackson modified the CSS3 Transitions and Animations modules to include a new timing function called “steps“. The name is fairly obvious: instead of having a continuous transition, the selected properties transition in a predefined number of steps. This timing function landed in WebKit last Thursday!
As you can see, all seven timing functions for transitions have been included in this example. The animated color-boxes in the JavaScript column show what a browser should be doing according to the specification, while the CSS column shows how your browser displays it. Furthermore, if you’re using a modern browser, you can see an animated graph displaying how a certain timing function works by clicking on its name!
Other updates which occurred in WebKit and Chromium last week include:
Using the middle mouse-button no longer fire the onclick event, matching IE and Firefox.
Finally, for the ones of you who like to be up-to-date as well, I’ve added RSS feeds for my Vendor Prefixed CSS Properties page (feed) and the overview pages of Google Chrome Command Line Switches (feed). Since most content on these pages gets updated automatically, I figured this would be a nice addition. While I cannot guarantee that they already work perfectly at this point, in theory they should be fine. See you next week!
Last week the CSS Working Group met at Opera’s office in Oslo, Norway, for a face to face meeting. Following tight planning, the members met three days in a row discussing topics ranging from the open CSS 2.1 issues, various CSS 3 modules and other subjects such as hit testing. Some of the results are clear: all open CSS 2.1 issues have been resolved and a range of specifications will have their priority increased (such as CSS Transitions and Transforms).
Furthermore, CSS 2.1 is expected to become a Proposed Recommendation by the end of the year. This would mean that the specification could be a W3C Recommendation early next year, allowing the working group to focus their attention to CSS 3 and beyond. During the meeting Mozilla’s David Baron also mentioned that Firefox will be implementing 3D Transforms, already available in Safari and Google Chrome.
As for Chromium and WebKit, a combined amount of 1282 commits were uploaded to their repositories. While this means there were fewer commits than to last week, there’s a lot more news to share about the projects. I’ll highlight some interesting items which occurred last week, and briefly list other interesting changes.
Firstly, it’s becoming more and more obvious to the Chrome team that their browser is lacking important features for the enterprise market. An area Google can tackle is policies. Policies are a way of defining the settings of the browser through the registry, Microsoft’s Administrative Template files or the, so far unannounced, ChromeOS Enterprise Daemon. Other policy and preference stores may be added in the future.
Policies allow companies to easily define common settings such as the proxy server to use, account synchronization and whether JavaScript should be enabled for websites. Unfortunately this also enables companies to block Chrome updates, but I’m sure the Chrome team will be looking at options to prevent people from doing this. Last week support for threenewpolicies was added.
Another large update is the initial inclusion of the Google Chrome Labs page. Most other Google products, as well as Google itself, include a page with experimental features. Considering Chrome supports about 320 command line flags it won’t surprise you that adding such a page makes certain tests a lot more accessible. Google’s Nico Weber committed the initial version just over four days ago. You can try it out yourself by downloading a recent nightly and visiting about:labs.
The WebKit team has invested a lot of time in improving their support for various standards. Adam Barth and Eric Seidel enabled the last part of the new HTML5 Tree Builder: fragment parsing. Furthermore support for HTML5 compliant doctype switching was added, symbolic CSS3 list-style-types are now supported and file inputs now respect HTML5’s fake path. Finally, due to this addition, you can now use HTML5’s date input types to start making plans for your birthday in the year 275759.
Now that the new Tree Builder has been completed, except for a lot of fine-tuning of course, thousands of lines of code were up for deletion. The old Tree Builder itself wast removed on the 24th of August. Further cleanups were done with the removal of their current implementation of Mozilla’s XML Binding Language (XBL). It hadn’t been maintained in years, so the decision was made to remove it in total.
Starting next Thursday I will be in Brighton, United Kingdom. Together with Krijn, Anne and Matijs I’ll be attending dConstruct 2010. Perhaps I’ll be seeing you there? 🙂
In the past few years a whole range of visual effects have been standardized. Future websites can render pretty much anything using bitmap canvasses, display 3D content using CSS 3D Transforms or WebGL and even implement entire key-frame based animations using nothing but CSS. Combined with specifications like the Application Cache and Local Storage, “HTML5? enables a whole new range of web-based applications.
Unfortunately, now that almost everything can be visualized on your monitor, the inability to synthesize, process, and analyse audio streams is becoming more and more obvious. While Flash provides fairly extensive APIs for working with sound, having a native (and preferably more extensive) API available to synthesize, process, and analyse any audio source is much more convenient. That’s why the W3C Audio Incubator Group was founded!
Don’t get too excited just yet: while an initial draft has been published by Google’s Chris Rogers, you shouldn’t expect the API to be finished within the year. The initial version received lots of input from six Apple engineers: Maciej Stachowiak, Eric Carlson, Chris Marrin, Jer Noble, Sam Weinig and Simon Fraser, and now frequently gets updated based on feedback received via the mailing list. The draft specifies various features for the API: spatialized audio, a convolution engine, real-time frequency analysis, biquad filters and sample-accurate scheduled sound playback. Wait, spatialized what?
The reason why it doesn’t exist already
The complexity involved with synthesizing, processing, and analysing audio is one of the key reasons why it doesn’t exist already. Most audio today has a sampling rate of just over 44 thousand samples per second; tracks of DVDs and blu-ray discs can be as high as 192 thousand samples per second. When multiplied by the number of sound channels and considering the decoding required to make sure the file makes sense, you can imagine the amount of work that goes into translating that MP3 file to waves our ears can interpret.
Of course, part of this process is handled by hardware, like converting the digital stream to an analog signal. However, applying effects to an audio stream happens entirely in software where each sample gets processed. In situations where effects are applied and the processed sound is played back almost simultaneously, you can imagine how critical things like buffering and timing are.
Another problem is JavaScript performance. While the scripting engines have become way more powerful in the last few years, they can be in the order of twenty time slower than well optimized native code. When used in combination with one of the SSE instruction sets, which enhance your processor with highly optimized abilities to do audio-related math, today’s scripting engines still got a long way to go.
Native processing to the rescue: just create an API
Performance can be improved by moving most of the processing away from JavaScript. By providing the Application Programming Interface (API) the Audio Incubator Group will likely be proposing that your script gains the ability to “describe” what you want to be doing, rather than doing it. Right now, however, work is being done to implement an interface allowing direct JavaScript processing in the API. Such an interface could be used to prototype audio processing algorithms and creating educational demos, something which already was a possibility using Adobe Flash and Mozilla’s Audio Data API.
The idea is simple: the “base” is an AudioContext interface which manages connections between the different Audio Nodes. The context contains a Destination Node by default, which represents the output device on your computer. This could be your speakers, your headphones or, perhaps in the future, even as a file on your harddrive.
Of course, there have to be audio sources as well. There are various kinds of sources: MediaElementAudio- SourceNode for <audio> and <video> tags and AudioBufferSourceNode for other kinds of input, like MP3 files requested via XHR. Other types are yet to be defined, but source nodes like DeviceElementSourceNode aren’t unthinkable, which could be used to process microphone input via the <device> element.
Between audio sources and destinations, there can be other types of nodes to perform various kinds of manipulations. The specification currently defines the following interfaces:
AudioGainNode Allowing you to change the volume of the audio.
AudioPannerNode Positioning and spatializing audio in a 3D space.
BiquadFilterNode Add lowpass, highpass, and other types of common filters to the audio.
ChorusNode Add a chorus effect to the audio.
ConvolverNode Add effects to audio, such as imitating the sound of a concert hall.
DelayNode Apply dynamically adjustable delays to an AudioNode.
WaveShaperNode Adding non-linear waveshaping effects, like distortion.
These nodes form the foundation of many of the features currently available in audio systems, but the specification is still far from finished and more types of nodes may be added. For analysis you could use a RealtimeAnalyserNode, which allows you to analyse the audio node in real time. This could be used for example, to display the tones output by a stream.
An example: dynamically changing the language of a video
Currently there is no clean way to switch between alternative audio streams for a HTML5 <video> element. The Audio API is ideal for such a purpose. When you keep a number of things in mind, like fragmenting the audio in smaller files to speed up the (initial) loading, it won’t be hard to create a language switcher:
Create an AudioContext,
Get the audio sources from the <video> element using a MediaElementAudioSourceNode,
Decrease the volume of the video using an AudioGainNode,
Get the new audio stream by requesting the MP3 via XHR and putting it in an AudioBufferSourceNode,
Combine the two using the Dynamics Compressor (DynamicsProcessorNode),
Play the audio stream.
This can be demonstrated using the following diagram:
These same techniques could be used to dynamically control background sounds for clips and create timed effects for games using an arbitrary number of output channels (which could be 2 for stereo, 5.1 for surround or even more!). Of course, more normal use-cases can be thought of as well: a beep when you click on a button, messages when interactive validation in a form fails or a music player featuring cross-over effects.
A number of examples demonstrating the capabilities of the Web Audio API are available as well, but keep in mind that you have to build WebKit yourself. They do show the involved JavaScript code however!
I’m really interested in the progress of the Audio Incubator Group and can see quite some benefits in being able to synthesize, process, and analyse audio through JavaScript. I’ve signed up to their mailing list and follow prototypes in Gecko and WebKit. Are you interested too? Consider following @AudioXG on Twitter or subscribe to the public-xg-audio mailing list at the W3C — lots of cool things are yet to be invented!
Thanks and credits to Chris Rogers and Koen ten Berg for their technical input and feedback!
Last Thursday, Kent Tamura landed support for the Interactive Validation UI using the new Shadow DOM model. With most bugs out of the way, messages and an –albeit internal– API being available for Chromium to enable the feature, it won’t be long before it becomes available in the nightlies.
After (re)starting the nightly, the exposed API will actually launch an element in full screen. Click on the icon to the left of this paragraph to display this very article in full screen. Using it to display 
In terms of standards compliance, getComputedStyle received 
Since the new HTML5 Parser and Tree Builder in WebKit kept timed script execution in mind, Tony Gentilcore was able 
