For Your Reading Pleasure: EasyCaptions

Introducing EasyCaptions: A simple system for adding captions and an interactive transcript to online videos. EasyCaptions uses progressive enhancement to provide the best possible experience for all visitors, regardless of their browser’s JavaScript, HTML5 or Flash support.
Demonstration

Background

I don’t produce much video these days, but as a web surfer I often encounter other people’s videos, and was recently impressed by two video implementations: a TED Talks video page, and an HTML5 video demo produced by Bruce Lawson.

The TED Talks page had a great feature I’d never really seen anywhere else: an interactive transcript of the video that you can read and click. For example, you can click the third sentence of the transcript and the video will jump to that point.

Bruce Lawson’s demo illustrated a dead-simple way to add captions to an HTML5 video using just a wee bit of HTML markup and JavaScript.

Both of these pages shared a unique attribute: they stored complete transcripts of the videos in the markup of the HTML page itself, NOT in an external XML file, as is most commonly seen. What’s the big deal about inline transcripts, you ask? Well, today’s most common captioning options require placing your caption text in an external XML file that loads via ActionScript or JavaScript/AJAX. Both the TED page and Bruce’s demo eschew that approach and place the full transcript in the HTML. This means the transcript is always available to the visitor, regardless of that browser’s support for HTML5, JavaScript or Flash Player.

In both cases, the transcript had been marked up as paragraphs, with extra markup denoting phrases that align with the video’s timecode. The TED site used a tags to mark each phrase, with inline JavaScript triggering the playback of the clicked link. This is functional when JavaScript is enabled, but fails when JavaScript is disabled, leaving you with a ton of bloated markup that doesn’t work:


<p>
<a href="#" class="transcriptLink" onclick="seekVideo(0); return false;">One way to change our genes is to make new ones,</a> 
<a href="#" class="transcriptLink" onclick="seekVideo(2000); return false;">as Craig Venter has so elegantly shown.</a>
</p>

Bruce’s span tags, on the other hand, are semantically sound — a span is a neutral, unobtrusive inline element meant to be a child of a block element such as p. Perfect. But wait, there’s more! Since this was an HTML5 demo, Bruce took advantage of the new data attribute that can be used on just about any HTML element. (The short version is: you can create any custom attribute you want, so long as the name begins with data-.) Bruce decided to create two attributes for each span: data-begin, which indicates (in seconds) when the phrase starts in the video, and data-end, which indicates when that phrase has ended in the video. Simple as can be, and extremely efficient:


<p>
<span data-begin=1 data-end=6>Hi, my name's Dr Archimedes Einstein 
and I'm a Dctor of Science at the University of Science</span>
</p>

Enter: EasyCaptions

I was incredibly inspired by these two pages and decided to combine their features, creating a new system I call EasyCaptions. The goal of EasyCaptions is to make it as simple as possible to add useful captions to your online videos. I want to eliminate the headaches of captioning as well as the excuses (too hard, confusing, etc.).

What EasyCaptions does:

  • Dynamically generates a div under your video that will display caption text (the div‘s style and positioning is completely configurable via CSS).
  • Dynamically makes each span in your transcript clickable, jumping to that point in the video. This is done via progressive enhancement and event delegation, leaving your markup clean and avoiding heavy-handed use of onclick.
  • Works out-of-the-box with HTML5.
  • Includes HTML5 video support detection, enabling you to use a Flash-based fallback if you desire.
  • Provides a hook for Flash-based fallback systems, enabling the captions and transcript to behave identically to the HTML5 version.

What you’ll need to do:

  • Type up the transcript of your video, using <span> tags to wrap each phrase, just like Bruce’s example above.
  • Place the transcript in a container element with a unique ID (such as div id="transcript").
  • Add a teeny bit of JavaScript (about 4 lines).

The final product. Successfully tested in Firefox, Safari, Opera, and Internet Explorer (IE and older versions of the other browsers all use a Flash Fallback if provided).

Peruse the test suite to get an idea of how flexible the system is.

The documentation and downloads are located on the EasyCaptions page.

HTML5 Video, minus Ogg

As you’ve probably read somewhere on the interwebs, HTML5 is bringing native video support to browsers. This will enable us to embed a video file directly in our HTML, much like a SWF or image.

Background

You may have also heard that there’s currently a big controversy over what kind of video files will be supported. The defacto standard is MP4/H.264, which is used by Adobe in their Flash video format, and by huge media sites like YouTube. Mozilla, the makers of Firefox, refuse to support the MP4/H.264 standard because it isn’t open-source and free from licensing constraints.

Turns out H.264 is not public domain. Although the company that owns the H.264 patent has temporarily agreed to waive royalty fees for the next decade or so, they reserve the right to charge fees later on. Mozilla says no way, we will only support a video format that is free from licensing issues and has no patent holders (because patent holders can decide to sue some day). Mozilla supports the completely free/open-source Ogg format.

Apple and Adobe, already knee-deep in MP4/H.264 with their Quicktime and Flash video products, vow to press on with H.264.  Google also supports H.264 because YouTube relies on it, and because Google’s new Chrome browser is based on the WebKit project, which has Apple as a main code contributor. In case you haven’t noticed, Apple, Adobe and Google have pretty much cornered the internet video market the past few years, so if they’re throwing their support behind H.264, you can count on it being around for a while. Not to mention that many mobile devices, including the iPhone and most Android phones, have hardware that is designed specifically to support H.264 video, enabling smoother playback and less battery drain than non-dedicated hardware.

(For what it’s worth, Opera is in agreement with Mozilla and supports Ogg. However, not many people seem to pay attention to Opera these days, so they don’t appear to have much influence in this race. Microsoft has endorsed H.264 with it’s upcoming IE9 browser, but it won’t be available for some time.)

The problem

Firefox and Opera are essentially forcing websites to offer two versions of each video: an Ogg version and an MP4 version. In my opinion — and the opinion of many others — this simply will not do. Providing two different video files is not realistic, Ogg’s quality is inferior to H.264, and many computers and mobile devices have direct hardware support for H.264 but not Ogg. In reality, without MP4 support, HTML5 video is rendered useless for most site developers in Firefox and Opera.

The most logical workaround is to code <video> elements to work for MP4 and have a Flash Player-based fallback for older browsers and browsers that only support Ogg. Since the <video> element is designed to allow for fallback content, just like the <object> element, we can do this:


<video id="myvideo" width="480" height="360" controls>
    <source src="/video/file.m4v" type="video/mp4"></source>
    <object data="flash-video-player.swf" type="application/x-shockwave-flash" width="480" height="360">  
        <param value="flash-video-player.swf" name="movie"/>  
        <param value="file=/video/file.m4v" name="flashvars"/>
    </object>
</video>

This works fine in Safari, Chrome, Internet Explorer and older versions of Firefox and Opera that don’t support the <video> element. However, Firefox 3.6 and Opera 10.5 do something very irritating: even though they KNOW their <video> doesn’t support “video/MP4”, they load the <video> element anyway. Which is … like … OMG so duh … because the video can’t possibly play!

If the <video> element is loaded, Firefox and Opera will never load the fallback <object> containing the Flash-based fallback.

Because this behavior cannot be fixed with markup, we’re forced find a scripted workaround (notice that we haven’t used a single bit of JavaScript yet). Thankfully, there’s a pretty straightforward solution: Delete the <video> element in Firefox and Opera.

A Solution

Here’s a simple script that will detect whether HTML 5 video is supported, and if it is, will check to see if MP4 is supported. If HTML5 video is supported but MP4 is NOT supported, the script deletes the specified <video> element but leaves the Flash fallback in its place.


var detectVideoSupport = function (){
    var detect = document.createElement('video') || false;
    this.html5 = detect && typeof detect.canPlayType !== "undefined";
    this.mp4 = this.html5 && (detect.canPlayType("video/mp4") === "maybe" || detect.canPlayType("video/mp4") === "probably");
    this.ogg = this.html5 && (detect.canPlayType("video/ogg") === "maybe" || detect.canPlayType("video/ogg") === "probably");
    return this;
};

var replaceVideoWithObject = function (video_id){    
    if(!video_id){ return false; }
    var video = document.getElementById(video_id);
    if(video){
        var obj = video.getElementsByTagName("object")[0];
        if(obj){
            var obj_copy = obj.cloneNode(true);
            video.parentNode.insertBefore(obj_copy, video);
            video.parentNode.removeChild(video);
        }
    }
};

window.onload = function (){
    var video = detectVideoSupport();
    //Both Opera and Firefox support OGG but lack MP4 support
    if(video.ogg && !video.mp4){
        replaceVideoWithObject("myvideo");
    }
};
</script>

Functioning demo.

A few notes

Tested successfully in:

  • Windows XP: Firefox 3.0, Firefox 3.5.8, Internet Explorer 6, Internet Explorer 8, Google Chrome 4.1.2
  • Windows Vista: Internet Explorer 7
  • Mac OS X (10.6): Firefox 3.6, Safari 4.01, Chrome 5 (beta), Opera 10.1, Opera 10.5b

(Note: IE6, IE7 & IE8 give an unexplained “object required” error in the demo page, but everything works fine. I will investigate as time permits.)

This demo uses the JW Media Player as the Flash fallback video player, but you can use any Flash-based video player on your page.

This demo doesn’t do any Flash Player version detection to ensure the visitor has a supported version of Flash Player. If you need to add Flash Player version detection, you can use SWFObject to embed your Flash file.

Update April 3, 2010: This post was updated to add Opera 10.5 to the list of non-behaving browsers and remove Firefox user agent sniffing.

Changes to pipwerks.com, part 2

In case you hadn’t heard, pipwerks.com was hacked last week. The entire database was erased. Bastages. Luckily, I had a recent backup.

While going through the pains of a new WordPress install (with new plugins, extra security, and imported posts/comments), I decided “why not throw a new layout in the mix, too?” I mean, if I’m going to make changes, I may as well do them all in one shot, eh?

I had been working on an HTML 5-based layout for some time, but had been reluctant to publish it because of HTML 5’s newness and uncertainty; where do I use ‘section’ versus ‘article’, etc.? Do I really want to let WordPress continue to abuse <ol> and <ul>, or does it make more sense to use <dl> in some cases?

The outcome of the argument is what you see today. I didn’t want to fight WordPress too much (esp. the plugins), so there are still a bunch of legacy <div>, <ul> and <ol> elements hanging around. I’m sure I’ve neglected a number of things, but for the most part I’m satisfied.

Here is a short list of fun stuff i implemented in this theme:

  • <dl> for comments (<dt> for author, <dd> for comment body). I feel this is more appropriate as it maintains a relationship between the comment author and the comment body. It also allows for interesting styling (though it makes threaded comments more difficult).
  • <nav> for page header
  • <section> to demarcate where the sidebars go, “related posts” lists, etc.
  • <article> to contain the post itself
  • <footer> to hold footer-ish stuff (though the definition of footer in HTML 5 is still a work in progress)
  • Widgets for sidebars and the footer, which make editing the template SO much easier!
  • Myriad Pro as the body font. Yes, many people won’t have it, and they’ll see Arial instead. But many of my readers probably have Adobe products, which include Myriad. So there you go. No @font-face required! 😛
  • Malarkey‘s Universal IE 6 CSS for the laggards who still use IE 6. No offense — we still have IE 6 at the office, too — but I simply am not interested in bending over backwards to support IE 6 anymore. Kthxbai, IE 6!
  • Alpha transparency in images to allow overlays (such as the cartoon head). Easy in every browser except IE 6 (see above)
  • CSS-based rounded corners in browsers that support them (Safari and Firefox at the moment)
  • The comments styling uses a CSS trick to create the triangle that gives the impression of a cartoon ‘talk’ bubble
  • Icons for external links via CSS3
  • Meyer’s CSS Reset
  • Gravatars
  • Sociable links
  • Related Posts displayed for each post
  • …and more!

I must say I really enjoyed working on this theme, though it took me way too long to get to this point. Please have a look around and let me know how you like the site (or, heaven forbid, you find any problems/errors). And of course, a big thank you to everyone who contributed to the CSS and plugins that I’m using! Your hard work makes my life easier and I appreciate it.

HTML 5: The strong element

I just saw something interesting I thought I’d pass along. In the new HTML 5 proposal, the strong element is being modified to represent “importance rather than strong emphasis.”

The WHATWG gives the following example:


<strong>Warning.</strong> This dungeon is dangerous. 
<strong>Avoid the ducks.</strong> Take any gold you find. 
<strong><strong>Do not take any of the diamonds</strong>, 
they are explosive and <strong>will destroy anything within 
ten meters.</strong></strong> You have been warned.

The b element is supposed to represent “a span of text to be stylistically offset from the normal prose without conveying any extra importance, such as key words in a document abstract, product names in a review, or other spans of text whose typical typographic presentation is boldened.”

The WHATWG gives the following example:

The <b>frobonitor</b> and <b>barbinator</b> components are fried.

To me, this is both exciting and confusing. Knowing we can nest the strong element opens up some new styling possibilities while keeping the markup valid and semantic. But using both the b and the strong elements, with different conditions for their usage, will ultimately be confusing for most people, including me.

Wonder how WYSIWYG editors will handle this.