PBS, Popcorn.js, POTUS

PBS is writing about our work with them this week on the President’s State of the Union Speech and Mozilla, so I thought I’d write something, too.

This week my team and I at Seneca’s CDOT were handed a great opportunity.  My colleague Brett Gaylor was in Washington, meeting with PBS and showing them some of the cool Mozilla projects within Web Made Movies.  The discussions went well.  So well that they asked if we’d like to do an experiment with Obama’s State of the Union speech later that night.  Um, yes we would!

PBS typically produces an annotated transcript, with commentary and resources linked to various paragraphs in the speech.  Luckily they have this data available already as json via a RESTful api.  The trouble with this is that it is disconnected from the performance of the speech itself.  What if you could use the video of the speech, and the timing therein, to drive the loading and display of this commentary, such that watching or seeking through the video gives you timely comments?  Wonder no longer: we did it.

We did it with HTML5 video and popcorn.js, a Mozilla Drumbeat project within Web Made Movies.  With popcorn.js we can write time line based web apps that are driven by video timing.  Where the web has traditionally been about documents, and film/video about linear narrative in time, popcorn.js bridges the two.

For this PBS demo I worked with Anna Sobiepanek and Scott Downe (researchers working with me at Seneca), and Rick Waldron (partner and developer at Bocoup in Boston).  Brett and Ben Moskowitz co-ordinated with PBS in Washington.  We also hacked some code to use the speech’s timing information from Universal Subtitles, who were translating the speech in parallel.  We worked fast, and within hours we had it working–a testament to the technologies in play here, and the skills of the people using them.

I give PBS a lot of credit for being open to working with Mozilla on this.  These are new technologies, new techniques, and the timing was crazy:

This may sound simple, but considering that we came into contact with the folks at the Mozilla Foundation on the morning of State of the Union, and we managed to get this far, this fast — despite every technical glitch imaginable and a big snowstorm — it is pretty remarkable. Our sincere thanks to all the developers who burned the midnight oil with us to make this possible!

I laughed while we were doing this work when someone pointed me to a reddit post about PBS’ coverage of the news and technology.  There is some truth in this, and it’s clear that PBS wants to use technologies like the open web to enhance the news vs. sell you something:

Part of the reason people join the PBS NewsHour for news events like the State of the Union is because we give you the primary source without too much clutter. There are no NFL-style dancing robots with whiz-bang sound effects. There are no real-time approval meters in multiple colors zig-zagging across the president’s face. We provide you the facts, the context and we let you make up your own mind.

It’s still early on for popcorn.js (we are releasing version 0.3 in a few weeks), but already we’re having some great successes with it, and there are more in the pipe.  I’d encourage you to join our growing community of developers and users in order to help us figure out what time-based video development means on the web.

I’d also like to mention just how proud I am of my students, who aren’t afraid to build things this important, or try experiments this crazy and time pressured.  It inspires me every day to work with them.