Experiments with audio, conclusion

I’ve been working with an amazing group of web, audio, and Mozilla developers on a project to expose audio data to JavaScript from Firefox’s audio and video elements. Today those experiments are over.

In December a few of us working on processing.js had an idea–what if we could visualize sound data coming out of an <audio> or <video> element?  My colleagues were good at thinking in terms of “how can we make what we have now work?” but I had another idea.  “Let’s try and teach Firefox how to do this.”  In December I set myself a challenge:

This post marks the beginning of what I expect to be a somewhat regular series of posts in which I will document my thinking, learning, and progress related to some Firefox development.  I say “Experiments” above, because I am going to try a number of things.  First, I’m going to push into a part of the Firefox source code where I haven’t worked before, namely, the DOM implementation (note: it scares me, to be honest).  Second, I’m going to do so in an open and pedagogic way, attempting to cast aside my own ego and hesitation at looking foolish–I don’t actually know how to do what I’m going to try, and will learn and fail as I go–on the way to producing an authentic model of open development for my students.  Third, I’m going to work with a few others who are also interested in extending themselves and extending the web.  The only thing I can assure you of at this early date is that these posts will be an honest account of the attempt.

Yesterday, the end result of that work landed in mozilla-central, on its way to inclusion in Firefox 4.  I’m immensely proud of the work we’ve done, and thrilled that my peers in the Mozilla community have also accepted it.  I’m also very tired :)

There’s lots of things that I could talk about in terms of the code and API, and probably we’ll do some of that soon (you can already use the API in a Firefox nightly, read about the API, and try live demos).  But what I wanted to end this series of posts by saying something about how Firefox, the Mozilla community, and the open web, make what we did possible.

The other day my family had some friends over, and we got talking about what I do.  Of course they had heard of Firefox, and used it themselves.  “But can you explain the difference between this and the other one I use, Internet Explorer.”  One of the big differences, I explained, is participation, the community of involvement, and the accountability that comes with this.

When we started these experiments, we did so without needing permission.  I didn’t have to sign an NDA, go talk to and convince the right people, or get approvals.  I just grabbed the source code and started messing around.  And I did make a mess, at first.  I learned as I went, and we iterated on the API a lot (I have over 80 versions of it here, not to mention the various implementations of those).  We weren’t judged for doing it wrong, or for the pace or directions we took.  Instead, we heard of lot of “this is very cool!” and “have you thought about this?”.  We were able to take one of the world’s premier applications (Firefox) and rework it.  It’s hard to overemphasize how significant this is.  We couldn’t do what we did in very many other contexts.

I said above that participation is paired with accountability, and this is also very important.  In the early months we built something that worked, but not what we have today.  As much as Mozilla made it possible for me to experiment, they also made sure that what got accepted was of the highest quality.  I haven’t blogged about audio much over the past three months, mostly because we’ve been too busy getting the patch fixed up based on reviews.  Before it could land we had to think about testing, security, JS performance, DOM manipulations, memory allocation, etc.  To get this landed we needed lots of advice from various people, who have been generous with their time and knowledge

What’s different about Mozilla?  I think of something Joe Hewitt wrote on twitter back in the spring that struck me:

Bottom line: we can currently only move as fast as employees of browser makers can go, and our imagination is limited by theirs. @joehewitt

Not one of the people who did this work is an employee of the Mozilla corporation.  When we decided to get serious about trying to include this in Firefox 4, one of the people working with us filed a bug, and the response was, “I don’t think we have the cycles to get this done in time.”  “That’s OK, we’ll do it then,” was the reply.  And we did.

The web is too big and too important to only go as fast and as far as a small group of employees can take it.  Mozilla gets this, and values community involvement like ours.  What we did is not unique–there are other great features and bug fixes coming in Firefox 4 that were done by community members vs. employees.  In fact the distinction between the two is often hard to see when you’re working on this stuff–we all work together.

Having said that, let me publicly congratulate my amazing audio peers, without whom this work wouldn’t have happened: Al MacDonald, Corban Brook, Yury Delendik, Charles Cliffe, Ricard Marxer, and our cheerleader and supporter, Chris Blizzard.  Also thanks to the dozen others who wrote demos with our stuff in the early days.  Demos are how you win (Chris Blizzard taught me that).

I’ll end with a video someone shot of my keynote talk at the recent Mozilla Summit.  I was doing a quick demonstration of what is possible with this API.  I look forward to seeing what the rest of the web will do with it.

Posted in CDOT, Experiments with audio, Mozilla, Mozilla Education, Seneca, Teaching Open Source | 8 Comments

On working with instead of on

I read an account of an event I attend recently that gave me pause.  In it, the author argued that he had observed a gathering where technology formed the only basis for the interactions in the sessions, and that those in attendance were all of a kind, without diversity, and, on the whole, not interested in people, society, history, etc.  This assessment was leveled against a community in which I find myself deeply situated, and as such I wanted to reflect on my own position within such a crowd.

We are told that open source is about “scratching an itch,” creating software to solve personal problems, and the culture the surrounds such an enterprise.  This is true, in my experience; however, it is not my experience.  Where open source is most often taken to mean source code, I have been primarily interested in what is meant by open, whether that be open education, open source, or open data.  For me, and I speak intentionally as an individual among a much larger crowd, open is about people more than technology.

Where some people choose software projects in order to solve problems, I have taken to choosing projects that allow me to work with various people.  I have given up the comfort of being an expert , and replaced it with a desire to be alongside my friends, or those with whom I would like to be friends, no matter where I find them.  My history among this crowd begins with friendships, many of which continue to this day.  I love almost none of the technology I use, and view it as a means to an end: working on things, for me, is about working with people.

This way of working, where collegiality subsumes technology or tools, is central to my personal and professional work.  Even looking back over the past two years, most of the work I’ve done is influenced by a deep desire to work with rather than on.  I need almost none of the software I build, I have no itch to scratch.  In order to be effective, working with requires that I be attentive to the other more than to the thing we do together.  Working with demands that I recast myself in the shape of the other, and vice versa, that I be willing to change, that I love and allow myself to be loved.  I cannot succeed if my gaze is only on the thing, especially if this thing is allowed to come between us.

In the context of working with, technology once again becomes the craft I both teach and am taught, it is what we share with one another, the occasion for our time together, the introduction, but not the reason, for our friendship.

And it can be difficult to observe what I am describing, for it will always show up as something much more mundane to those not working this way.  To watch us work, it will seem that we only build things, that we talk only of certain things.  It is possible for someone to work on something together with others who are working with each other.  But it is not the same thing.  It is not what we do.  Looking only at the thing we build together, it is easy to miss what happens around it, and who is there.

Posted in Idea Factory | Leave a comment

Fruit

Our weekend was punctuated with experiences related to fruit.  It began with the most enjoyable visit to our local farmer’s market I’ve had, since we were all four able to ride the 10Km on bicycle.  Tricycles, training wheels, and now rails to trails–the girls are at an amazing stage.

When we got to the market we got chatting with a local backer who makes European pastries, breads, and preserves.  She had the girls sample more than a dozen fruit preserves, all of them more unique and interesting than anything I’ve seen at the store–Banana Ginger, Peach Mint, Apricot Chile, etc.  We ended-up with a few bottles and an overflowing bag of Dutch almond twists (“No, you take more than that!).

On Sunday, after a slow and pleasant rain, I ventured out with the girls to go and see what we could find in the woods.  “Dad, the rain has cleaned everything!”  This season has been one of the best I can remember in terms of bugs and being able to move in the woods during the summer.  We crossed the creek at the back of the woods and noticed Raspberries.  A little further up we found the first ripe Elderberries of the season (very early).  After picking all we could in our baskets we headed home, but not before coming upon a beautiful apple tree with large red apples.  We got a basket of those too.

My grandma asked me on the phone today if we are enjoying our woods, and I could honestly say that we are.  I told her about the mother deer and its fawn that we stumbled upon as we walked home, and about the fruit we’d picked.  One has to go into the woods in order to receive what it has to offer; but once there, the gifts are abundant.

Posted in Food, Home School, Nature, family | Leave a comment

Update on DXR work

As I wrote previously, I’ve switched to DXR work for the summer, and have been making some good progress over the past month.  I wanted to show you some of the things I’ve done, and where I’m going in the coming weeks.

Much of my work thus far has been a backend rewrite.  This was long overdue and somewhat painful, but has enabled me to start fixing bugs and adding new features really easily.  We’ve setup a DXR instance at Mozilla, which you can find at http://dxr.mozilla.org.  I’ve also put my Dehydra/JSHydra debugging tool there, too (see http://dxr.mozilla.org/viewsource).  Note: not all of the features I’ll be discussing below are installed on dxr.mozilla.org yet, but they will be soon.

I’ve been fixing things in the UI, and starting to layer new kinds of data into the index.  The first was to add compiler warnings.  You can now search the warnings using keywords and optionally limiting to certain paths:

Search for Warnings

The more common use of this warning info is encountering it within the source while looking at a piece of code.  Warnings appear with an ‘!’ icon beside them, and hovering over the line shows the warning text.  Here’s an example:

Warnings in Source

Another feature I’ve had numerous requests for, and is now implemented, is source code navigation aids. When you are in a large file, and looking at line 2456, it would be nice to know that this line is part of Foo::Bar(), and the signature of Bar(). Here’s what it looks like:

Navigation Sync

Notice that AddRequest is highlighted in the sidebar, and a light-blue signature bar has appeared at the top of the source. These happen as you move your mouse, depending on the line, and disappear on their own.

I’m now moving on to improve/replace the current source info popup. I haven’t decided whether to use an improved popup, add another panel to the left sidebar, or do something with inline expanding divs. The way Flickr’s new popup works is one way I could go:

Improved popup idea for DXR

The inline expanding divs might work like this:

I’m going to spend another week or two on UI fixes before moving on to data analysis fixes and index improvements (there are some great build system changes coming soon that are going to make this easier–hello xpidl.py, I’m looking at you!).  If you have UI input, please give it to me soon.  Meanwhile, the fearless Joshua Cranmer, my lone and valued contributor, is working on making DXR generalized for projects other than Mozilla.  I hope to integrate some of his build/index changes coming up.

More to come soon!

Posted in CDOT, DXR, Mozilla, Mozilla Education, Seneca | 4 Comments

“What we say no to”

Paul Graham has an interesting essay up about the Acceleration of Addictiveness, in which he argues that our ability to refine and improve things through technology is leaving us increasingly susceptible to addictions, many of which develop too quickly for cultural norms to temper:

As far as I know there’s no word for something we like too much. The closest is the colloquial sense of “addictive.” That usage has become increasingly common during my lifetime. And it’s clear why: there are an increasing number of things we need it for. At the extreme end of the spectrum are crack and meth. Food has been transformed by a combination of factory farming and innovations in food processing into something with way more immediate bang for the buck, and you can see the results in any town in America. Checkers and solitaire have been replaced by World of Warcraft and FarmVille. TV has become much more engaging, and even so it can’t compete with Facebook.

While I don’t agree that there’s no word for something we like too much, I think Graham comes to a more interesting place as he ends:

But if I’m right about the acceleration of addictiveness, then this kind of lonely squirming to avoid it will increasingly be the fate of anyone who wants to get things done. We’ll increasingly be defined by what we say no to.

Here he says something really important, making a point one doesn’t encounter enough on the web, nor in the street.  I’m talking about the seemingly irreconcilable link between freedom and personal renunciation.  Illich discusses it length:

Christians who imitate him [Christ] soon discover that little practices of renunciation, of what I won’t do, even through it’s legitimate, are a necessary habit I have to form in order to practice freedom

…it reminds me of the things which, in the modern world, we can give up — not because we want a more beautiful life, but because we want to become aware of how much we are attached to the world as it is and how much we can get along without it. These unnecessary things have now multiplied to such an extent that you can’t easily give a social shape to them. Some people will give up writing letters on a computer — not because it’s bad, and not because they don’t like to have to answer letters at the speed of e-mail. Others will give up the services of physicians or, as somebody who I know has done, guaranteeing that each of his children will get a college degree.

The certainty that you can do without is one of the most efficacious ways of convincing yourself, no matter where you stand on the intellectual or emotional ladder, that you are free. Self-imposed limits provide a basis and a preparation for the discussion of what we can can renounce as a group of friends or a neighborhood. I have seen it, and I can witness to it. For many people who suffer from great fears and a sense of impotence and depersonalization, renunciation provides a very simple way back to a self which stands above the constraints of the world. (Illich, “The Rivers North of the Future,” 101-2)

Graham writing about his refusal to own an iPhone, and further, his recognition of this act as a personal, and therefore inessential, move, is significant.  We don’t lack for models of consumption, excess, and self-fulfillment online; what we need are more acts like this that are disengaged from the web yet which nevertheless exert an influence through it.  Our need to learn to say no is not simply about addiction; it is more importantly about the possibility of the self.

Posted in Idea Factory, Reading | Comments closed

On the possibility of expressing love online

Today a colleague of mine used an abbreviation online that I needed to look up.  I never was able to locate it, but while I searched, I found this one:

ly – love you

I wondered aloud on twitter that I’d never encountered this, and this thought caused me to pause to ponder whether expressions of love online are possible.  “I’ve never seen that,” is what I heard from most people on twitter, the rest responding with humour.

Online communication, and in particular the kind practiced on irc, is, in my experience, funny, witty, clever, engaging, technical, often childish, and sometimes deeply meaningful; however, I can count on one hand the number of times it has offered a moment in which to express love between friends.  I don’t mean sexual love, but rather a deep friendship.

Perhaps it’s that most relationships one engages in online are not truly deep; perhaps it’s that the public nature of the communication precludes certain kinds of more personal engagement; perhaps it’s not really possible to express oneself this way online.  Whatever is happening, it’s interesting to think about online communication tools, what they make available and what they remove from the functioning of friendships online.

Posted in Idea Factory | Comments closed

Restarting DXR Development

It’s summer and with the change in weather, I’m going to take a holiday from much of my work on web media and audio/visual code for Mozilla.  I’ll still be working on the audio data api patch, but the rest of my work I’m going to put on ice for a few months so I can return to an old love, DXR.

If you aren’t familiar with DXR, it’s a web-based source code navigation tool that uses semantic data from many sources (dehydra, treehydra, xpidl, build system, C preprocessor, etc.) to help developers understand what they are looking at in large code bases.  Imagine being able to click on a variable named ‘i’ and getting back information on that particular i vs. the 10,000 other ones, or seeing all callers of SomeClass.ToString() vs. SomeOtherClass.ToString() and you get the idea.  I’ve written about DXR before if you want to learn more.

One of the first things I’m going to do is get DXR running on a server at Mozilla.  For the past year I’ve had a desktop machine running a DXR instance so people could test it.  I’ll be working on this Mozilla DXR and updating it frequently as I fix bugs and add new features.  When it’s running I’ll blog again with a URL.

I’ve been talking with people a lot about the things they want to see in DXR, and I’ve put together a list of things that need to be done.  If you’d like to add to this list, feel free.  You can also talk to me about things on irc in the #static channel.

If you look at that list you’ll see that there is way more than I can possibly do in the next two months on my own.  So I’m putting out the call for other developers to join me.  DXR is a strange mix of static analysis, server side programming, tool hacking, and client side work.  Don’t be put off by the fact that this uses static-analysis if that’s not something you know (I only barely know it myself!).  This is just a lot of scripts and hacks to various tools, and there’s room here for you to help improve it.

Posted in CDOT, DXR, Mozilla, Mozilla Education, Seneca | Comments closed

On the Reading of Books

Much of my time writing on this blog is spent advocating for the open web, and the potential it brings for distributed, global, collaboration.  I don’t believe in this any less, despite the critique that follows.

At the same time that I am passionately involved in the creation of the open web, I am also intimately tied to book culture.  Unlike some of my fellow open web advocates, I am not against the book as a result of being for the web.  While I’m usually not one to make predictions, I feel confident in saying that the book is not going to disappear.  There are many reasons for this, but let me pick-up on one for now.

Reading on the web is mainly interested in information, in results, data, etc.  It is about  manifest- much more than latent content.  It is driven by the immediate reaction, by Liking, by linking, by query results.  Reading on the web is influenced by time: the time it takes to find it among so many other results, the time it takes to load, the time it takes to read, the likelihood that this text will be here tomorrow.

Reading on the web is also very often (always?) a first reading.  “Have you seen this?”  “Yes, I saw it.”  It is what I read, not what I am reading.

The reading of book reading is different from the reading of the web.  This reading is much slower.  It takes time.  It participates in history.  Recently my wife and I went to a nearby inn for two days, in order rest and spend time together.  We each took along books to read, and when she saw me reading my book she asked, “I thought you were finished that?”  She was right, I’ve read that book many times, and will likely read it many more.

The reading of book reading is rereading.  The book provides a place of return in a way that isn’t impossible on the web, but in practice never happens.  What does the replacement of progression with return do?  I believe that one of the most important things it does is to open a place for thinking.

The reading of rereading is how we get on the path to thinking.  The way to thinking is a path that is easily lost.  Tracking such a path, one is forced to return to the last known sign and walk in a slow circle around it, fanning out ever so slowly, until it can be picked up yet again.  Without return there can be no progress.  Without going back there is no going forward.  Books can provide such signs, and allow us to once again stumble on the path toward thinking.  This is important, since the path to thinking is so long.  One needs reliable signs from the past, places of return, places which are left alone rather than being trampled by progress.

Posted in Reading | Comments closed

Two Experiences with the Local

1) I’ve been amazed this year at the amount of Canola planted in the fields in our county. I’m used to seeing a lot of soy beans, corn, and winter wheat. But the shock of Canola visually, a bright, beautiful, electric yellow, never ceases to give me pause. And this year it’s absolutely everywhere.

I was interested in this, and wondered if it was new to this area. I did some reading and was surprised to learn that Canola isn’t an import to this region, but actually a Canadian cash crop. Canola, or the unfortunately named rapeseed, is grown in order to produce an oil edible by humans, one with a low erucic acid content. It was originally cultivated in the 1970s in Canada, and by 1978 the the name was changed to “Canadian Oil Low Acid,” or Canola.

2) On Father’s day I took our girls out to the field in front of our house.  I wanted to try and show them something I’d seen the night before while mowing the grass.  We snaked our way around various conifers, bent low, looking.  Eventually I spotted it, sitting totally still, almost perfectly hidden at the base of a spruce.  I motioned to the girls to go around to the other side of the tree, so we stood a better chance of catching it.  I reached my hand down and was able to lightly pet its perfect fur, but when I tried to put my hand beneath it, it took off.  And so began a 15 minute chase around trees and through long grass.  Eventually my wife joined us and she and my eldest daughter were able to catch it.

Hares, unlike rabbits, are not born naked and helpless underground.  Instead, they are born with fur, and able to run.  This baby literally fit in the palm of my hand, with ears and feet no longer than the tip of my index finger.  It’s fur was more like bird down than course fur.  The four of us were mesmerized by this wonderful creature, which finally settled down in my daughters’ loving arms.

After we let it go (if it hadn’t been Father’s day, I think I would have been overruled and we’d have had another pet), I pulled out some guide books to figure out what we’d caught.  It was clearly a hare, but I wanted to know more about what kind it was.  The book led me believe that we’d caught a European Hare, a species that was introduced to Ontario in the early 1900s, and most interestingly to me, in my home town of Brantford.

Posted in Nature, family | Comments closed

Great Horned Owl

One of the things I look for is the Great Horned Owl. When I was a small child I saw my first. It had been killed (by a car as I recall) and my uncle, a taxidermist, was going to stuff it. He brought my brothers and I to see it, and the three of us held it, wings outstretched. The size of this bird was incredible to me, as were its talons.

From the point on, I have always kept one eye open for these great owls. I have never seen one alive in the wild, until this past week. Our woods is sometimes home to a Great Horned Owl. I have spent many evenings sitting on our screened porch listening to it in the dark. The whole woods goes quiet to listen, and the call travels for miles, making it hard to pinpoint exactly where it is. However, you know it’s a Great Horned Owl when you hear it.

Once, while standing on our deck and observing a full moon, I saw it fly silently across the sky. Only its silhouette was visible, but even that was magnificent. Recently, my wife drove into our lane one evening and her headlights caught a large animal on the ground. “It looked like a groundhog, but then it stood up and flew away.”

Last Thursday we took our girls out for a walk before bed, following the edge of the woods, pausing as we came to the mouth of a creek. As we approached the creek’s edge, I saw a bird descend from a tree to my left. At first I mistook it for a heron, but its flight pattern was wrong. As I turned my head I could see that its body was too short and round, and its colouring too dark for a heron. “That’s an owl!” I cried, as it flew into the woods. I ran after it. I couldn’t see where it had gone, but hoped that we had spooked it and it would land a short distance away. After running 500 feet, I paused, trying to decided which way to go. Ahead and to my right I could hear a great commotion: a Red Wing Blackbird or Eastern Kingbird was making a great raucous, and clearly upset that something had come within the vicinity of its nest. I took off toward the sound.

Deeper into the woods I came to a tree with a limb extending parallel to the ground, low, and bare of leaves. There in the middle, with its back to me, and a furious Red Wing Blackbird encircling it, was the owl. I slowed my pace and walked as quietly as I could. Approaching from behind, and with the little bird masking my sound, I managed to get underneath, and then beyond the owl. As I looked up at it, it swivelled its great head around and down toward me. It’s two yellow eyes pinned me to the ground. We both sat motionless, looking at each other for 30 seconds or more.

Eventually I decided to call to it, and did my best owl call. Upon hearing me, it cocked its head to one side and leaned in toward me. Figuring I had done something right, I tried again. This time it took off deep into the woods. I didn’t go after it, but stood there pondering what I had just seen.

I have spent the past 4 years looking for that particular owl. An owl, like so much in nature, is not something you can see because you want to, or have because you desire it. It does not exist for your pleasure or consumption. It is not on display like the work of art, or available like the manufactured good. It is, quite simply, a gift to have this experience. Tonight I went and walked the same path, and saw nothing. I might never see it again, but I will continue to be open to receiving this gift, which is the proper mode of being in the world.

Posted in Nature, family | Comments closed