Tom MacWright

tom@macwright.com

windchime

windchime is a generative music system that adds another dimension to the act of composing written text, whether it be software or English.

To be honest, I intended to post this four months ago, and delayed because I never got the time to make a high level of fit and finish for things. I’ve recently pushed it to the level where it’s compilable and has a minimal feature-set but this is still a proof of concept.

It’s just a first try at something simple.

Writing code is making guesses and trying to please a picky machine. What we think of as coding aids, like testing, debugging tools, and such, pale in comparison to the things we’ve tuned out, like being able to see the code being written, and the magic of syntax coloring.

Screenshot of color forth

This work was inspired by color forth, a computer language in which color has meaning. It was created because Chuck Moore was losing his youthful eyesight, but the implications go much farther - that there can be meaning in the margins of what we consider to be data.

Color Forth is putting information in the white space of your source listing - Chuck Moore

An unmatched parenthesis is tension unresolved, and semicolons in C-like languages and periods in Latin languages are signals of phrasing.

Word painting music notation

It’s like word painting, a style of music in which the content dictates the presentation to inform at the edges. The words ‘lower, lower, lower’ would be sung, in successively lower notes, essentially using multiple channels of communication, like we use with facial expressions and speaking tone normally.

It also comes from the kind of feelings that Bret Victor has written about and what I heard from Brandon Martin-Anderson about whether his bike ever broken on his cross-country ride. He said yes, it did, but you could always tell what would go wrong far in advance. He’d hear a tire going flat or feel his chain getting old, buy one at a town, and fix it when it broke.

In comparison, computers feel like guessing, and making risky decisions with large consequences almost blindly.

This tries to be assistive technology in the mainstream. Windchime aims to only represent structure and feeling, to signal when things are right or wrong or stylistically different. And it’s non-lexical - audio is something that most people already work to, and which doesn’t interrupt trains of thought.

Technical Details and Audio

I give no thanks to the legions of Objective-C developers out there who don’t share noncritical parts of their code as open source. They’re just perpetuating an O’Reilly book-driven slow progress system based on closed-source hoarding.

windchime uses the same core for getting keystroke data as my minute project. It uses the same hook in the accessibility API, so it also requires access for assistive devices to be enabled:

Dialog of enabling assistive access

The only project that contained a full, working implementation of an OSX app sending midi to an external source was the accordion project, which came up in a random Google search. Without that code example, this project probably wouldn’t have happened.

Composition

I really think it is possible that our grandchildren will look at us in wonder and say: “You mean you used to listen to exactly the same thing over and over again? - Brian Eno

A direct mapping of keys to notes would result in a chromatic instrument that would probably be less than appetizing. Instead, it makes sense to follow the pattern of other modern instruments like propellerheads Figure and Brian Eno’s Bloom - assigning music to specific, safe subsets like the pentatonic scale.

Recently I’ve added many more options - 16 scales from this ~excellently useful post~. It’s interesting to hear which ones work well. As a hack musician myself, I expected the simpler options like a blues pentatonic scale to fare well, since a simple instrument like an appalachian dulcimer tends to use a subset. But in practice, more complex, harmonic and minor scales seem to sound more natural or correct.

The sound is sometimes quite helpful and pleasing and sometimes frankly annoying. Since it goes through speakers, there’s the problem of volume - ideally I would want it to be as loud as an old IBM keyboard clicking and no louder.

windchime is not a text editor - it passes keystrokes through, so you can use any editor or not a text editor at all.

Visualization of typing and sound

For Now

There’s an open repository for the code as it stands - contributions are welcome. I’ve posted an early build in binary form - it should work on OSX 10.7+. Let me know what you think on twitter or in the GitHub issue tracker.