DCTN

The little local project I’ve been working on is DCTN. It’s a list of upcoming live music in Washington, DC.

Personal projects live on multiple axes: some flirt with global relevance, others personal connection. DCTN comes after projects that embraced the idea of developers for developers: mistakes.io, for instance, is only relevant to the JavaScript literate, and chartpipe to the information visualization set. Creating these things is easier in a way, because I’ve never had to research the people who would be interested in them. Imagining what other people think and feel is difficult. And it’s easy to anticipate people who are a lot like yourself.

DCTN lives a little outside that dynamic. I’ve been in the “DC Music Scene” for years, as part of Teen Mom and other bands. But I don’t go to shows all the time, or follow news outlets like Brightest Young Things. And the potential is limited: after all, DC is just one city, and the $8-12 indie band show market is barely staying afloat.

Local and Time

Incredibly local things are fascinating: in college, I built the Swem Signal, a way to share your location in the dungeon-like Swem library. It’s a crude PHP hack, built before Foursquare or Twitter were popular. But it still works, and students still use it. I’d like to think that it’s making nerds like me a little happier because they can tell their friends where to find them and to talk a little, before getting shushed.

Which connects to the other thing: time. Having worked on a lot of technology projects and observed the tech industry for years, it’s shocking how the usage curves vary - projects can maintain a flat curve of dedicated users for decades, and others can skyrocket to hundreds of thousands and then vanish into obscurity or shutdown. Different sorts of enjoyment, in both production and consumption.

React

DCTN is a React application. The Facebook-developed framework is pleasant to work with: its approach to building pages is not too many steps away from building in d3, but feels like it minimizes boilerplate. Each view can be written as a separate file and stitched together with browserify.

I have my gripes with React: building a touch application requires some sketchy hacking of the React source code to include a pre-release plugin. This will surely sort itself out, but wasn’t very obvious in documentation or code. React’s JSX is surprisingly nice - I’ll trade templates for source transforms any day of the week, especially when the JSX transform has decent error reporting. Unfortunately, JSHint doesn’t play nice with JSX, so I need to use a pre-release version and lots of jshint:ignore tags to avoid error noise in vim.

There’s plenty more to say about React, but I’ll leave that for a separate article.

The Scraper

DCTN has a server component. The core data - listings of concert times and details - comes directly from venue websites by the concerted work of scrapers. Each scraper is written in node.js and cheerio, and is a module with unit tests.

The server component is not client facing: it pushes data on a schedule to S3, which is then consumed by a static website. DCTN.org is hosted on Github Pages and requests data from an Amazon S3 endpoint. That data gets there by a server process, but never does a client request require anything new from the server.

The scraper framework, tonight-sources, is written as a node.js web app, running both an http server for status checks and a cron-like process that does the scraping. It also contains some simple metadata for each venue as GeoJSON.

tonight-sources currently scrapes:

• DC9 Nightclub
• The Black Cat
• Rock & Roll Hotel
• Velvet Lounge
• Fort Reno

There are plenty of other ones in the works and various stages of completion. Scrapers for U Hall & 9:30 Club are complete, but the venues apparently block my server.

Node.js for Scrapers

Despite the cheerio module module, this isn’t node.js’s brightest area:

• Handling failures at the request, tokenize, parse, and output stages - all of which happen often - is cumbersome
• Async i/o isn’t necessary or beneficial in this case, but sync request libraries are poorly maintained and tested

Go, with goquery, is extremely promising in this regard, and I’ve started to see nice scrapers being written in it.

Dokku

This is the first app I’ve built with Dokku. Dokku is a Heroku clone written in bash, on top of Docker. To review terms:

• Heroku is a ‘platform as a service app’. Unlike traditional web servers, you don’t have to install node.js or PHP or such: you just push code to it, just like you’d push to Github, Heroku detects what that code requires, installs it, and runs it for you.
• Docker is a lightweight virtual machine system. It’s a way of packaging small images of computers as portable files that you can run anywhere: you can make a Docker image that has Linux and PHP and MySQL, and run it as a server, just like you’d use VirtualBox to run Windows on your Mac. Unlike VirtualBox, Docker is simple, small, and efficient.
• Dokku is a Heroku clone written with Docker. It connects magic detection & installation code with an interface that receives code, and then connects to Docker to run applications. It’s really neat.

Dokku means that when DCTN has more components, they can be packaged and run separately, as separate services. This is how I run tonight-tweets, which is the twitter bot that runs @dctnght. The twitter client is a separate node app that runs in a separate virtual server on the same machine. So, if one crashes, the other doesn’t, and I can update them individually. Since the datastore is separate - it’s S3 - they can be totally decoupled.

DCTN Today

DCTN is an ongoing project: it’s something that I’m building for myself, for friends, and for people in DC. Right now, it has a tiny handful of users, orders of magnitude less than ‘popular’ projects or established blogs. I Hope it’ll grow. Unlike other DC music outlets, it’s entirely automated, so day-to-day accuracy doesn’t rely on my assistance.

I’d love help - this project lives in my limited free-time and has lots of unexplored potential.

Code

• tonight-tweets: nodejs twitter bot
• tonight-sources: nodejs + cheerio scraper
• tonight: React+browserify frontend