Tom MacWright

tom@macwright.com

Working on documentation.js

This week, the second week after I left my full-time job, I’m going to devote some good, focused time to hacking on documentation.js. This is the ‘kickoff post’, so here’s some context for why, what, and how.

documentation.js is an API documentation generator for JavaScript. It’s intended for structured, comprehensive documentation, as opposed to narrative documentation, and is a documentation generator in that it reads source code and writes documentation.

I started documentation.js because I needed a great tool for generating documentation for Mapbox GL JS, Simple Statistics, and Turf. documentation.js is inspired by, and uses the comment syntax from, JSDoc, a really great project by Michael Matthews, currently maintained by Jeff Williams at Google. JSDoc was, and is great, but there were some things I wanted to improve on:

  1. The Babel toolchain is just so incredibly exciting: it gives us the ability to reuse lots of great work and support bleeding-edge JavaScript.
  2. Type systems for JavaScript - Flow and TypeScript - mean that a lot of the type information previously encoded only in JSDoc comments are now written in code, and a great documentation generator could reuse that type information in documentation, greatly reducing documentation tediousness.
  3. I want to write something that’s modular first, that’s designed for multiple output mechanisms and usable as an API. Along with this means investing lots of time creating a great theme system and fabulous themes for the defaults.

Personally, there’s just that documentation.js is really, really challenging. Even with my natural inclinations to reduce scope, it’s a really big, hard, challenging problem. That’s why I’m so pumped to work on it: looking at the expectations around the project, thinking of the infinite ways that people express ideas in code and imaging a system that works well with 99% of them - it’s inspiring. I admire folks who have built the big, essential parts of JavaScript tooling, like Babel, eslint, Flow, prettier, and I want to dig in and understand the problem space really well. I feel impostor syndrome when I work on the project, and I intend to push though it.

Here’s what I’m planning on working on this week:

  1. Principles. Principles are so important: to have the basic theory and concepts around which reality is formed. Some parts of documentation.js have these, but many don’t, and as a result my efforts have been ambiguous and iterative. I intend to learn, deeply, some of the solutions to long-running bugs and flaws - understanding tree datastructures really well, researching JSDoc namespaces, learning inference patterns.
  2. Research into history. I haven’t read all of JSDoc yet, and I should have read it months ago. There are also lots of projects that do similar things and I want to read them - essentially, read the code, end to end. I’ve done it with many other projects, and it’s always been useful. I adore the Alan Kay quote that “The lack of interest, the disdain for history is what makes computing not-quite-a-field.”, and I want to do better.
  3. Community. I’ve had some great conversations with folks closer to the Babel project, as well as some project maintainers. I want to spend as much time as I can bringing new people on board or learning about projects that might gel with documentation.js.
  4. Project management and support. Essentially, triaging the many issues we have filed, finding categories for them, and planning out releases that have themes so I can feel like we’re getting toward 100%, rather than chipping away at issues.

Some more thoughts. I really agree with substack, who says “Recent commits tell you that a piece of code hasn’t yet made up its mind about what it needs to be. It is not a sign of health.”. I would love to be at a place where I just don’t have as much to do on documentation.js. Part of the scope for documentation.js will always be something of a moving target, because it’s a piece of JavaScript tooling and JavaScript itself is moving fast. But that will be once we’re tracking the bleeding edge: first we need to get there by supporting 100% of JavaScript and Flow.

More thoughts. The irony that spending my free time on a project that will benefit some companies is not lost on me. In the larger context too, some of the top developers of the most vital software out there have few ties to any BigCorp. It’s kind of a curse of generality - companies that ‘leverage’ open source to ‘focus’ on delivering sandwiches or an internet of things API for your chair tend to avoid investing in open source projects, much less maintaining them. Documentation.js is kind of “that thing everyone needs and nobody wants to build themselves but they’ll use it once you’re done” - not specific enough to any startup that they would justify the time it takes to build it, but as a shared resource for many startups, far more valuable than the hours would cost.

There might be ways past that, which I’ll leave as an exercise for the reader. For my personal motivation and context the feeling of a real need and a real opportunity to create something great is enough: it will be privilege to learn and build it. It’ll be a lot of fun.

I’m feeling really focused and optimistic right now: along with my Mapbox repositories, I’m also unsubscribed from all of Turf, osmlab, and openstreetmap organizations, so my notifications are sparser than ever. Plus, using unfollow-everyone, and app I created on Glitch, I unfollowed nearly everyone, and it feels good. I’m really proud of the work I’ve done on other projects, but other maintainers will take the reins and I’ll happily move a bit further from burnout and closer to focus.

I’ll keep posting back with notes and journal-style updates, as well as hopefully finishing up a few queued-up old blog posts and getting those live.