To Build a Website (Part 2)

I kept putting off updating this site for far too long.

Everytime I thought about updating it, I opened it and remembered how rough it was to update. That killed my motivation every time.

I was very happy with the tooling I had built to compile and optimize the source files for the site from the raw source. That wasn't the problem -- or at least wasn't the only or the biggest problem.

In no particular order, the first problem I had was that the files were awkwardly separated into folders based on their type rather than what they were related to. Originally I thought this made sense, since I was reusing a lot of common functionality. I was baking all of the javascript directly into the page itself (saves on extra server round-trips making the page feel snappier to load, and since the javascript was incredibly small, weighing far less than a typical image, I figured no one would notice or complain that their browser wasn't caching that extra 10kB as a separate file). But quickly became cumbersome as the site grew larger. All the javascript for all of the fancy animations on the home page were separate from the template files and searching through folders to jump back and forth between them to make a change wasted a lot of time.

Another issue that I had to deal with was that the build tool I wrote was still a little buggy. Since it was written in javascript, which is inherently single threaded, I paid no attention to trying to make sure that it was thread safe or anything of that sort. But I did use promises and async/await very extensively. Normally this would be fine, the promises create the data race barriers so that everything Just Works™. So this one was my own fault -- I had intended to make my build tool watch for file changes and have it immediately update the output files so that iteration was super easy. I had also intended to only process the files that had changed and cache all of the other results, so for example, if the HTML template for a page changed, then it would re-use the javascript that was already compiled and minified from typescript. The tricks that I hadn't thought to deal with were:

1. when you make multiple quick edits so that the file isn't finished being processed before another change happened causing the pipeline to be re-triggered
2. what to do when processing a file results in an error -- like having a typo in the typescript source file

Finally, because I had written most of it on the weekends while bored and probably after having a couple of beers, it wasn't tested or even particularly well documented. Which really didn't make it easy to just go in and fix all the issues.

All this put together meant that I really needed to fix the build system.

If there's anything that can be learned from this experience, I would say that you should either not try to roll your own build system -- there are plenty already out there to choose from, that are probably better than you or I could easily make, and if you really need a specific feature, customization with small tweaks is probably easier. Or that you should really make sure that it is solid from the very beginning, don't start the foundation on rocky ground.