Writing like a coder

tl;dr: here are some scripts I used to streamline my writing process using all free and open source tools 🎉

I spend a good chunk of my time coding and another good chunk of my time doing some creative writing. These are rather similar hobbies— both involve me sitting sedentary in front of a computer and typing. Somewhere along the way, I decided they should involve similar tools as well. There’s a ton of really good coding editors and tools out there but in my opinion not really any good and free writing tools. There are of course Word-esque products of various free-ness, but I’ve always had trouble writing creatively in a format that seems so final. I prefer something that makes the writing seem more like a draft, something that can still and should be edited, in order to relax and not get it all stressed trying to make the current words perfect. Something that still needs a ‘compilation’ step, for a coding metaphor, to be a final product.

Here’s what I wanted out of my writing process:

• Fast and easy way to type up a story with occasional formatting (italics, section separators)
• Version control for peace of mind and to make deleting things not permanent
• Easy way to search through all chapters
• Compilation into different formats (pdf, epub, etc.)

After a bunch of trials and experimentations, I settled on these tools, many chosen because they were the tools I already used for coding:

• Markdown for easy typing and formatting
• Git for version control, using GitHub’s now free private repositories
• VS Code as an editor with extensions for word count and spelling. Search is easy if the whole folder containing all the chapters is the workspace.
• Pandoc for compilation to different formats with markdown as an input

I wrote a bit more in depth a while ago about each of these components and the pain points they solved here.

My folder structure for my repo of a book might look like this:

book/
├── part-1
│   ├── chapter-01.md
│   ├── chapter-02.md
│   ├── chapter-03.md
│   ├── chapter-04.md
│   ├── chapter-05.md
│   └── chapter-06.md
└── part-2
    ├── chapter-07.md
    ├── chapter-08.md
    └── chapter-09.md

One annoyance is there isn’t really an easy way to figure out your book’s total word count outside of adding up each chapter’s word count. Luckily, we can throw together a script that’ll do it for us:

# word_count.sh
#!/bin/bash
FPATH=$1
if [ -z "$1" ]
then
    FPATH='.'
fi
find "$FPATH" -type f -name 'chapter*.md' | sort | xargs wc -w

This would output the word count of each chapter, with a summation at the end:

./word_count.sh book-1/
1117 book-1//part-1/chapter-01.md
3301 book-1//part-1/chapter-02.md
4447 book-1//part-1/chapter-03.md
2600 book-1//part-1/chapter-04.md
5882 book-1//part-1/chapter-05.md
4733 book-1//part-1/chapter-06.md
4348 book-1//part-2/chapter-07.md
5032 book-1//part-2/chapter-08.md
4976 book-1//part-2/chapter-09.md
...
95767 total

And to generate a PDF of all of the chapters together, I’d have another script that uses pandoc. I imagine a tool such as Scrivener which was developed for writers can do this as well, but that costs money while pandoc is free, open source, and highly customizable 😎

#!/bin/bash
pandoc -H options.sty --toc `find book -type f -name 'chapter*.md' | sort` -o book/book.pdf

This tells pandoc to use a provided options.sty style file, to make table of contents, and to use all of the markdown files in the given folder that match the expression chapter*.md. Then output it to book.pdf. And there you go, a nicely generated pdf document of all your chapters to send out to all of your eager beta readers!

Here’s the options.sty I use, though you could customize it however you want using LaTeX options (pandoc uses pdflatex under the hood).

\usepackage{setspace}
\onehalfspacing                                 # 1.5 spacing
\usepackage[vmargin=1in,hmargin=1in]{geometry}  # margin size
\setlength{\parindent}{2em}               # size of paragraph indent
\usepackage{indentfirst}                  # indent the first paragraph of every section
\usepackage{palatino}                     # use the Palatino font

Ah, one of your beta readers would like to read your shiny new book on their ereader! PDFs don’t work too well with ereaders, but no fear, pandoc can also convert your markdown files into a nice ebook with a few modifications to the above script.

pandoc -H ../options.sty \
--toc --metadata title="My Shiny New Book And Me" \
--metadata creator="Chip Skylark" \
`find ../book-1 -type f -name 'chapter*.md' | sort` \
-o ../book-1/book1.epub

Pretty much the only change is the output format. In this case, you can also pass in some metadata arguments that will affect how your file shows up on an ereader (title and author) or you can include it in a metadata section of one of your markdown files. More info on that here. And if you want the format specifically for Kindle, you can use something like Calibre (also open source 😎) to convert your epub file to mobi and it’ll work great.

Speaking of highly customizable, here’s another cool thing you could do with this writing workflow— incorporate git hooks!

While rereading some of my writing from a while back, I thought “Yikes, this is darker than I normally write. What was I going through at this time?” My life is pretty uneventful, but I do go through a lot of events when I read, which made me remember that I had been reading some Elena Ferrante at the time which is some devastating stuff. This definitely influenced my writing at the time. So I thought, hey, what if I recorded what books I was reading during every scene I wrote? And if a ton of people did it, maybe some sort of network of books that influenced other books would arise.

Since I already had this workflow set up, I cobbled together (in collaboration with Jimmy Zeng) a git hook that would figure out what books I was reading on goodreads, then append those to my commit messages as I wrote. Here’s the last few commit messages I had:

Git commit in writing repository with currently reading books

For reference, that git hook is here as a python package. But you could do anything you want! ✨

Overall, I’ve found this process works really well for me. I can focus solely on writing instead of formatting and can also incorporate weirdo ideas with git hooks in case I ever decided I wanted to do some analysis on my writing. Pandoc lets me export my writing in formats that anyone can read, and I don’t have to download much outside of what I normally already have on my dev computer 😄

I imagine this is only helpful to a very specific type of person, but if you’ve made it this far I wish you happy coding and happy writing!