Eureka! I Have a Workflow

Warning sign WARNING: Geeky HTML and Makefile stuff in this post will render your eyeballs glassy if you’re not into that sort of thing.

I have been trying for weeks to create a workflow that allows me to edit my manuscript in Markdown format and then, with a simple command, generates all of the document formats I need to publish my book. That way I can create review copies and send them out, get feedback and make the changes I want, and then do it again — all in a precisely correct, reproducible way. Yesterday, in a long session, I managed to get the tools to do what I want. Maybe sharing this will help someone someday?

The requirements I had were:

  1. No manual steps at all. Just edit the manuscript and type make to get all of the required output files, ready to print or upload to Amazon, Google, Apple, my own Kindle, or whatever.

  2. Allow me to tag text with classes in my manuscript so I can include things like a real title page in a title-page-y font, a haiku, emails or text messages formatted so they look like they would on a computer screen, or even a THE END at the end of my story that is more than just some bold text on a line by itself. I wanted control over my inline and block element classes so I could do custom CSS for these special things.

  3. Let me include fonts in the output files, minimized to just the glyphs I use in my manuscript and ready to be used by the eBook reader if the user wishes it.

I use Python Markdown to generate HTML from the Markdown formatted manuscript and Calibre‘s ebook-convert command to do the multi-format conversion, taking my HTML, CSS, and font files as input and generating EPUB, AZW3, PDF, and maybe MOBI format.

Even though I could have Calibre invoke it for me, I use the Python Markdown tool directly. I was having trouble getting Calibre to turn on the Markdown extensions I needed, and I just decided to Hell with it. Don’t get me wrong — Calibre is a wonderful tool, and I thank the obviously brilliant Kovid Goyal for creating it for us all to use. By all means, if you use Calibre, please consider donating some money or some code. The Markdown extensions I ended up needing for Dying to Live Forever were attr_list, smarty, fenced_code, and sane_lists. The output of the Python Markdown step is an HTML file with inline and block element classes I can style using my CSS to make everything look just the way I want it.

Some notes about Calibre, in case anyone wonders why I did things the way I did.

First, I couldn’t find a way to get Calibre to substitute my scene breaks with what I wanted. The HTML file I gave Calibre used simple <hr />, but nothing I did — and I tried for quite a while — seemed to be able to trick Calibre into substituting what I wanted, which was <div class="hr">&larr;&nbsp;&rarr;</div>. So I just brute-forced this using a global search-and-replace operation on the HTML before I gave it Calibre to handle the rest.

Second, as I mentioned before, I couldn’t get Calibre to use the extensions I wanted to pass to markdown_py, so I just brute forced that too.

After the Markdown conversion, I run ebook-convert for each file format I need. Today I have EPUB and AZW3 formats, but I expect to add PDF soon so I can give a dead trees copy of the book to some folks who don’t use eBook readers. Along the way, this dead trees format was very nice for marked up review copies with scribbles everywhere for comments and corrections.

This ZIP file contains the Makefile and CSS I used. I used Calibre version 2.4 and Python Markdown version 2.4 (coincidence?).