Thursday, February 29, 2024

Toolchain changes

 Skimming back over my blog entries, I realize I left out something important—at least to me. In addition to switching to Obsidian on the front end, and Pandoc for first-stage back-end (it's getting complicated), I dropped LaTeX in favor of Typst.

What is typst, you ask?

Let me back up.

My goal is to write a novel once, and debug it once. This never happens.  

Instead, I write a novel in Obsidian (Markdown), turn it section by section into .docx for my writing group. They respond with the track changes and comments feature in Word, or any of the other word-ish programs they use. I use TextMaker for this. So I go through their comments and changes and manually apply all the changes, and any other changes I see fit to make.

Then, when the novel's done, I compile the whole thing into a .docx and send it to E.C. Tobler. She corrects and comments, and I go through the entire novel, all 392 pages for Dead of Winter, and implement her corrections and most (if not all) of her suggestions.

Then the fun really starts. Once the book is ready, I make it into an epub, which Amazon can digest into a Kindle book, and into PDFs, which Amazon can turn into MoD books. If you're keeping score, that's five different output formats: single-chapter docx for the writing group, huge honking docx for E.C., EPUB, pdf for paperback and pdf for hardback. (though the last two are identical except for the internal ISBN number.)

 Then, after I release it, I find typos. Sometimes they're editing scars, things I introduced during the preceding steps. Sometimes they're search/replaces run amok. Sometimes they're things I missed in E.C.s edit comments. Doesn't matter. The point is, I have to regenerate three of those five formats again.

If I were to typeset the PDF versions with conventional typesetting software (InDesign, Affinity Publisher, etc), change tracking would be the bane of my existence. Did I fix the comma on page 205 in both print versions? It would only get worse as I add more MoD houses and Ebook sellers. 

I have to have, must have, for sanity's sake, I need one source of truth for my novels, one place I can make a correction and propagate it to all the different output formats in a completely automated format. For me, that source is the markdown file.

Pandoc can translate markdown into HTML, and thence into epub in one step. I can control what it looks like, to the extent that the Kindle App respects what I tell it to do (not much) by twiddling the Pandoc template file and the CSS file. These are the same for all the novels in this series.

For the PDFs, I used to use LaTeX. LaTeX is ancient. LaTeX is vast. LaTeX is poorly understood. I've described writing LaTeX code as "write code, sacrifice a chicken, and hope for the best, and sometimes it just breaks, even with the same input." 

This is where typst comes in. Typst takes a markdown language (its own, naturally) and writes it out into pdf, according to code logic. In short, it does exactly what LaTeX does, except that it's new, written in Rust (so stability is good), well documented, with coding paradigms that don't seem like you need a grimoire to understand them No chickens need be sacrificed.

Even better, pandoc has support for typst. So I can make a change, bake the document together (from individual scene files) in obsidian, pandoc it into typst source according to a very customized template, and compile *that* into a PDF. This is repeatable. Which is good, because I wind up repeating it. A lot. For my hardcopy books, both Amazon at the moment, these come from one amazon 5.5x8.5 pandoc template. All the novels in the series should use the same template here, too.

Typst has a few warts. Like LaTeX, it tends to go bananas with hyphen-breaking words. You can turn that off. The biggest wart for me is that it does not yet have the ability to make page blocks in a spread the same length. In fact, its widow/orphan/slug avoiding code will *always* break equal page lengths any time it cycles. I'm looking forward to when they fix that. But I can live with it.

I see I've put the sly implication above that I might be releasing books through more venues than just Amazon/KDP. I have to lay some groundwork for that. Right now, my books all have Amazon ISBN numbers, among other things, and there are a lot of moving parts I have to take care of before I can fix *that*. Once I do, this toolchain will become even more important. Everyone's EPUBs are different. If I'm doing MoD, their formats are probably subtly different. With one source of truth, the markdown files, I can live with that.

Way more info than you wanted? Probably. But maybe it will help someone else trying to put together a similar toolchain. If you are putting such a thing together, and you'd like to see my template files, pop me an email or comment.

Links here:

Obsidian with the following plugins: Advanced New File, April's Automatic Timelines, Easy Bake, Enhancing Export, File Order, Force Note View Mode, Global Search and Replace, LanguageTool Integration, Linter, MonoNote, Note Refactor, Novel Word Count, Outliner, Shell Commands, Smart Typography, and Templater.

Pandoc

Typst 

Languagetool (The desktop server) 

Affinity Publisher (Which I use to typeset my covers)

Canva (Where I create my cover art)

No comments:

Blog Archive