Kindle Conniptions: How I Published My First E-Book

2/5/10Follow @wroush

This is the 80th edition of World Wide Wade since this weekly column on technology trends began in April 2008. Since we have so much material piling up in the archive, we at Xconomy decided to do something a bit different: We’re collecting all of the columns in the form of an e-book. It’s called Pixel Nation: 80 Weeks of World Wide Wade. Not only does the e-book bring all of the columns together in one easy-to-read place, but it includes a new introduction and updates on many of the early columns, which, frankly can benefit from some updating in the fast-moving world of high-tech. We’re offering both a free PDF version that you can read on your PC, and a $4.99 Kindle version that you can read on your Amazon Kindle or your iPhone.

Pixel Nation, Kindle Edition

I really hope you’ll check out Pixel Nation, because it took me a boatload of work to assemble it. I’m not complaining—I learned a ton, and I took on the project mainly for the experience. But what an experience it’s been! I’ve discovered that it’s damnably difficult to publish your own e-book, at least if you want to get it onto Amazon’s Kindle, the dominant digital book platform (for the moment). The whole ordeal has given me some new empathy for authors who have been complaining about the Kindle for years.

It all came as a bit of a surprise, considering that we’re more than a decade into the era of electronic book publishing, and new e-reading devices are popping up left and right. Did you ever hear the phrase “write once, run anywhere?” It’s the slogan Sun invented to describe the idea behind Java, a computer language that’s supposed to work on any device or operating system. I had figured that by now, publishers too would be in a “write once, read everywhere” world. Books and articles obviously start out in all sorts of formats (Word documents, Web pages, etc.), but you’d think that there would be some easy-to-use software capable of reformatting this material for any e-book device, right?

No such luck. Instead, there’s still a welter of incompatible e-publishing formats, each championed by different factions of the publishing world with conflicting business interests, and you have to customize your book for each one by hand. If you had to compare the current situation with e-books to the historical evolution of the Web, it’s as if we were stuck in 1996 or so, back when Netscape and Internet Explorer displayed Web pages differently, and you couldn’t publish a website without learning HTML and learning how to tweak the code to make sure your pages looked the same in both browsers.

If this is what the future looks like, I can understand why the big New York publishing houses aren’t dancing with joy about the e-book revolution. In addition to all the traditional design and typesetting work that goes into creating the print versions of their books, publishers who want to distribute their books digitally must now hire production lackeys to pore through each book, paragraph by paragraph, reformatting them for Amazon’s digital bookstore—and Sony’s, and Barnes & Noble’s, and soon Apple’s.

It seemed only fitting to sum up my self-publishing experience in today’s column, which is also Chapter 80 in the book. It’s a cautionary tale. E-publishing may be great for independent authors from a financial point of view—especially once Amazon starts offering 70 percent royalties this summer—but it’s still a nightmare from a technical one.

My first step toward creating Pixel Nation was simply to gather up all of my old columns, which meant copying and pasting them from the Web pages on Xconomy into a Word document. I would never have attempted this task before November 2009, when we added a single-page view option that lets you see an entire article on one page. Many of my columns are fairly long, so they get broken into two, three, or four pages on the site, and it would have taken forever to stitch them all together from these separate pages.

Pixel Nation: 80 Weeks of World Wide WadeNext I deleted most of the pictures, as photos tend to add greatly to the file size of an e-book. Then I went through all the old columns and added updates and wrote an introduction. Using a graphics program and a photo that I staged on my dining room table, I whipped together the cover image you see here and inserted it into the Word file. That was the fun part.

Now I was left with a big, long Word file. On a Mac, it’s easy to export a Word file to PDF, so creating that version of the e-book was child’s play. It was the Kindle version that really gave me fits.

Now, I am very fond of my Kindle. I got it in May 2009, and use it every day. I love the fact that it comes with an email address (like “barneyfife@kindle.com”) that you can use to e-mail Word and PDF files to Amazon; for only 15 cents per megabyte, Amazon will then convert the files into the Kindle format (called AZW) and transmit them wirelessly to your device.

In an ideal world, publishing an e-book—that is, getting it converted to AZW and listed in the Kindle Store and the Amazon.com website—would be just as easy. Unfortunately, Amazon’s conversion software doesn’t have much of a sense of style. The Word files that you mail to yourself never look as nice as the e-books that you can purchase and download. The conversion process tends to leave ridiculously large gaps between paragraphs, for example. And the files lack all the pleasant conventions of professionally published books, such as consistent chapter headings or a hyperlinked table of contents.

It turns out that if you want that stuff in your e-book, you have to build it all yourself. Did I learn this from Amazon? No, the company actually shares very little information about how to format books for the Kindle. The meager scraps of information that are available from the Help section of Amazon’s Digital Text Platform, the site where authors and publishers submit books for the Kindle Store, are cryptic and poorly organized. Almost everything I now know about this subject, I learned from Kindle Formatting: The Complete Guide to Formatting Books for the Amazon Kindle, by Joshua Tallent.

Kindle Formatting is itself a $9.99 Kindle e-book, although you can also order a paperback for $19.95. It’s a worthwhile purchase either way, because it’s chock full of arcane little details that Amazon doesn’t tell you about and that you’d never figure out on your own.

For example, I learned from Mr. Tallent (a digital publishing consultant with a firm called eBook Architects) that the only format that really looks right once it’s converted to AZW—and the only one that gives you the control you need over the book’s final appearance and behavior—is plain old HTML. And the easiest way to create a Kindle-ready HTML file is to make sure that your initial Word file uses consistent styles for elements like body text and chapter headings.

Since I use Word all day every day, I had thought I understood the program. But Tallent’s book forced me to figure out previously ignored features such as the styles pallette, so that I could then spend a couple of hours going back through the book’s 80 chapters and making sure that every headline was in “Heading 1″ style, every dateline was in “Heading 5″ style, and so forth.

Once that was done, I could save the Word file as a Web page (being sure to click the option for “Save only display information”—another arcane but crucial detail) and be reasonably sure that the formatting would be consistent once the book was on a Kindle. The next step, however, was to use a text editor to go back and remove the unfathomable amount of cruft that Word leaves behind whenever it saves a file in HTML. This is pretty much a manual process, though decent text editors—I used one called TextWrangler—have global search-and-replace functions that can speed it up. I was able to complete this part of my e-book project while I was stuck on a plane and had nothing better to do. But it’s a good thing it was a 10-hour flight to Alaska, or I would never have finished.

Oh, did I mention niceties like a hyperlinked table of contents? If you want one of those in your e-book, it’s a good idea to create it in Word before you convert it to HTML. The best reason to make sure that all of your chapter headings are in the same style is that this makes it far easier to locate them when you’re searching for the anchor text for the internal hyperlinks—another tip from the talented Mr. Tallent.

Let’s fast forward to the end—assuming you’re still with me. What you get, with enough endurance, is a stripped-down HTML file that will look nice almost anywhere, whether in a Web browser window or on an e-reader. Amazon’s Digital Text Platform lets you upload this HTML file to Amazon’s servers and specify a title, a price, and other details. If you do everything correctly, your e-book shows up in the Kindle Store and on Amazon.com within three days or so, and you can start your career as a best-selling e-book author.

I’m a geek who is comfortable with, though not totally fluent in, HTML, and this project severely tested my patience. I spent an estimated 10 hours trying to figure out how to do this, and then another 30 or 40 hours doing the grunt work to make it happen. So I think it’s safe to say that until somebody comes up with a way to automate e-book production, we won’t see a huge flood of authors self-publishing for the Kindle or the other e-book platforms. Perhaps I should have hired a consultant. Tallent charges a reasonable $60 per hour, and says he can convert a typical novel to Kindle-ready HTML in 1 to 3 hours and a longer non-fiction title in 3 to 6 hours. But then I wouldn’t have had this wonderful story to tell.

The situation is unfortunate, because part of the promise of the digital publishing revolution was that it would finally give authors a way to bypass the old literary establishment—the agents and editors and publishers whose job is to make sure that trees get sacrificed, and costly marketing campaigns get mounted, only for the most commercially viable titles. The way things are now, only big publishers, or maybe authors with some money to burn (and how many of those do you know?), are going to go to the trouble of getting their books onto the major e-book platforms.

Of course, all that’s needed to solve this problem is for one clever programmer to come along and build a usable e-book editing program. That’s roughly what happened on the Web back in 1995, when Cambridge, MA-based Vermeer Technologies introduced FrontPage, the first successful WYSIWYG HTML editor. Microsoft eventually bought Vermeer and FrontPage for $133 million, and I bet that the first company to build a decent e-book editor would get snapped up by Amazon or Apple. Entrepreneurs, are you listening?

For a full list of my columns, check out the World Wide Wade Archive. You can also subscribe to the column via RSS or e-mail, and you can download Pixel Nation, an e-book version of the first 80 columns, as a free PDF file or a $4.99 Kindle edition.

Wade Roush is Chief Correspondent and Editor At Large at Xconomy. You can subscribe to his Google Group or e-mail him at wroush@xconomy.com. Follow @wroush

By posting a comment, you agree to our terms and conditions.

  • Pingback: Kindle Conniptions: How I Published My First E-Book – Xconomy | Bargain Ebook Readers

  • phil

    If Mr. Tallent can convert a book in one to six hours, what’s the problem? If I were an Indie author and could make 70% profit on each eBook sale, that cost for a contractor seems like nothing. Sounds like there’s certainly no need for a major publisher, if that’s all the work it takes to format a book.

  • P J Evans

    LEarning how to create a Word formatting template took me several hours, with an instruction book, the one time I did it. That was for a manual at work (about 160 pages, with illustrations, which means a format for captions also).
    The entire manual required two or three months, because I had to redo all the images in it also. (At least I got paid for part of it – most of the work I did at home, on my own time.)

    Anyone who thinks ebooks can be done without editors needs to try it.

  • Bill

    One reason you had so much trouble is that Word produces *really* *horrid* HTML, as opposed to clean well-formed HTML. That’s partly because they’re Microsoft, and partly because Word is a product that’s evolved over a couple of decades rather than being built cleanly from scratch, but it’s also partly because HTML and Word are designed for radically different problems. Word is designed to let you make pretty-looking marks on screens and specific-shaped paper, while HTML is designed to let you categorize the *information* in a document, so that somebody (not necessarily you) can display it on a display that doesn’t necessarily look like yours.

    An HTML reader might be displaying contents on an 8.5″x11″ portrait-mode piece of white paper, or on a wristwatch, iPhone, or billboard, so what HTML really cares about is that one bunch of words are an H1 header and that you’d like to anchor a given picture at this point in the document, not whether the bunch of words are in 14-point boldface small caps or have enough room before a page break. There’s an auxiliary standard, CSS, that lets an HTML author give some hints about how you’d *like* the reader to display the document, if they can, but it’s just hints, because that’s the reader’s decision, not the author’s. And since Word is primarily about display (and PDF almost entirely about it), it’s got to add a bunch of ugly format requests around every HTML information tag, because the author might have changed the heading format partway through the document, or might have just specified that a given paragraph was in 14-point bold-face without indicating that it was an H1 header.

  • bowerbird

    wade said:
    > Since I use Word all day every day,
    > I had thought I understood the program.
    > But Tallent’s book forced me to figure out
    > previously ignored features such as
    > the styles pallette

    whoa. you’ve “ignored” styles? seriously?

    that’s a pretty brave thing to be confessing!

    all this time i thought you knew your stuff.

    > I bet that the first company to build
    > a decent e-book editor would get
    > snapped up by Amazon or Apple.
    > Entrepreneurs, are you listening?

    oh please. haven’t the “entrepreneurs” caused
    more than enough damage in the e-book space?

    i’m coming out soon with a simple light-markup
    authoring-tool that lets writers write in plain text,
    not bothersome, clumsy, heavy, intrusive markup.

    button-clicks will output e-books in powerful .pdf,
    and several versions of .html (for different purposes),
    as well as .rtf. other converter programs out there can
    then auto-generate the other formats you might want.

    my authoring-tool will be cross-platform and cost-free.

    -bowerbird

    p.s. send me your word file or your .html file, or better yet
    both, and i’ll show you how easy your book could have been.

  • Pingback: Introducing Pixel Nation: 80 Weeks of World Wide Wade | Travels with Rhody

  • http://ebookarchitects.com Joshua Tallent

    Wade,

    Thank you very much for purchasing my book and for your kind comments on it. I am very glad to know that I helped you in your conversion project! Hopefully the next book will be much easier to do, and do feel free to drop me a line if you ever need advice or assistance.

    - Joshua

  • http://www.ebookpartnership.com Matt

    Interesting article, I wonder whether the next version of Word will (as some whisperers are suggesting) have an export to ePUB feature. Seems like that would be a good idea and make life for a lot of self-publishing eBook authors a lot easier.

  • Outtanames999

    So it turns out that the Kindle (and probably all other ebook readers – uh the machines, not the people) are the dumbest devices ever invented. And to think, people pay money for them.

    For you see, the dirty little secret is that you could read these Kindle books on your non-smart phone circa 2007 because that is how dumb the Kindle devices are.

    Amazon themselves says their Digital Text Platform (DTP) “handles .mobi file formatting and images very well. For more information about the Mobipocket format and the Mobipocket eBook Creator, please visit the eBook Creator homepage mobipocket.com Important: Only unencrypted mobi files are supported.”

    Turns out Mobipocket Creator is free.

  • http://www.davidbatterson.com David

    Great Post! I am currently finishing up my first book project and I plan on self-publishing it. Any words of advice would be greatly appreciated!

  • Pingback: Saying Farewell to World Wide Wade | Xconomy