iBook Lessons: Adding ASL support to iBooks

iBook Lessons is a continuing series about ebook writing and publishing.

Bilingual books have existed for a long time. Bilingual ebooks have also shown a presence in online stores. Adding American Sign Language (ASL) editions to ebooks? That's a proposition that has been hard to accomplish although possible under current EPUB standards.

The reason is that video and text must coexist on the virtual page. That's hard to do with ebooks, and impossible in conventional books. With the iPad and iBooks Author, that challenge has now become possible.

Recently, author Adam Stone published his first ASL/English bilingual ebook. Called Pointy Three, it tells the story of a fork that's missing one of its prongs but not, as the description points out, its spirit. The fork journeys through the land of Dinnertime, having adventures and looking for a place where he belongs.

The book's possibly unique ASL/English approach offers something new and special. Stone explains that children who use both languages, or who are learning ASL, benefit from this bilingual approach. On his blog, he writes, "[It] is not simply an English story translated into ASL; it is a story created with both languages in mind, swirling around the creative consciousness."

His motivation sprang from a desire to let children play with both languages. With iBooks Author, Apple provided the perfect tool for his needs. "I want to show everybody that it can be done easily, quickly, and cheaply," he wrote on his blog. "You don't need to talk to a publisher; you are the publisher." He added in a note to TUAW that "Apple products constantly open new frontiers."

I had the pleasure of sitting down with him to discuss his journey into iBooks Author, and talk about how the tool had inspired him.

TUAW: So how did you first hear about iBooks Author? And did you immediately think about ASL?

Stone: I first heard about it during Apple's education event this past Winter. I'd already been experimenting with American Sign Language ebooks. I was trying to use Composer by Demibooks but I was having a lot of trouble with that particular tool.

When I saw the iBooks Author product I immediately knew it was perfect for what I had in mind. I work at P.S. 347 The ASL and English Lower School (it's an ASL/English bilingual school). I use ASL all day so it's always on my mind. (I dedicate the book to the school, in fact - at the end of the book.)

iBooks Author looked super easy to use. Obviously video was a prominent part of the iBooks Author presentation. The layout tools looked flexible. And I especially liked how it was already linked to the iBookstore; it meant that publishing it would be easy.

TUAW: How did you develop the story for Pointy Three? And did you always intend to be writing for children?

Stone: I wanted this book to be a point of inspiration for others. There are a lot of us who are concerned about the lack of ASL/English materials for children and who are thinking of ways to ameliorate that. So this book was first and foremost to set an example -- to tell others, "You can do this too! We all can do this!"

I wanted to do a children's book first. I'm already a first grade teacher; I read children's books every day. I think adults should read children's books more often. They really are magical.

Pointy Three came to me out of the blue about a week after the Apple presentation. I was sitting on the N train and suddenly I thought of a three-pronged fork. And I typed out the whole story using Notes on my iPhone. It hasn't changed very much since then. I think wanting to belong somewhere is such an universal theme. Any child can relate to that.

TUAW: What kind of development effort in terms of hours did it take to build this book?

Stone: I think it took me about 40 hours, max. I polished the story and shared it with a couple of friends. Then I found an illustrator, Joyce, and we met a couple of times where she showed me some sketches. Then I connected with Lauren, the ASL storyteller, and we did the whole video shoot in less than three hours in her living room in Williamsburg, Brooklyn. I sent a rough edit of the video to Joyce and she did the illustrations based off Lauren's telling of the story.

The illustrations took about a month and half. That was the longest part of the development process--waiting for them to be done. When I got them, it was just a matter of cleaning them up, adding them into iBooks Author along with the video, laying them out just right with the text and changing the text to match the video and the illustrations. I showed a PDF version to a few people for edits, and then publishing.

I started around the middle of February, I think, and finished in mid-June.

TUAW: Were there any lessons you learned specifically about shooting video for inclusion in an iBooks Author product?

Stone: The interesting thing about having an ASL component is that you can't go back and reshoot the ASL - the signer will look different; the lighting will be different, and so forth. So whatever footage I had, I had to make sure that the English text matched it.

There's one part in the book where the ASL version and the English version of a specific sentence are on separate pages. It was a goof made while shooting the video, and I tried to change the English text to match, but it didn't quite work. But that's okay. What is significant in one language may not be as significant in another language. English and ASL are different languages with very different storytelling and cultural properties.

But one big lesson I learned - I edited it all in iMovie '09 (or is it '10? whatever the latest version is) and exported it to HD format using Media Browser in the Share menu. Much to my surprise, that format is not compatible with iBooks Author.

I had to compress and convert it to "Apple TV" format with a data stream of 2 MBps in 720p - which was perfect. I used MPEG Streamclip. This discussion thread helped a lot. So - that was a big surprise for me - that iMovie couldn't easily export to a format compatible with iBooks Author. No biggie, though.

TUAW: Do you worry about the product size? It's over 100MB, about 116MB if I remember correctly, but I think you nicely avoided the problem of a product that was too ginormous.

Stone: I knew video would make the iBook big. I experimented with different data stream rates, basically 1 MBps, 1.5, 2, and 3. I found that 2 was perfect. I also wasn't sure just how big the image files could be; each page has about a 1.5-2 MB PNG file for the illustration.

TUAW: Did you ever consider doing this project as an application instead?

Stone: I have zero app programming skills. To do so would incur lots of overhead costs and stuff like that. I didn't have time for that. I knew all I wanted to do was make a book and iBooks Author fit the bill. I know of other teams working on ASL/English storybook apps, though.

Of course I would love to make the book as interactive as I can: let kids play with Pointy Three, move it around the screen and fun stuff like that. But iBooks Author is strictly for making books with interactivity that is very boxed-in in the form of widgets.

I see that at the end of conventional iBooks from major publishers: the very last page has a nice widget where you can immediately give a star rating and write a review. I tried to research on how to do the same thing for my book but couldn't find the solution. So my implementation is very clunky: a hyperlink to the book's iTunes Store page.

TUAW: Are there any other features you'd put into future books?

Stone: Since publishing, I've got some feedback that sound/reading aloud feels missing from the book. People have gotten accustomed to children's books on the iPad speaking aloud. My book is silent - like a normal book, and like any other book I read on the iPad - I'm deaf so reading is always a silent experience.

So I didn't really think about that until some people told me they wanted the book to talk aloud to them, too. I'm still researching on how that works. It's not a native feature of iBooks Author, but I heard people have found ways to add read aloud to iBooks. But it sounds difficult. And you know what...it' s interesting because Apple has built in voice over. Why can't Apple allow iBooks readers to access that directly? Seems simple.

TUAW: Any more thoughts you want to add about this project?

I loved doing it. I want to do more. Surely this isn't Pointy Three's last adventure! Most of all, I hope others do it as well. I told some people, "If you can put together a Powerpoint, you can do this." That's how easy it was.

If people are interested in adding ASL support to their books and want to hire me, you can reach me at my blog or Facebook page. I also hang out on Twitter.

TUAW: Thank you for taking the time to chat!