Towards Editoria 1.0

We are approaching Editoria 1.0. Here is a quick insight into how it all fits together.

First of all there are basically 3 modules (so far) that comprise the system – the Dashboard, the Book Builder, and the Editor.

The Dashboard

The Editoria Dashboard displays the books that you are working on.

d

On the Dashboard you can do the following operations:

  1. Add a book
  2. Rename a book
  3. Edit a book
  4. Delete a book

The following short video demonstrates these actions (I recommend clicking on the icon at the bottom right to view the video fullscreen).

Book Builder

The Book Builder module is so named because this is how you ‘build a book’. When you first create a book from the Dashboard and click ‘edit’ you will see the default display of the new book.

b

You will notice there are empty Divisions – Front Matter, Body, and Back Matter. To these you can add new Components. Components can be either Parts or Chapters (note, we follow naming conventions from the Chicago Manual of Style where possible).

Front Matter and Back Matter components can be called anything you wish or you may select from a default list comprising of items such as Table of Contents, Preface, Introduction (Front Matter) or Appendix A, Glossary etc (Back Matter).

The Body components are not selected from a default list since these are the titles of the Chapters and Parts of the book you are working on

Using the Book Builder you can build up the structure of the book as shown in this short demonstration video.

The Book Builder also enables you to:

  1. Set the status for any component (this also effects access permissions)
  2. Set the pagination-break rule per component (left/right breaking page)
  3. Delete a component
  4. Rename a component
  5. Edit a component

We’ll cover these in a later post. One additional feature that is important to cover however is the uploading of Microsoft Word files into the system. Editoria supports uploading a docx file into a component. The content is converted automatically into HTML and loaded into the system.

The Editor

We have built a very special editor for Editoria. This is designed by the University of California Press staff to reflect how production staff ‘think’ about content, and supports scholarly publishing workflows for preparing a text for publishing.

ed

This editor is very sophisticated and can be customized to meet the needs of different publishers. Currently the features include:

  1. Adding / removing inline styles (Italics, superscript etc)
  2. Source notes
  3. Track Changes
  4. Annotations (commenting), including replying, resolving etc
  5. Notes management (with track changes and commenting)
  6. Independently scrolling notes panel
  7. Block Styling (headings, lists, prose/poetry styles etc)
  8. Image placement
  9. Undo/redo
  10. Linking

The following brief demonstration video illustrates some of these features in action.

Collaboration Support

Editoria supports collaboration. The model we have chosen to support for this upcoming release is a component lock paradigm. This means no two people can edit the same component at the same time. Hence, if one person is editing the Introduction, then they have the lock on that content until they are end their editing session.

To support this type of collaboration we have synchronized component locks and other items on the Book Builder. You can see this in action in the following demonstration video (changes on left mirrored on right).

Additional Features

In addition to the above there are other major features which we will demonstrate another time, including:

  1. Team management (role based)
  2. Access permissions
  3. Export to EPUB

Where to Get Editoria

Editoria is 100% free. It is open source (MIT license). Editoria is a web-based platform which means you need to install it on a server and access it through your browser. If you know someone that can do this then the can find the latest source code here:

https://gitlab.coko.foundation/editoria/editoria

If you wish to join the Editoria community subscribe to the newsletter. Mailing lists, community events and much more coming soon! Look for us also at SSP and AAUP 2017.

Editoria at SSP and AAUP

The spring/summer conference season is about to kick into high gear, and we’re here to help you plan your travel. We’ll be busy at both the Society for Scholarly Publishing and Association of American University Presses conferences in May and June, and we’d love to connect with you to talk about Editoria.

Society for Scholarly Publishing, Boston, MA, May 31st-June 2nd

Erich van Rijn will be participating in several session at this year’s SSP meeting in Boston between May 31st and June 2nd, and is also available for individual appointments to talk about Editoria. Come find us at the following sessions:

Open Source Tools for Scholarly Publishing — This is a pre-meeting session, organized by Kristen Ratan of the Collaborative Knowledge Foundation, during which we’ll be talking about incorporating open source tools into scholarly publishing.  We’ll explore how to build them and enhance adoption, and how tools like Editoria can improve scholarly publishing workflows.

Open for Discovery? Open Access Monographs in Scholarly Research Workflows — Erich van Rijn will be moderating a panel on discoverability of open access monographs.  The panel will include presentations from Jennifer Kemp of Crossref, Frank Smith of JSTOR, and Wendy Queen of ProjectMUSE.

Preview Session: New and Noteworth Product Presentations — We’ll be doing a five-minute pitch on Editoria alongside a number of other cool new tools during this session.  Please come check this session out. From what I’ve seen, there should be a number of great presentations in addition to our own.

If you’d like to set up a time to talk outside of these sessions, please get in touch.

Association of American University Presses, Austin, TX, June 12th-13th

Interest Group Meeting — I’m organizing a breakfast session at this year’s AAUP, which will happen on June 12th.  Please use this form to get in touch or tweet me (@erichvanrijn) if you’re interested in attending.  I’ll be giving a brief overview of the software, answering your questions, and discussing how we can create a community of Editoria adopters, developers, and partners.

Individual Meetings — If you’d like to schedule an individual meeting to discuss Editoria, please use the following form to request a meeting, and we’ll be in touch.  I’d be more than happy to get together with anyone who’s interested in a deeper dive.

We look forward to seeing you this spring!

 

 

Editoria Update — January 2017

This month’s Editoria update helps us usher in 2017 a bit belatedly. January has passed us by quickly, but there have been no shortage of Editoria developments that we are eager to share with you all, so please read on.

 

To begin with, we are embarking on a new era in our outreach efforts for Editoria.  For the last twelve months, we have been largely directing our communications and progress reports to a small, dedicated group of advisers to the project.  This group has been tremendously valuable to us in our early efforts to build the system.  But starting with this newsletter, we will be delivering more regular missives to a larger group of contacts we have cultivated during our work on Editoria who have asked to be kept better apprised of its development.  If you’re new to these updates and would like some further background, you might want to head over to our blog at editoria.pub and check out last month’s update.  There is some other good information about the project, and we are trying to keep it up to date.

 

So, where are we?

 

As I mentioned, January has been a big month for Editoria.  To begin with, our friend, Adam Hyde, the co-founder of Coko Foundation and one of the chief facilitators of the development of Editoriahas actually just used the system to produce the first short book that has ever been run through the system.  The book is called, The Cabbage Tree Method, and it outlines his methodology for open source collaborative product development.  As Adam writes on his blog, “The book was written and rendered in entirely open source tools including Editoria and Vivliostyle.  HTML all the way home….”  An impressive achievement.  You can download a copy here (PDF) or here (EPUB).  I’m reading it right now, and it’s not just a good example of HTML typesetting, it’s a great read.

 

In other news, Coko’s development team has been hard at work implementing a very important feature of the system—the ability to track changes to text in Editoria‘s WYSIWYG editor.  This is critical functionality because it allows editors and authors to work in the system and better understand how they are interacting with the text and with each other in the collaborative effort to produce the book.  We will be turning our project editors and freelance copyeditors loose on this soon, but here is an early look at the current implementation, version 0.1, of this feature:screen-shot-2017-01-26-at-9-07-29-pm

 

February and March will see more important updates to the system, and we will also begin doing some of the heavy-lifting involved in user testing of the application during the coming months, but this is all great progress.  Again, see last month’s update if you would like a rundown of other features that we’ve been building in the system.

 

We will keep you further informed of our progress going forward.  Please feel free to share this update with any other folks who might be interested in Editoria.  Also, if you have questions, want to talk over our approach, would like more information about the application, or just express your enthusiasm for HTML-based book production workflows, we are always glad to hear from you, and we can be reached at either evanrijn@ucpress.edu or catherine.mitchell@ucop.edu.

Happy New Year from Editoria

This is a slightly edited version of an update on the Editoria project that was sent to our advisory board in December.  Going forward, we will be posting our monthly notes here on the blog, and delivering them to a broader group of contacs that we have cultivated in our conversations and travels over the course of the last couple of years working on the project.  If you are not currently on our email list and would like to receive regular updates, I encourage you to fill out our contact form.
The Editoria project is indeed moving into an exciting phase of its development, and I wanted to share some of our recent accomplishments and what lies ahead for us in 2017.
What We’ve Accomplished
Adam Hyde at the Collaborative Knowledge Foundation has an excellent post on his blog that summarizes a lot of the work that’s been done on the overall architecture of the system, and also specific features that have been built into the system.  I encourage you to read Adam’s post, which I think summarizes the current state of the application quite nicely.  However, I will briefly recap some of the important work that’s been done:
  1. Built team manager functionality that will allow the various members of a team involved in the production of a book (e.g. project editor, copyeditor, author) to interact with the text based upon permission assigned to them.
  2. Designed and implemented a commenting and querying system in the editor that allows for participants in a project to interactively comment on the text.
  3. Built out the libraries, known as XSweet, that are required for converting Word files to HTML for editing and display in the Editoria system.
  4. Developed and implemented a chapter structure navigation in the right column of the editor that allows a participant to easily navigate to any component of a chapter.
  5. Output print-ready PDFs from HTML using the Vivliostyle HTML/CSS typesetting engine.
What’s Up Next
Between now and mid to late April, which is when our current project plan has us beginning to pilot books through Editoria, we will be working on several critical pieces of functionality necessary to successfully use the the system in production.  These include:
  1. Adding track changes functionality.
  2. Integrating the system with the INK file conversion service that allows for automated upload and conversion of Word files.
  3. Adding an index builder.
  4. Building of the image management functionality necessary for image placement.
  5. Automating the output of PDF and EPUB files.
In addition, we will be doing some extensive unit testing of the various components of the system. This phase is currently planned for February/March.  We will be scheduling some webinar overviews of the system in the next couple of months—watch out for announcements.

Why Pages Matter and What We’re Doing about It in Editoria

I came up for air recently try to catch up on what’s been going on in scholarly publishing. I love Bill Trippe’s Flipboard magazine on university press publishing, and I value the crowdsourced knowledge of all the hundreds of folks I follow on Twitter, but I wanted a deeper dive.  I wanted to create my own news stream, so I installed a good old fashioned RSS reader, and started loading up some of my favorite blogs and websites.  Today, I am going to use my limited powers of social media persuasion to encourage readers of this blog to check out the new Paged Media blog.

Paged Media was started by Dave Cramer of Hachette and Adam Hyde of Collaborative Knowledge Foundation.  In addition to contributions from its founders, Paged Media so far has contributions from several folks who are trying to make the open web support books and other paginated content better:

Paged Media is dedicated to a simple mission: “To educate and inspire publishers to use HTML, CSS and JavaScript(JS) to produce print and paginated electronic content.”

But some of you may be asking, “In this era of digital scholarly publishing, why is pagination important?”  I can tell you that pagination is crucially important to digital scholarly publishing, and it has been a critical development concern for Editoria.  Why?  Consider these use cases, reprinted from the first public working draft of the W3C’s “Portable Web Publications Use Cases and Requirements” document:

  • Ann reads War and Peace, which, when printed, is over 1200 pages. In order to have a better sense of her progress in the book and to make navigation within the book easier (i.e., to support usability) she decides to switch her reading environment to paged view.
  • Susan uses CSS, images, flexboxes, and more to create a rich, interactive publication on the history of a city. Each major historical milestone is defined as a standalone unit that would be a single page when printed, with a timeline with the main events in the footer area of the page.
  • Fatima wants to choose between a scrolled view and a paginated view of content that extends across multiple HTML documents.
  • IndyPublisher wants to provide transition effects between pages, both within and across content documents.
  • Mr. Oayia, a classroom teacher, says, “turn to page 137 of your textbook.” Students with both print and digital editions need to find the same text.

All of these are use cases that regularly present themselves in long-form scholarly communications, and any systems built to support text-based scholarly communications, including Editoria, need to be able to support them in some fashion. While the Portable Web Publication is being developed to enhance HTML-based formats to accommodate pagination, a lot of the functionality that is represented in these use cases can already be found in the good old PDF.

Our approach to this with Editoria has been to use the Vivliostyle CSS typesetting engine.  Vivliostyle has allowed us to develop the Editoria application in a way that supports both paginated (PDF) and non-paginated (EPUB) outputs at the end of the book production process.  While pages, particularly printed pages, have their limitations (eg. multimedia content), in many cases, pages are critically important to monographs as well as to the production other types of scholarly and educational materials, and to making them useful to readers.

While controversial, the proposed IDPF/W3C merger, if successful, and the work of the group developing the portable web publication specification may eventually succeed in bringing much of the functionality necessary to represent the concept of pagination to the EPUB specification.  The W3C is the right forum for this work to be done, and would likely help advance the use cases for pagination that scholarly publishing demands of EPUB and help abolish the bifurcation of the PDF/EPUB world and our reliance on the two-dimensional PDF format.  If that world emerges, Editoria will stand ready to support it. For now, most monographs will have pages, and Editoria will be developed in such a way that those pages can be represented at the end of the production process, and allow publishers using the system to give readers the choice of whether they prefer to read in paginated or unpaginated formats.

 

Word to HTML Conversion—Progress Report

In a blog post, Martin Eve, Professor of Literature, Technology and Publishing at Birkbeck, University of London, and Editoria advisory board member, summed up the impact of Microsoft Word on the current system of scholarly communication:

It may sound overblown, but a crucial stumbling block in reconfiguring the economics of scholarly communications for the digital age is Microsoft Word. Specifically, the fact that users are wedded to this format presents typesetting and conversion costs that are completely out of proportion to the needs of the system.

I encourage everyone to read Martin’s post, which summarizes nicely one of the central challenges posed by the Editoria project: how to get manuscripts out of Word as early in the editorial production process as possible.

From the outset, we felt that the only viable way to both control the outputs of the editorial production process and build a collaborative editing environment was to remove Word from the process entirely.  Because the use of Word is so ubiquitous among authors of monographs and journal articles there are many pieces of software that have been built over the years as add-ins or plug-ins for Microsoft Word.  While this approach can be very successful in an environment where documents are passed from one participant to the next, it is not appropriate for a collaborative, web-based editorial production environment, which was what we intended to build.  Moreover, it forces publishers to continue to rely on a piece of proprietary software that was never designed with the needs of scholarly communication in mind.

Thus, one of the first tasks we set ourselves in the development of Editoria was developing an ingest and conversion mechanism that would allow project editors (or anyone else on the team for that matter) to upload a Word document and convert it to HTML, which is how content is stored in Editoria. This would allow authors to continue to use their tool of choice, Word, to craft the manuscript, and allow us to enforce document structure and semantics once the manuscript is submitted and we take over production.  We’ve been working with Wendell Piez, Alex Theg, and Adam Hyde at the Collaborative Knowledge Foundation on developing a set of XSLT 2.0 stylesheets for extraction and refinement of data from MS Office Open XML (.docx) format and producing HTML for editorial workflows, which will be incorporated into Editoria. 

As it exists now, the first part of the conversion is to extract as much information as possible from the Word document and clean it up.  Initially there’s some of noise in it—things like duplicate tags, Word-specific tags we don’t need in the HTML, or empty tags left over from user actions in Word (say, bolding and then unbolding some text), so there’s a fair amount of cleanup to do.  We want to make this extraction and cleanup as robust as possible so it can handle lots of author inputs.

The second part of the conversion is enhancing the clean HTML.  Currently, the conversion translates footnotes and endnotes from Word into the HTML as numbered links that connect the endnotes and their inline callouts.  It’s also capturing indentation for paragraphs and block quotes.  Soon the converter will be able to recognize things like links, potential headings, text alignments, etc.  It’s a long road, but the Coko team has made significant progress on developing enhancements and refining the tool.

Because the MS Office Open XML format is so complex, and because there is so much required to tidy up a document that was authored in Word and present it as a structured HTML document, there will always be a crucial role to be played in styling and structuring a manuscript that will necessarily need to be done post-conversion by a skilled editor or someone familiar with a book structure. But, the work that the Coko team have been able to do on creating a conversion pipeline, is getting us much closer to the day when all work on a manuscript after submission will be able to happen on the web.

Editoria 0.4 Planning – Collaboration & Infrastructure

With the Book Builder interface nearly complete, the team is now turning their attention to the authoring and editing interface which, in addition to supporting composition and styling, will also need to provide tools for commenting and task management. To meet these needs, the development team will be working on both interface and infrastructure challenges in the month ahead.

editoria_03_authoring_editingCollaboration interface

Editors, authors, copyeditors & proofreaders all need a way to mark up text, engage in conversations around proposed changes, and track tasks related to these comments. Editoria will make use of space directly to the side of the main “page” area to enable this functionality. Comment threads will link directly to the text that needs changing, and threaded conversations related to those comments will be enabled. Users will also be able to use quick shortcuts to create tasks within comments or mark conversation threads as ‘resolved’ so everyone can keep on top of what work remains.

Authoring & editing infrastructure

One of the main goals of Editoria is to meet authors where they already do their work. This means that, in addition to providing an intuitive browser-based authoring tool, Editoria also needs to support import of existing manuscript materials in Microsoft Word *.docx format and perform some clean-up on imported files to match the publisher’s style guidelines. The development team will deliver an initial prototype of this functionality alongside the new collaboration interface in Editoria 0.4.

Looking ahead

We are also actively gathering requirements for remaining development challenges scheduled for completion by January 2017, including:

  • User and role management
  • Support for basic art placement
  • Tag-based index generation
  • Automated typesetting and multi-format output