Document Drafts
Description
Context
When a document is edited in realtime, the changes are auto-saved directly into the document, which means they are published immediately. As a consequence, work-in-progress content becomes visible to everyone as soon as it is typed, and any merge conflict with concurrent changes has to be resolved at auto-save time, in the middle of the editing session.
This proposal introduces document drafts: the changes made while editing are saved in a draft, separate from the published document, and the authors publish the draft when they consider that the content is in good shape.
Use Cases
- Several users edit a page in realtime. The changes are auto-saved in a draft and the published page stays untouched until one of the authors publishes the draft.
- The server is restarted during a realtime editing session. When the users come back, they resume the editing session from the draft, without losing any of the auto-saved changes.
- A user starts editing a page that doesn't exist yet in realtime. The page is created only when the draft is published.
- A user adds a new entry from a Live Data. The new row is backed by a draft that is published when the row is committed (e.g. when all the mandatory fields have been filled).
- A user edits a page whose name doesn't follow the configured naming convention. Editing is allowed, as a draft, but publishing is blocked until the name is fixed.
Requirements
Persistence
A document draft must survive a server restart. For instance, the changes auto-saved from a realtime editing session must not be lost when the server is restarted, so that the users can resume the realtime editing session. Keeping drafts in memory is not an option: it would require special handling for replication in cluster setups, and we already know from temporary attachments, which are stored in memory (in the servlet session), that this approach causes problems.
Multiple Drafts per Document
A document can have multiple drafts. Users of different editors cannot join the same realtime editing session: the realtime wiki editor can propagate changes that contain partial syntax (e.g. the start of a link) which can't be represented in a WYSIWYG editor, and different WYSIWYG editors (CKEditor and BlockNote) can use different technologies to synchronize the content. Each editor / edit mode that supports realtime editing therefore needs its own draft.
The Inline Form edit mode might be challenging in this regard, because a single form can contain fields that use different editors or pickers (e.g. a field that holds code and uses the wiki syntax editor, next to a field that holds a rich text description and uses the WYSIWYG editor).
Publishing
Users must be able to publish a document draft, which means merging the changes from the draft into the original document. In the realtime editing use case, the draft can be published once the realtime session that created it has ended.
Since a document can have multiple drafts, some of them can be published before others, and the document can also be modified directly, without a draft. Merge conflicts are thus possible and the user has to resolve them at publishing time. This is actually one of the benefits of drafts: merge conflict resolution is delayed until publishing, instead of happening on each auto-save. On the other hand, users may need to be informed when the original document and the draft have diverged, and be offered an action to rebase the draft changes on top of the latest version of the original document.
Awareness
Users must know that they are editing an existing draft when they enter edit mode. It would also be useful to know that a draft exists for another edit mode (e.g. when entering the Wiki edit mode, inform the user that a draft exists for the WYSIWYG edit mode). In view mode, users should be able to see whether the displayed page has drafts.
Drafts for Non-Existing Documents
It must be possible to have a draft for a document that doesn't exist yet. The main use case is entering realtime editing on a new document: the document is created when the draft is published. Another use case is adding new entries from a Live Data, where the new row is mapped to a draft document that is published when the row is committed.
Proposal
Storage
We propose to implement drafts as a core feature, stored in the database using the existing tables. Drafts should behave like regular documents (with attachments, objects, etc.), so we propose to add new columns to the existing tables in order to distinguish draft documents, attachments and objects from their original versions.
Entities are linked to their document in two ways:
- by document id, which is common when the linked entity is not shared between document translations. No extra column is needed in this case, since the entity can simply be linked to the draft document using the draft's id. For example, attachments are linked to their owner document by id.
- by document reference, which is common when the linked entity is shared between document translations (and is thus linked to the default translation). In this case the corresponding database tables need extra columns. For example, objects are linked to their owner document by reference.
Should a draft have the same reference as its original document? We would prefer so, but this requires adding a new column to all the tables that link to a document by reference. This could be an integer column whose value indicates which draft it is (WYSIWYG, wiki, etc.). It was also suggested to use this opportunity to make the mechanism more generic and support other kinds of special documents besides drafts, such as deleted documents, in which case the new column would rather indicate the document type.
For attachments, we need to check whether the files of the original document and those of the draft can conflict in the file system attachment store. We would also like to avoid duplicating attachment binaries in the file system store, by having the draft reference the attachments of the original document rather than copying them.
Versioning
We need to check whether document revisions are linked to their document by id or by reference, to know if the revision tables need the new column as well.
Search (Solr)
The Solr index may need a new field to distinguish drafts from regular documents. Do we need a new entity type (e.g. DOCUMENT_DRAFT)? If so, a corresponding type would be needed for every other entity that can be bound to a draft (attachments, objects, etc.).
Events
- Existing event listeners must not be called for events generated by drafts.
- However, we probably want notifications for drafts, and some listeners, such as the access rights object listener, may need to be called.
- Two options are considered:
- introduce new events, e.g. DraftDocumentUpdated
- add a flag to the existing document events (e.g. isDraft) so that event triggering can be filtered: by default, event listeners don't receive draft events, but they can opt in by setting the flag.
Export
- Drafts should probably be included in XAR exports. Exporting only the draft doesn't seem necessary, since it is unclear what importing a draft should do when the original document doesn't exist.
- It is not clear whether PDF export needs to support drafts.
Page Creation and API
- Drafts must be supported for documents that don't exist yet.
- We imagine a DraftXWikiDocument class. The API would receive a document reference and create the draft by "copying" the specified document, or create an empty draft if the specified document doesn't exist.
Page Index
- A dedicated Drafts tab in the Page Index would be useful to access previously created drafts.
Publishing
- Users could choose whether to squash the draft revisions into a single revision of the original document, or to keep them all. Squashing raises the question of how to preserve the authors, which Change Request (CR) needs as well.
- One option is to support multiple authors per revision. Since the original author is only used for display, it could become a list of authors.
- Should the draft be deleted once it is published?
Annotations
- Drafts can have their own annotations. We need to check whether publishing a draft can cause annotation merge conflicts. At publishing time, we may offer the user the choice to keep the draft annotations or not.
Naming Conventions
- Editing could be allowed, as a draft, on documents that don't follow the configured naming convention, while publishing the draft remains blocked until the name is fixed.
Open Questions
- Should a draft have the same reference as its original document?
- Should the new database column only identify the draft kind (WYSIWYG, wiki, etc.), or a more generic document type that also covers other special documents such as deleted documents?
- Can draft attachments conflict with the original document's attachments in the file system attachment store, and how do we avoid duplicating attachment binaries?
- Are document revisions linked to their document by id or by reference?
- Do we need a new Solr entity type (e.g. DOCUMENT_DRAFT), and then new types for every entity that can be bound to a draft?
- Should draft events be new event types (e.g. DraftDocumentUpdated), or existing document events carrying an isDraft flag?
- Should it be possible to export only a draft to XAR, and what would importing it mean when the original document doesn't exist?
- Does PDF export need to support drafts?
- Should publishing squash the draft revisions, and if so, how do we preserve all the authors (e.g. multiple authors per revision)?
- Should the draft be deleted after it is published?
- Can publishing a draft cause annotation merge conflicts, and should the user choose whether to keep the draft annotations?
- How does the Inline Form edit mode handle fields that use different editors, each requiring its own draft?
Mohamed Boussaa
Simon Urli
Marius Dumitru Florea