XWiki Patch Service
Description
This page contains ideas related to the design and implementation of a P2P replication engine for XWiki, in relation with the XWiki Concerto research project in which INRIA, ENST, EISTI and Mandriva are teaming up until 2009.
The paragraphs below proposes to manage changes through the generation of patches using a process inspired by distributed revision control systems. The page contains a proposal for the following elements:
- the structure of patches that will be exchanged between XWiki core and the XWiki replication service
- an XWiki API for dealing with patches
Component diagram
The diagram below represents the PatchService and the ReplicationService in the context of existing XWiki services.
Data structures proposal
Patch
Defintion: a patch contains a sequence of operations sorted chronologically. There is no creation operation: a document is considered as created as soon as an operation is set on the document. A patch contains a series of metadata: short description, description, author id, date, hash. A patch can be signed by a key, and can be encrypted.
PatchId getId()
List<Operation> getOperations()
/** short description of the patch */
String getDescription()
/** authorId consists of: host id, xwiki id, author login.
* Example: xwiki312124:xwiki:XWiki.John\_Doe */
String getAuthorId()
long getTime()
String getHash()
String toXML()
A patch can be serialized in XML, as in the example below.
document_id="myxwiki.mydocument.id"
author="author_id"
date="20070823446657808">
<?xml version="1.0" encoding="UTF-16"?>
<patch>
<operation type="content-insert">
<text> as/<>"'&d </text>
<position after="after " before="before" column="3" row="2"/>
</operation>
<operation type="content-delete">
<text>Here </text>
<position column="0" row="0"/>
</operation>
<operation type="property-set">
<property name="prop"/>
<text>val</text>
</operation>
<operation type="class-property-add">
<class name="XWiki.Class">
<property name="field" type="com.xpn.xwiki.objects.classes.BooleanClass">
<property name="prettyName" value="The field"/>
<property name="displayType" value="yesno"/>
<property name="unmodifiable" value="0"/>
<property name="name" value="field"/>
<property name="displayFormType" value="select"/>
<property name="number" value="1"/>
</property>
</class>
</operation>
<operation type="class-property-change">
<class name="XWiki.Class">
<property name="field" type="com.xpn.xwiki.objects.classes.BooleanClass">
<property name="prettyName" value="The field"/>
<property name="displayType" value="yesno"/>
<property name="unmodifiable" value="0"/>
<property name="name" value="field"/>
<property name="displayFormType" value="select"/>
<property name="number" value="1"/>
</property>
</class>
</operation>
<operation type="class-property-delete">
<class name="XWiki.Class">
<property name="prop1"/>
</class>
</operation>
<operation type="object-add">
<object type="XWiki.Class"/>
</operation>
<operation type="object-delete">
<object number="2" type="XWiki.Class"/>
</operation>
<operation type="object-property-set">
<object number="2" type="XWiki.Class">
<property name="propertyName"/>
</object>
<text>value</text>
</operation>
<operation type="object-property-textinsert">
<object number="0" type="XWiki.Class">
<property name="property"/>
</object>
<text>inserted text</text>
<position column="0" row="2"/>
</operation>
<operation type="object-property-textdelete">
<object number="0" type="XWiki.Class">
<property name="property"/>
</object>
<text>deleted text</text>
<position column="0" row="2"/>
</operation>
<operation type="attachment-add">
<attachment author="XWiki.Me" name="file">[content in Base64 encoding]</attachment>
</operation>
<operation type="attachment-set">
<attachment author="XWiki.Me" name="file">aGVsbG8=</attachment>
</operation>
<operation type="attachment-delete">
<attachment name="file"/>
</operation>
</patch>
PatchId
Definition: a PatchId identifies a patch. It consists of the ID of the peer where the patch was generated + a logical clock stamp + the ID of the document the patch relates to.
getHostId()
//logical time
getLogicalTime()
//host time of the operation
getTime()
getDocumentId()
Operation
Definition: when hitting the XWiki "Save" button for a document, a set of changes is sent (change of the content, of the document's metadata, of the attached objects etc.) to the XWiki kernel. This set of changes is represented as a set of operations. An operation consists of an atomic XWiki change such as: addition of an object, update of an object's property value etc. Operations that can be applied to a document consist of the following:
- addition of characters at one or different positions
- removal of characters
- addition of metadata to a document: name, language, default_language, translation, date, creation_date, author, creator, parent, title, default_template, minor_edit, comment
- addition or update of objects attached to a documents
- removal of objects
- addition of attachments
- removeal of attachments
- addition of XWiki classes
- removal of XWiki classes
{
/* document's content */
boolean insert(String text, Position position);
boolean delete(String text, Position position);
/* Operations affeting the document metadata (name, author, language etc. */
boolean setProperty(String property, String value);
/* Operations affeting the XObjectDefinition stored in a document */
boolean createType(String className, String propertyName, String propertyType, Map properties);
boolean modifyType(String className, String propertyName, Map properties);
boolean deleteType(String className, String propertyName);
/* Operations affeting the document's objects */
boolean addObject(String objectClass);
boolean deleteObject(String objectClass, int index);
boolean setObjectProperty(String objectClass, int index, String propertyName, String value);
boolean insertInProperty(String objectClass, int index, String property, String text,
Position position);
boolean deleteFromProperty(String objectClass, int index, String property, String text,
Position position);
/* Operations affeting the attachments */
boolean addAttachment(InputStream is, String filename, String author);
boolean setAttachment(InputStream is, String filename, String author);
boolean deleteAttachment(String name);
/* Operation metadata */
void setType(String type);
}
public interface Operation extends XmlSerializable
{
String getType();
/* Apply the current operation on a document */
boolean apply(XWikiDocument doc, XWikiContext context);
}
public interface XmlSerializable
{
/* Element and Doc are W3C DOM L3 interfaces */
Element toXML(Document doc);
void fromXML(Element e);
}
XWikiPatchService API
The XWiki PatchService generates and applies patches, and provides other services with patches. It will be notified by XWiki kernel services of each change. The service can be accessed both natively through Java calls or through a REST API. It can generate a patch from two XWiki document versions. It stores patches in a database and makes them available through Java calls or REST calls. We may consider using a type that is more general than XWikiDocument for making the service as general as possible.
/** called by XWiki engine for logging a patch */
void logPatch(Patch p)
/** called by XWiki engine for generating a patch from two document versions */
Patch generatePatch(XWikiDocument version1, XWikiDocument version2)
/** returns a specific patch */
Patch getPatch(PatchId id)
/** retrieves the set of patches that occurred on this host after the patch
* "from" was applied. This method is inclusive: the from Patch is in the
* returned list. */
Iterator<Patch> getUpdatesFrom(PatchId from)
Iterator<Patch> getAllPatches()
/** This method is inclusive: from and to patches are included */
Iterator<Patch> getDelta(PatchId fromPatch, PatchId toPatch)
/** do we need to introduce a concept of dependencies between patches? */
Iterator<Patch> getDependencies(PatchId)
/** a patch can be signed using a key, that is registered beforehand to the service:
* see the registerKey method. The Result object contains information about what
* happened after the apply tentative. */
Result applyPatch(Patch p, PatchId latestPatch)
/** registers a key in the key ring so that the service can verify that the
* patched it receives are signed by a known key. */
registerKey(Key k)
/** unregisters a key */
unregisterKey(Key k)
XWikiPatchService - REST API
Th XWikiPatchService will be accessible through the following REST API:
| Description | URL | Parameters | HTTP method | Response |
|---|---|---|---|---|
| Retrieval of the XML log of all patches applied since a given patch | http://xwikiconcerto1/patchservice/updates/id | id | GET | XML file with all patches' XML |
| Retrieve all patches from the origin | http://xwikiconcerto1/patches/all | GET | ||
| Get a given patch based on its id | http://xwikiconcerto1/patches/id | patch id | GET | XML representations of the patch |
| Retrieve delta between two patches (in XML) | http://xwikiconcerto1/patches/delta | patch_id_from patch_id_to | GET | |
| Apply patch request | http://xwikiconcerto1/patches/patch | patch=XML file, patch_id=latest patch id | PUT | result |
| Registration of a key | http://xwikiconcerto1/patches/key | key=key's value, key_name= | PUT | response |
| Get registered keys | http://xwikiconcerto1/patches/keys | GET | keys in XML |
Each XWiki peer will provide the list of the latest patches that have been generated locally through an RSS feed, with their description and a link to the XML content of each patch, similarly as what darcs proposes (see for example this example)
Implementation note related to the applyPatch method: when the replication engine requests the application of a patch, XWiki should make sure that all patches produced locally have been consumed by the replication service. If they have, the patch is applied. If not, XWiki has to let the replication service know that it should consume the pending patches and then issue the request again. The method "applyPatch" should receive two parameters: the Patch to be applied and the PatchId of the latest patch handled. It has to return a result that let the replication service know if the patch was applied or not. If it wasn't, the missing patches are to be retrieved via the method "getUpdatesFrom(PatchId)".
Scenario example
The schema below illustrates a replication scenario.
