Extensible API for accessing structured data
Description
This API has been implemented as experimental prototypes within the UCF project:
- on the server: see Example code in Velocity
- on the client: see Example code in JavaScript
Goal
Support a high-level API for accessing data from both server and client-side. This is a need that starts to appear in several research projects. Since this is interesting for XWiki as project we put this design idea here for discussion.
High-level requirements
- There must be a unified way of using the API from Javascript, Velocity or Groovy, i.e., an homogeneous method names and signature.
- A "general" API for storing and retrieving elements of the application domain must always be present, i.e., a set of standard methods that are used for performing CRUD operations.
- The application editor must be able to extend the "general" API with specific business logic that is specific to the application.
- The API - the "general" and the "extended" ones - should be accessible also via HTTP (i.e., it must be exposed also as a REST API)
- The API should be easy to use for people writing extensions.
Use Cases
- uc1 - Get the last 5 objects of the current application (XClass), sorted by a specific field.
- uc2 - Get objects of a given application (we know the name of it) using an advanced query
- uc3 - Create a new object of my application's type and store it
- uc4 - Get an object, alter a field then store it
- uc5 - medium priority - Add functions to an app, this implies the "app" object is not purely data.
- uc6 - Understand what are the field values in an application item.
- uc7 - low priority - Get a historical version of an item
- uc8 - Integrate smoothly with AngularJS - this means the javascript representation of the objects should be iterable as basic map
- uc9 - low priority - Change the content and title of an XWikiDocument without any AWM project
General API
This API is used to have a generic CRUD interface with the application, and must be automatically available for every AWM application.
EntryPoint
- Velocity: $services.xapp
- Groovy: services.xapp
- Javascript: require(['xapp'], function (XApp) { });
xapp.getApp
Parameters |
|
---|---|
Returns | An object containing all the methods for accessing applications data (see next sections) |
xapp.getCurrent
Parameters | (none) |
---|---|
Returns | An object containing all the methods for accessing applications data (see next sections) |
Remarks | xapp.current and xapp.getCurrent() should both work in javascript and in velocity. |
application.getItem
Parameters |
|
---|---|
Returns | An object sutable for storage using storeItem |
REST binding |
|
Remarks | In the event that an object does not exist, an empty "new" object should be returned, matching the behavior of XWiki.getDocument() |
application.getItems
Parameters |
|
---|---|
Returns | An array of maps, each one containing the following fields:
|
REST binding |
|
Remarks |
|
application.storeItem
Parameters | item the item to be stored |
---|---|
Returns | An error if there is any, otherwise null/undefined. |
REST binding |
|
application.deleteItem
Parameters |
|
---|---|
Returns |
|
REST binding |
|
application.getSchema
Parameters | None | ||
---|---|---|---|
Returns | A map with a key for every attribute that define the application item. The value associated to each key is a map that describe the type of the corresponding attribute. | ||
REST binding |
|
Serialization
Schema serialization
This can be a JSON/Map version that mimicks what is already defined in the REST API (e.g., http://localhost:8080/xwiki/rest/wikis/xwiki/classes/XWiki.XWikiUsers)
In particlar this serialization should provide for every property that is defined in a class, all the information that can be relevant for a client. An important aspect, for example, is to communicate to clients what are the admissible values for list properties for validation purposes. Database lists could pose an issue here, since the set of admissible values is dynamic.
'name' : {
'type': 'String'
},
'genre' : {
'type': 'Enum',
'values': ['action', 'drama', 'scifi']
},
'releaseDate' : {
'type': 'Timestamp',
}
}
Item data serialization
This can be a JSON/Map that associate every item property to its desired value. The schema gives information about what kind of types must be used as values. We will leave it to the implementor to determine how best to represent the items in Velocity, Groovy and Javascript with the constraint that they must provide toString() methods which self-serialize to readable JSON and that in Javascript, the item's elements must be enumerable and accessable such that it can be easily integrated with AngularJS (as if they were plain JSON objects).
'name': 'The Imitation Game',
'genre': 'drama',
'releaseDate': 1419462000000
}
Examples of API usage
Velocity
#set($app = $services.xapp.current)
## Get some items
#set($movies = $app.getItems({"limit": 50})
#foreach($movie in $movies)
| $movie.name | $movie.genre | $datetool.toDate($movie.releaseDate)
#end
## Create a new item or alter an existing one, the semantics are identical
#set($foo = $app.getItem("foo"))
$foo.set('genre', 'drama')
$foo.set('releaseDate', $datetool.getSystemTime())
$app.storeItem($foo)
Javascript
var app = XApp.current;
/* Asynchronous api is identical to nodejs */
app.getItems({limit: 50}, function (err, result) {
if (err) { throw err; } // Or handle somehow else...
for(var i = 0; i < result.length; i++) {
//Do something with result[i].data.name, etc.
}
});
app.getItem("foo", function (err, foo) {
if (err) { throw err; }
foo.set('genre', 'drama');
foo.set('releaseDate', (new Date()).getTime());
app.storeItem(foo, function (err) {
if (err) { console.log('something went wrong ! ' + err.stack); }
});
});
});
Open issues
Adding extended API methods
Extended API methods should be used to provide clients with specific ways for getting information related to the application domain. These methods can perform very specific tasks, retrieve information using complex queries and return it in more convenient formats (e.g., movies.getMovieBudgetForYear(2014))
There are several issues linked to this:
- How to allow application developer to declare these extended API methods, and provide their implementation.
- How to derive the API object that can be used by clients in the different programming environments.
- How to derive the REST API associated to it so that Javascript Client can access the server API
A simple approach is that the developer writes everything (i.e., the Groovy code that is published as an XWiki service accessible via Groovy and Velocity scripts, the JavaScript Code that will call the REST API, and the Groovy code that will handle the REST endpoint)
This sounds like a lot of work. Maybe by declaratively defining the structure of the API (i.e., the parameters, and the return values) a subset of the code could be automatically generated. For example, a developer would provide just the implementation of the Groovy method - using some constraints - and when published the system will automatically create the corresponding XWiki Service, the REST Endpoint and the Javascript for interacting with it. However this sound like a task that could take a lot of time.
Javascript asynchronous semantics
Javascript implementation (which is a wrapper to the REST API) must be asynchronous. This can influence a bit the way things are implemented.