Extensible API for accessing structured data (Proposal.ExtensibleAPIforaccessingstructuredata)

1

This API has been implemented as experimental prototypes within the [[UCF>>http://www.ubiquitus-content-framework.fr/]] project:

2

* on the server: see [[Example code in Velocity>>http://extensions.xwiki.org/xwiki/bin/view/Extension/Structured+data+access+API]]

3

* on the client: see [[Example code in JavaScript>>http://extensions.xwiki.org/xwiki/bin/view/Extension/Structured+data+access+JS+API]]

= Goal =

Support a high-level API for accessing data from both server and client-side. This is a need that starts to appear in several research projects. Since this is interesting for XWiki as project we put this design idea here for discussion.

8

9

== High-level requirements ==

10

11

* There must be a unified way of using the API from Javascript, Velocity or Groovy, i.e., an homogeneous method names and signature.

12

* A "general" API for storing and retrieving elements of the application domain must always be present, i.e., a set of standard methods that are used for performing CRUD operations.

13

* The application editor must be able to extend the "general" API with specific business logic that is specific to the application.

14

* The API - the "general" and the "extended" ones - should be accessible also via HTTP (i.e., it must be exposed also as a REST API)

15

* The API should be easy to use for people writing extensions.

== Use Cases ==

* uc1 - Get the last 5 objects of the current application (XClass), sorted by a specific field.

20

* uc2 - Get objects of a given application (we know the name of it) using an advanced query

21

* uc3 - Create a new object of my application's type and store it

22

* uc4 - Get an object, alter a field then store it

23

* uc5 - **medium priority** - Add functions to an app, this implies the "app" object is not purely data.

24

* uc6 - Understand what are the field values in an application item.

25

* uc7 - **low priority** - Get a historical version of an item

26

* uc8 - Integrate smoothly with AngularJS - this means the javascript representation of the objects should be iterable as basic map

27

* uc9 - **low priority** - Change the content and title of an XWikiDocument without any AWM project

== General API ==

This API is used to have a generic CRUD interface with the application, and must be automatically available for every AWM application.

=== EntryPoint ===

* Velocity: {{code language="none"}}$services.xapp{{/code}}

36

* Groovy: {{code language="java"}}services.xapp{{/code}}

37

* Javascript: {{code language="javascript"}}require(['xapp'], function (XApp) { });{{/code}}

38

39

=== **##xapp.getApp##** ===

40

41

|= Parameters |(((

42

* ##id##: The application id (e.g., the one used in AWM)

43

)))

44

|= Returns | An object containing all the methods for accessing applications data (see next sections)

----

=== **##xapp.getCurrent##** ===

49

50

|= Parameters | (none)

51

|= Returns | An object containing all the methods for accessing applications data (see next sections)

52

|= Remarks | xapp.current and xapp.getCurrent() should both work in javascript and in velocity.

----

=== **##application.getItem##** ===

|= Parameters |(((

* ##id## The item id

)))

|= Returns | An object sutable for storage using ##storeItem##

62

|= REST binding |(((

63

* **Request:** ##GET .../{applicationId}/items##

64

** ##options## parameters can be individually specified as query string paratemeters (e.g. ##.../{applicationId}/items?limit=42##)

65

* **Response:** A JSON serialization of the return value + appropriate status code

66

)))

67

|= Remarks | In the event that an object does not exist, an empty "new" object should be returned, matching the behavior of XWiki.getDocument()

----

=== **##application.getItems##** ===

72

73

|= Parameters |(((

74

* ##options##: A map containing a set of options that can be used for customizing the items fetching. Some of them could be:

75

** ##offset##: The offset from where to start fetching the items (pagination)

76

** ##limit##: The number of items to be returned (pagination)

77

** ##query##: A kind of simple query specification for filtering

78

)))

79

|= Returns |(((

80

An array of maps, each one containing the following fields:

81

82

* ##id## The item id

83

* ##data## The serialized item data using a standard schema for XWikiClasses (see next section)

84

)))

85

|= REST binding |(((

86

* **Request:** ##GET .../{applicationId}/items##

87

** ##options## parameters can be individually specified as query string paratemeters (e.g. ##.../{applicationId}/items?limit=42##)

88

* **Response:** A JSON serialization of the return value + appropriate status code

89

)))

90

|= Remarks |(((

91

* This will likely prove to be a difficult API function to implement and development here needs to be monitored.

)))

----

=== **##application.storeItem##** ===

97

98

|= Parameters |##item## the item to be stored

99

|= Returns | An error if there is any, otherwise null/undefined.

100

|= REST binding |(((

101

* **Request:** ##PUT .../{applicationId}/items/{itemId}##

102

* **Request body:** A JSON serialization of the item data using a standard schema for XWikiClasses (see next section)

103

* **Response:** A JSON serialization of the return value + appropriate status code

)))

----

=== **##application.deleteItem##** ===

|= Parameters |(((

* ##id## The item id

)))

|= Returns |(((

* In case of success: An empty map

115

* In case of error: A map containing the following fields:

116

** ##error## The error description

117

)))

118

|= REST binding |(((

119

* **Request:** ##DELETE .../{applicationId}/items/{itemId}##

120

** ##options## parameters can be individually specified as query string paratemeters (e.g. ##.../{applicationId}/items?limit=42##)

121

* **Response:** A JSON serialization of the return value + appropriate status code

)))

----

=== **##application.getSchema##** ===

|= Parameters | None

|= Returns |(((

A map with a key for every attribute that define the application item. The value associated to each key is a map that describe the type of the corresponding attribute.

131

)))

132

|= REST binding |(((

133

* **Request:** ##GET .../{applicationId}/schema##

134

* **Response:** A JSON serialization of the return value + appropriate status code

135

136

|= Remarks | The way for describing the schema migh be tricky. This is however needed in order to retrieve important information about the item data (e.g., which values are admissible in a static list, what are the format constraints, and so on)

)))

== Serialization ==

=== Schema serialization ===

142

143

This can be a JSON/Map version that mimicks what is already defined in the REST API (e.g., http://localhost:8080/xwiki/rest/wikis/xwiki/classes/XWiki.XWikiUsers)

144

145

In particlar this serialization should provide for every property that is defined in a class, all the information that can be relevant for a client. An important aspect, for example, is to communicate to clients what are the admissible values for list properties for validation purposes. Database lists could pose an issue here, since the set of admissible values is dynamic.

{

'name' : {

'type': 'String'

},

'genre' : {

'type': 'Enum',

'values': ['action', 'drama', 'scifi']

},

'releaseDate' : {

'type': 'Timestamp',

}

}

=== Item data serialization ===

163

164

This can be a JSON/Map that associate every item property to its desired value. The schema gives information about what kind of types must be used as values. We will leave it to the implementor to determine how best to represent the items in Velocity, Groovy and Javascript with the constraint that they must provide ##toString()## methods which self-serialize to readable JSON and that in Javascript, the item's elements must be enumerable and accessable such that it can be easily integrated with AngularJS (as if they were plain JSON objects).

{

'name': 'The Imitation Game',

169

'genre': 'drama',

170

'releaseDate': 1419462000000

}

== Examples of API usage ==

=== Velocity ===

#set($app = $services.xapp.current)

181

182

## Get some items

183

#set($movies = $app.getItems({"limit": 50})

184

#foreach($movie in $movies)

185

| $movie.name | $movie.genre | $datetool.toDate($movie.releaseDate)

186

#end

187

188

## Create a new item or alter an existing one, the semantics are identical

189

#set($foo = $app.getItem("foo"))

190

$foo.set('genre', 'drama')

191

$foo.set('releaseDate', $datetool.getSystemTime())

$app.storeItem($foo)

=== Javascript ===

require(['xapp'], function (XApp) {

200

var app = XApp.current;

201

202

/* Asynchronous api is identical to nodejs */

203

app.getItems({limit: 50}, function (err, result) {

204

if (err) { throw err; } // Or handle somehow else...

205

for(var i = 0; i < result.length; i++) {

206

//Do something with result[i].data.name, etc.

}

});

app.getItem("foo", function (err, foo) {

211

if (err) { throw err; }

212

foo.set('genre', 'drama');

213

foo.set('releaseDate', (new Date()).getTime());

214

app.storeItem(foo, function (err) {

215

if (err) { console.log('something went wrong ! ' + err.stack); }

});

});

});

== Open issues ==

== Adding extended API methods ==

225

226

Extended API methods should be used to provide clients with specific ways for getting information related to the application domain. These methods can perform very specific tasks, retrieve information using complex queries and return it in more convenient formats (e.g., movies.getMovieBudgetForYear(2014))

227

228

There are several issues linked to this:

229

230

* How to allow application developer to declare these extended API methods, and provide their implementation.

231

* How to derive the API object that can be used by clients in the different programming environments.

232

* How to derive the REST API associated to it so that Javascript Client can access the server API

233

234

A simple approach is that the developer writes everything (i.e., the Groovy code that is published as an XWiki service accessible via Groovy and Velocity scripts, the JavaScript Code that will call the REST API, and the Groovy code that will handle the REST endpoint)

235

236

This sounds like a lot of work. Maybe by declaratively defining the structure of the API (i.e., the parameters, and the return values) a subset of the code could be automatically generated. For example, a developer would provide just the implementation of the Groovy method - using some constraints - and when published the system will automatically create the corresponding XWiki Service, the REST Endpoint and the Javascript for interacting with it. However this sound like a task that could take a lot of time.

237

238

== Javascript asynchronous semantics ==

239

240

Javascript implementation (which is a wrapper to the REST API) must be asynchronous. This can influence a bit the way things are implemented.