Extensible API for accessing structured data

Last modified by Paul Libbrecht on 2020/01/28 14:36

 Feature
 Completed

Description

This API has been implemented as experimental prototypes within the UCF project:

Goal

Support a high-level API for accessing data from both server and client-side. This is a need that starts to appear in several research projects. Since this is interesting for XWiki as project we put this design idea here for discussion.

High-level requirements

  • There must be a unified way of using the API from Javascript, Velocity or Groovy, i.e., an homogeneous method names and signature.
  • A "general" API for storing and retrieving elements of the application domain must always be present, i.e., a set of standard methods that are used for performing CRUD operations.
  • The application editor must be able to extend the "general" API with specific business logic that is specific to the application.
  • The API - the "general" and the "extended" ones - should be accessible also via HTTP (i.e., it must be exposed also as a REST API)
  • The API should be easy to use for people writing extensions.

Use Cases

  • uc1 - Get the last 5 objects of the current application (XClass), sorted by a specific field.
  • uc2 - Get objects of a given application (we know the name of it) using an advanced query
  • uc3 - Create a new object of my application's type and store it
  • uc4 - Get an object, alter a field then store it
  • uc5 - medium priority - Add functions to an app, this implies the "app" object is not purely data.
  • uc6 - Understand what are the field values in an application item.
  • uc7 - low priority - Get a historical version of an item
  • uc8 - Integrate smoothly with AngularJS - this means the javascript representation of the objects should be iterable as basic map
  • uc9 - low priority - Change the content and title of an XWikiDocument without any AWM project

General API

This API is used to have a generic CRUD interface with the application, and must be automatically available for every AWM application.

EntryPoint

  • Velocity: $services.xapp
  • Groovy: services.xapp
  • Javascript: require(['xapp'], function (XApp) {  });

xapp.getApp

 Parameters 
  • id: The application id (e.g., the one used in AWM)
 Returns  An object containing all the methods for accessing applications data (see next sections)

xapp.getCurrent

 Parameters  (none)
 Returns  An object containing all the methods for accessing applications data (see next sections)
 Remarks  xapp.current and xapp.getCurrent() should both work in javascript and in velocity.

application.getItem

 Parameters 
  • id The item id
 Returns  An object sutable for storage using storeItem
 REST binding 
  • Request: GET .../{applicationId}/items
    • options parameters can be individually specified as query string paratemeters (e.g. .../{applicationId}/items?limit=42)
  • Response: A JSON serialization of the return value + appropriate status code
 Remarks  In the event that an object does not exist, an empty "new" object should be returned, matching the behavior of XWiki.getDocument()

application.getItems

 Parameters 
  • options: A map containing a set of options that can be used for customizing the items fetching. Some of them could be:
    • offset: The offset from where to start fetching the items (pagination)
    • limit: The number of items to be returned (pagination)
    • query: A kind of simple query specification for filtering
 Returns 

An array of maps, each one containing the following fields:

  • id The item id
  • data The serialized item data using a standard schema for XWikiClasses (see next section)
 REST binding 
  • Request: GET .../{applicationId}/items
    • options parameters can be individually specified as query string paratemeters (e.g. .../{applicationId}/items?limit=42)
  • Response: A JSON serialization of the return value + appropriate status code
 Remarks 
  • This will likely prove to be a difficult API function to implement and development here needs to be monitored.

application.storeItem

 Parameters item the item to be stored
 Returns  An error if there is any, otherwise null/undefined.
 REST binding 
  • Request: PUT .../{applicationId}/items/{itemId}
  • Request body: A JSON serialization of the item data using a standard schema for XWikiClasses (see next section)
  • Response: A JSON serialization of the return value + appropriate status code

application.deleteItem

 Parameters 
  • id The item id
 Returns 
  • In case of success: An empty map
  • In case of error: A map containing the following fields:
    • error The error description
 REST binding 
  • Request: DELETE .../{applicationId}/items/{itemId}
    • options parameters can be individually specified as query string paratemeters (e.g. .../{applicationId}/items?limit=42)
  • Response: A JSON serialization of the return value + appropriate status code

application.getSchema

 Parameters  None
 Returns 

A map with a key for every attribute that define the application item. The value associated to each key is a map that describe the type of the corresponding attribute.

 REST binding 
  • Request: GET .../{applicationId}/schema
  • Response: A JSON serialization of the return value + appropriate status code
 Remarks  The way for describing the schema migh be tricky. This is however needed in order to retrieve important information about the item data (e.g., which values are admissible in a static list, what are the format constraints, and so on)

Serialization

Schema serialization

This can be a JSON/Map version that mimicks what is already defined in the REST API (e.g., http://localhost:8080/xwiki/rest/wikis/xwiki/classes/XWiki.XWikiUsers)

In particlar this serialization should provide for every property that is defined in a class, all the information that can be relevant for a client. An important aspect, for example, is to communicate to clients what are the admissible values for list properties for validation purposes. Database lists could pose an issue here, since the set of admissible values is dynamic.

{
 'name' : {
   'type': 'String'
  },
 'genre' : {
   'type': 'Enum',
   'values': ['action', 'drama', 'scifi']
  },
 'releaseDate' : {
   'type': 'Timestamp',
  }
}

Item data serialization

This can be a JSON/Map that associate every item property to its desired value. The schema gives information about what kind of types must be used as values. We will leave it to the implementor to determine how best to represent the items in Velocity, Groovy and Javascript with the constraint that they must provide toString() methods which self-serialize to readable JSON and that in Javascript, the item's elements must be enumerable and accessable such that it can be easily integrated with AngularJS (as if they were plain JSON objects).

{
 'name': 'The Imitation Game',
 'genre': 'drama',
 'releaseDate': 1419462000000
}

Examples of API usage

Velocity


#set($app = $services.xapp.current)

## Get some items
#set($movies = $app.getItems({"limit": 50})
#foreach($movie in $movies)
| $movie.name | $movie.genre | $datetool.toDate($movie.releaseDate)
#end

## Create a new item or alter an existing one, the semantics are identical
#set($foo = $app.getItem("foo"))
$foo.set('genre', 'drama')
$foo.set('releaseDate', $datetool.getSystemTime())
$app.storeItem($foo)

Javascript

require(['xapp'], function (XApp) {
 var app = XApp.current;

 /* Asynchronous api is identical to nodejs */
  app.getItems({limit: 50}, function (err, result) {
   if (err) { throw err; } // Or handle somehow else...
   for(var i = 0; i < result.length; i++) {
     //Do something with result[i].data.name, etc.
   }
  });

  app.getItem("foo", function (err, foo) {
   if (err) { throw err; }
    foo.set('genre', 'drama');
    foo.set('releaseDate', (new Date()).getTime());
    app.storeItem(foo, function (err) {
       if (err) { console.log('something went wrong ! ' + err.stack); }
    });
  });
});

Open issues

Adding extended API methods

Extended API methods should be used to provide clients with specific ways for getting information related to the application domain. These methods can perform very specific tasks, retrieve information using complex queries and return it in more convenient formats (e.g., movies.getMovieBudgetForYear(2014))

There are several issues linked to this:

  • How to allow application developer to declare these extended API methods, and provide their implementation.
  • How to derive the API object that can be used by clients in the different programming environments.
  • How to derive the REST API associated to it so that Javascript Client can access the server API

A simple approach is that the developer writes everything (i.e., the Groovy code that is published as an XWiki service accessible via Groovy and Velocity scripts, the JavaScript Code that will call the REST API, and the Groovy code that will handle the REST endpoint)

This sounds like a lot of work. Maybe by declaratively defining the structure of the API (i.e., the parameters, and the return values) a subset of the code could be automatically generated. For example, a developer would provide just the implementation of the Groovy method - using some constraints - and when published the system will automatically create the corresponding XWiki Service, the REST Endpoint and the Javascript for interacting with it. However this sound like a task that could take a lot of time.

Javascript asynchronous semantics

Javascript implementation (which is a wrapper to the REST API) must be asynchronous. This can influence a bit the way things are implemented.


 


Tags:
Created by Fabio Mancinelli on 2015/02/11 10:13
    

Get Connected