Show last authors
1 This page lists the specifications of the new live table.
3 {{toc depth="3"/}}
5 = Functional Requirements =
7 R1: **Load**
9 * Load the data asynchronously (don't block the page load)
10 * Show a loading animation
12 R2: **Display**
14 * Display the data as a table (with visible table header, rows and columns)
15 * Responsive display on small screens
16 * Be able to mark columns that can be hidden when there's not enough horizontal space
17 * Display the data as a grid (card layout)
19 R3: **Pagination**
21 * Load only a limited set of rows (one page)
22 * Show the total number rows
23 * Be able to change the number of rows displayed per page (10, 15, etc.)
24 * Show page numbers vs. show only First / Next / Previous / Last links
26 R4: **Sort**
28 * Be able to sort the live table rows by a single column, both ascending and descending
29 * Be able to sort on multiple columns
30 * Indicate the sort column and the sort order (ascending / descending)
32 R5: **Filter**
34 * Be able to filter the live table rows by one or multiple columns
35 * Indicate the columns with active filters
36 * Have a single filter/search input vs. have a filter per column
37 * When filtering by multiple columns, use the **AND** operator (match all)
38 * Be able to specify multiple constraints for a single column, and allow to choose whether to apply all (AND) or any (OR)
39 * Be able to filter by exact value, partial value, prefix, suffix, less than, greater than
41 R6: **Selection**
43 * Be able to select live table entries one by one, even from different pages
44 * Be able to select all / none of the entries from the current page (loaded entries)
45 * Be able to select all / none of the entire entries (including entries that haven't been loaded yet)
47 R7: **Actions**
49 * Be able to perform actions that target a single row (e.g. view details, delete a row)
50 * Be able to perform **batch** actions that affect the current selection of rows
51 * Standard actions: view, edit, delete, copy, export
53 R8: **Bookmark**
55 * Be able to bookmark or share the current state of the live table (pagination, sort, filters, selection) with other users (through the URL)
56 * Be able to obtain the wiki syntax needed to recreate the current state of the live table (to copy & paste in another wiki page)
58 R9: **Synchronization**
60 * Be able to see in real-time the changes done by other users through the **same** live table (e.g. if another user deletes a row then update my live table to remove that row)
62 R10: **In-place edit**
64 * Be able to **edit** the live table data in-place
65 * Be able to **add** a new live table entry from the live table
67 = Architecture =
69 The following components are involved:
71 * A **Velocity macro** that loads the required JavaScript modules and CSS stylesheets and outputs the HTML placeholder
72 * A **wiki syntax macro** that calls the Velocity macro
73 * A generic/reusable **data source** that retrieves the data from wiki pages and their objects, and that can be extended or customized
74 * A (responsive) **stylesheet**
75 * Multiple JavaScript modules:
76 ** **Core**: looks for the HTML placeholder in the DOM and enhances it, then fetches the data and fills the table
77 ** **Filters**: separate modules that enhance the live table filters (e.g. help the user pick a date range to filter a date column)
78 ** **Displayers**: separate modules that handle the display of the data inside the live table (e.g. user displayer, date displayer, etc.)
79 ** **Synchronization**: the module that handles the live table synchronization (real-time updates)
81 Would be nice to also have a wizard to help the users generate the live table configuration.
83 Let's see how the listed components interact with each other:
85 * The user inserts the live table **wiki syntax macro** in a page.
86 * When the page is rendered the wiki syntax macro is executed (on the server).
87 * The wiki syntax macro then
88 ** reads the live table **configuration** (from its macro parameters, or from its macro content or from another source)
89 ** and calls the live table Velocity macro, passing the configuration.
90 * The **Velocity macro** (on the server)
91 ** includes the required JavaScript modules (core, filters, displayers, synchronization), depending on the live table configuration,
92 ** and the stylesheets,
93 ** then outputs the HTML placeholder, also passing the live table configuration for the client side using HTML data attributes
94 * The browser parses the page HTML and loads the JavaScript and CSS resources
95 * When the live table **core JavaScript module** is loaded
96 ** it looks for the HTML placeholder in the DOM and reads the live table configuration
97 ** enhances the HTML structure
98 ** makes and HTTP request to fetch the live table data (using the **data source** specified in the live table configuration)
99 *** (on the server, the data source receives the HTTP request and executes the necessary database queries to gather the data)
100 ** fires the events needed to trigger the initialization of the live table **filters** (which are handled by separate JavaScript modules)
101 ** fires the events needed to trigger the display of the live table data (based on the configuration of each column)
102 ** initializes the **real-time** session (synchronization)
104 {{info}}
105 **TODO**: Would be nice to have a diagram to view the interaction.
106 {{/info}}
108 Let's look at each component in detail.
110 == Wiki Syntax Macro ==
112 {{code language="none"}}
113 {{livetable ... /}}
114 {{/code}}
116 The wiki syntax macro is meant to be used by both **simple users** and advanced users so it needs to have a good balance between easy to understand parameters and more advanced parameters that have good default values. The live table settings could be read from:
118 * macro parameters and/or the macro content (= in-line configuration source)
119 * an external configuration source (e.g. from some dedicated objects attached to a wiki page)
120 * both, using the external configuration source as a fallback
122 Here's a list of common live table settings that the user should be able to specify when calling the wiki syntax macro:
124 * the **data source**, can be specified as:
125 ** the reference of a wiki page that holds a class definition (the default / generic data source is used)
126 ** the reference of a wiki page that implements a custom data source (could extend the default data source)
127 ** a URL for a custom data source
128 * the **list of columns**
129 * the columns to **sort** the live table on initially, including the sort **order** (ascending / descending)
130 * the maximum **number of rows** to display per page
131 * the **translation** key prefix
132 * **filter values**
133 ** the values used to pre-fill the live table (visible) filters
134 ** and the values passed to the data source in order to restrict the data set (hidden filters)
136 Based on this the wiki syntax macro could have the following parameters:
138 {{code language="none"}}
139 {{livetable
141 ## Optional parameters that map to the corresponding HTML attributes, helping developers customize the live table
142 ## styles and behavior.
143 id="foo"
144 class="bar"
146 ## Optional parameter that specifies the source of the live table data. The parameter value is a component hint. A
147 ## default implementation is provided (mapped to the current XWiki.LiveTableResults page).
148 source="default"
150 ## Optional source parameters, used mainly to apply hidden filters (i.e. filters that cannot be removed from the UI).
151 sourceParameters="className=XWiki.XWikiUsers&location=Users"
153 ## Optional comma-separated list of columns. The default list of columns is specified by the data source.
154 columns="one,two,three"
156 ## Optional comma-separated list of columns to sort on. Sort order can be specified.
157 sort="one,two:desc"
159 ## Optionally initialize the visible live table filters (the users can change these values from the UI).
160 filter="one=blue&three=pending"
162 ## Optionally limit the number of live table rows displayed.
163 limit="10"
165 ## Optional translation key prefix. Should be used only to overwrite the default labels.
166 translationPrefix="foo.livetable."
168 /}}
169 {{/code}}
171 Check [[this page>>Design.LiveTable20Macro]] for a related proposal. Note that column properties (e.g. whether the column is filterable or sortable) are not included in the list of macro parameters because they should be specified by the data source.
173 The wiki syntax macro can be used directly in wiki syntax (e.g. from the Wiki editor) but it can also be inserted from the WYSIWYG editor, using the Insert Macro wizard. We need to make sure the macro parameters are easy to set by simple users and for this we need to provide pickers (e.g. a page / xclass picker for the data source).
175 The wiki syntax macro serves as a wrapper (facade) for the Velocity macro, so it doesn't know and it doesn't care how the live table is implemented. See the [[Documents Macro>>extensions:Extension.Documents Macro]] for an example of such a wiki syntax macro (the code can be found in the ##XWiki.DocumentsMacro## wiki page).
177 == Velocity Macro ==
179 {{code language="none"}}
180 #livetable('foo' $columns $columnsProperties $options)
181 {{/code}}
183 This is meant to be used only by developers. All the needed information should be passed through macro parameters. The goal of this macro is to include the required JavaScript and CSS resources and to output the live table HTML placeholder, based on the provided live table configuration. It must not depend on wiki pages, because we must be able to use this macro on an empty wiki, from a Velocity template.
185 The Velocity macro is aware of the JavaScript widget used to implement the live table and the HTML output it generates might have to be specific to this widget (to meet its expectations). So changing the widget used to implement the live table will require updating the Velocity macro (but should not require an update of the wiki syntax macro or the data source). The Velocity macro is independent of the data source specified in the live table configuration.
187 See the existing [[live table Velocity macro>>extensions:Extension.Livetable Macro]] for more details.
189 == Data Source ==
191 We have to provide a default data source that retrieves data from wiki pages and the objects attached to them. This default data source is generic and can be parameterized (the users can pass configuration parameters through the live table configuration). Developers can extend the default data source or write their custom data sources. The data returned by the data source should include:
193 * the number of rows available on the server (needed for pagination)
194 * the offset and limit used (for pagination)
195 * a list of rows (the data to display on each table row)
196 ** something to identify the row (we need this for synchronization most probably)
197 ** access rights (whether the current user can edit, delete, etc. this row)
198 ** URLs of supported actions (edit, delete, copy, etc.)
199 ** the list of cells (the data to display on each cell)
200 *** additional information that may be needed to display the table cell (e.g. data type, pretty name, etc.)
202 The JSON currently looks like this:
204 {{code language="js"}}
205 {
206 "totalrows": 35,
207 "returnedrows": 15,
208 "offset": 1,
209 "rows": [
210 {
211 //
212 // Identify the row
213 //
214 "doc_wiki": "xwiki",
215 "doc_fullName": "Blog.BlogIntroduction",
217 //
218 // Actions & access rights
219 //
220 "doc_viewable": true,
221 "doc_url": "/xwiki/bin/view/Blog/BlogIntroduction",
223 "doc_hasadmin": true,
225 "doc_hasedit": true,
226 "doc_edit_url": "/xwiki/bin/edit/Blog/BlogIntroduction",
228 "doc_hasdelete": true,
229 "doc_delete_url": "/xwiki/bin/delete/Blog/BlogIntroduction",
231 "doc_hascopy": true,
232 "doc_copy_url": "/xwiki/bin/view/Blog/BlogIntroduction?xpage=copy",
234 "doc_hasrename": true,
235 "doc_rename_url": "/xwiki/bin/view/Blog/BlogIntroduction?xpage=rename&step=1",
237 "doc_hasrights": true,
238 "doc_rights_url": "/xwiki/bin/edit/Blog/BlogIntroduction?editor=rights",
240 //
241 // Cell values
242 //
243 "doc_title": "First blog post",
244 "category": "News",
245 "publishDate": 1243846800000
246 ...
247 },
248 ...
249 ]
250 }
251 {{/code}}
253 See the [[XWiki.LiveTableResults>>]] page which is our current default data source.
255 It's important that the data source doesn't depend on the JavaScript widget used to implement the live table. If necessary, we can write adapters on the JavaScript side to modify the JSON returned by the data source in order to meet the format expected by the JavaScript widget.
257 == The Live Table Widget ==
259 This component is responsible for displaying the live table. It reads the live table configuration from the DOM element being enhanced, fetches the live table data from the server and displays each row by calling the right cell displayer.
261 Basically this component is responsible for:
263 * loading the initial data
264 * reloading the data when the user uses the pagination, sorts, filters or deletes rows
265 * displaying the rows (delegates to the cell displayers)
266 * pagination, sorting, filtering, selection and row actions (e.g. delete)
267 * adding new rows
268 * editing cells
269 * re-ordering columns, removing / hiding columns, adding / showing columns
270 * URL hashing (to be able to bookmark the live table state)
272 This component needs to have at least 2 sub-components:
274 * a generic live table / grid widget; for this we should use an **existing** open-source JavaScript library with **WebJar** packaging
275 * the code that integrates the existing live table widget in XWiki
277 There are 2 ways in which the live table initialization is triggered:
279 * automatic, based on some CSS class, e.g. ##xwiki-livetable##
280 ** on page load
281 ** on ##xwiki:dom:updated## event
282 * manual, through a jQuery plugin:(((
283 {{code language="none"}}
284 $('selector').livetable({
285 // configuration
286 });
287 {{/code}}
288 )))
290 The live table widget configuration could look like this:
292 {{code language="js"}}
293 {
294 //
295 // Modifiable
296 //
298 // The list of available columns to choose from when adding a column, along with their descriptor.
299 // Creating a new column adds a new entry (descriptor) to this list.
300 columnDescriptors: [
301 {
302 // Used when displaying and filtering the live table rows.
303 id: 'doc.title',
305 // Displayed in the table header.
306 name: 'Title',
308 // Displayed on hover over the column name, if specified.
309 description: '...',
311 // Displayed before the column name, if specified.
312 icon: '...',
314 // The column type, chosen when creating the column. The live table widget uses this only to prefill the column
315 // descriptor. When the column is mapped to an xclass property this specifies the property type.
316 type: 'String',
318 // If not specified then the column is not sortable.
319 sortable: true,
321 // Used to compute the name of the RequireJS module that will be used to display/edit the values from this column.
322 // If not specified then the default 'text' displayer is used.
323 displayer: 'link',
325 // This is used only by the 'link' displayer (which receives the row data and the column descriptor).
326 linkType: '...',
328 // Used to compute the name of the RequireJS module that will be used to filter the values from this column.
329 // If this is not set or set to false then the column is not filterable.
330 filter: 'text',
332 // This is used only by the 'text' filter (which receives the column descriptor).
333 match: 'prefix',
335 // Optional CSS class name to add to the TH element.
336 styleName: '...'
337 }, {
338 id: 'birthdate',
339 name: 'Birthdate',
340 icon: '...',
341 sortable: true,
342 displayer: 'date-timeago',
343 filter: 'date-range'
344 }, {
345 ...
346 }, {
347 id: '_actions',
348 name: 'Actions',
350 displayer: 'actions',
351 // This is used only by the 'actions' displayer (which receives the row data and the column descriptor).
352 actions: [
353 {id: 'delete', label: 'Delete', icon: '...'},
354 ...
355 ]
356 }
357 ],
359 // The list of columns to display.
360 columns: ['doc.title', 'birthdate', ...],
362 // The list of columns to sort on. The sort order can be optionally specified.
363 sort: ['birthdate:desc'],
365 // The values used to filter the live table rows. The live table filters are prefilled with these values. When a new
366 // filter is applied this configuration is updated.
367 filterValues: {
368 birthdate: ['...', ...],
369 ...
370 },
372 //
373 // Read-only
374 //
376 // The list of known column types. If this is specified then allow the user to choose the column type when creating a
377 // column (and prefill the column descriptor with the settings from the column type).
378 columnTypes: [
379 {id: 'String', name: 'String', icon: '...', sortable: true, displayer: 'text', filter: 'text'},
380 {id: 'Date', name: 'Date', icon: '...', sortable: true, displayer: 'date', filter: 'date-range'},
381 ...
382 ],
384 // The list of known filters to choose from when creating a new column.
385 filters: ['text', 'date-range', 'user', ...],
387 // The list of known displayers to choose from when creating a new column
388 displayers: ['text', 'html', 'link', 'date', 'date-timeago', 'user', ...],
390 // Limit the number of rows displayed.
391 limit: 10,
393 // The maximum number of page links to display in the pagination.
394 maxPages: 10,
396 // Used to build the page size drop down.
397 pageSizeBounds: {min: 10, max: 100, step: 10},
399 // This is used to compute the name of the RequireJS module that will be used to get the live table data. If not
400 // specified, a default source is used.
401 source: '...',
403 // This is used by the live table source module to get the live table data.
404 url: '...'
405 }
406 {{/code}}
408 If we want to make this generic in order to handle any data representation (table, card, calendar, etc.) then the configuration could looks like this:
410 {{code language="js"}}
411 {
412 //
413 // The query
414 //
416 query: {
417 // The list of properties to fetch.
418 properties: ['title', 'year', ...],
420 source: {
421 // This is used to compute the name of the RequireJS module that will be used to get the data. If not specified, a
422 // default source is used.
423 id: '...',
425 // This is a parameter used by the specified source module.
426 url: '...'
427 },
429 // The constraints to apply on the property values. These are hidden constraints that the user cannot change.
430 hiddenFilters: {
431 year: ['...', ...],
432 ...
433 },
435 // The visible filter values that the user can change. The filters are prefilled with these values. When fetching
436 // the data this is merged with the hidden filters (see above).
437 filters: {
438 year: ['...', ...],
439 ...
440 },
442 // The list of properties to sort on. The sort order can be optionally specified.
443 sort: [{
444 property: 'birthdate',
445 descending: false
446 }, ...],
448 // Indicates where the current page starts.
449 offset: 0,
451 // The number of entries to fetch (the page size).
452 limit: 10
453 },
455 //
456 // The data
457 //
459 data: {
460 // The total number of entries available (on the server side).
461 count: 54,
463 entries: [
464 {
465 // property: value
466 title: 'Work from home',
467 year: 2020,
468 ...
469 },
470 ...
471 ],
472 },
474 //
475 // The meta data (used to control how we interact with the data)
476 //
478 meta: {
479 // Describes the properties that may appear in the data set. This determines the list of known (available)
480 // properties. Creating new properties, removing existing properties as well as editing the property descriptor
481 // should be done through this array.
482 propertyDescriptors: [
483 {
484 // Identifies the property that this descriptor corresponds to.
485 id: 'title',
487 // The property name. Could be displayed before the property value.
488 name: 'Title',
490 // Could be displayed when hovering the property name.
491 description: '...',
493 // Could be displayed before the property name, if specified.
494 icon: '...',
496 // The property type, selected when creating the property. It is used to prefill the property descriptor.
497 // Could be mapped to an xclass property type.
498 type: 'string',
500 // Whether the user can sort on this property or not. If not specified then the property is not sortable.
501 sortable: true,
503 // Displayer configuration.
504 displayer: {
505 // Used to compute the name of the RequireJS module that will be used to display/edit the values from this
506 // property. If not specified then the default 'text' displayer is used.
507 id: 'link',
509 // This is used only by the 'link' displayer (which receives the data entry and the property descriptor).
510 linkType: '...'
511 },
513 // Filter configuration.
514 filter: {
515 // Used to compute the name of the RequireJS module that will be used to filter the values from this property.
516 // If this is not set or set to false then the property is not filterable.
517 id: 'text',
519 // This is used only by the 'text' filter (which receives the property descriptor).
520 match: 'prefix'
521 },
523 // Optional CSS class name to add to the HTML element used to display this property.
524 styleName: '...'
525 }
526 ],
528 // The list of known property types. When creating a new property the user can select from this list and the
529 // property descriptor will be prefilled based on the selected property type.
530 propertyTypes: [
531 {id: 'string', name: 'String', icon: '...', sortable: true, displayer: {...}, filter: {...}},
532 ...
533 ],
535 // The list of known filters to choose from when editing the property descriptor.
536 filters: [
537 {id: 'text', ...},
538 {id: 'date-range', ...},
539 {id: 'suggest', ...},
540 ...
541 ],
543 // The list of known property displayers to choose from when editing the property descriptor.
544 displayers: [
545 {id: 'text', ...},
546 {id: 'html', ...},
547 {id: 'link', ...},
548 {id: 'actions', ...},
549 ...
550 ],
552 // Configure the pagination display.
553 pagination: {
554 // The maximum number of page links to display in the pagination.
555 maxShownPages: 10,
557 // Used to build the page size drop down that allows the user to change the number of entries displayed per page.
558 pageSizeBounds: {min: 10, max: 100, step: 10}
559 }
560 }
561 }
562 {{/code}}
564 The live table widget needs to trigger some events. See the current list of [[live table events>>xwiki:Documentation.DevGuide.FrontendResources.JavaScriptAPI.WebHome||anchor="HLivetableevents28livetable.js29"]] for details.
566 * when the live table is ready (initialized)
567 * whenever the live table rows are updated (due to pagination, sorting, filtering, etc.)
568 * after a row is displayed
569 * when a column is created, added, removed, edited or deleted
570 * when a row is created
572 The live table widget is currently implemented by [[livetable.js>>]].
574 == Filters ==
576 The filter JavaScript modules help the user pick the live table filter values. There are 2 types of (visual) live table filters:
578 * column filters: displayed below the column header, they filter the values from that column
579 * external filters: displayed outside the live table (e.g. before), they are not bound to a column and they filter the live table rows based on values that are not necessarily displayed
581 We need at least the following pickers for column filters:
583 * [[date range>>]]: allow the user to select a date range using for instance 2 calendars
584 * [[list with multiple selection>>]]: allow the user to filter the column by selecting one or multiple values from a limited list of known values
585 * [[suggest picker>>]]: help the user select a value from an unlimited list of values by suggesting the existing values
587 We need to support at least the following external filters:
589 * tag cloud: filters the live table rows based on the tags associated to wiki pages (when the live table rows represent wiki pages)
591 The live table widget shouldn't be aware of the filter pickers and external filters. The filter picker modules are included by the Velocity macro based on the live table configuration (e.g. the date range picker is included only if there is a date column) and they enhance the live table filters using the events triggered by the live table widget.
593 == Displayers ==
595 The displayer modules are responsible for displaying each cell, both when viewing and editing the live table data. They could be implemented as RequireJS modules and the live table configuration would specify which displayer module to use for each column.
597 {{code language="js"}}
598 define('xwiki-livetable-displayer-date', [...], function(...) {
599 return {
600 view: function(column, rowData, tableConfig) {
601 ...
602 return domElement;
603 },
604 edit: function(column, rowData, tableConfig) {
605 ...
606 return domElement;
607 }
608 }
609 });
610 {{/code}}
612 The following displayers are needed:
614 * **default**: simply display the value for view and a plain text input for edit
615 * **date**: support time-ago format and a date picker for edit
616 * **page**: link to page on view and page picker when editing
617 * **user**: user avatar and link to user profile on view and user picker on edit
619 == Synchronization ==
621 This module is responsible for updating the live table when the data is modified on the server. There are two types of synchronization that we can achieve:
623 * **Closed synchronization** (less complex)
624 ** Doesn't depend on the data source
625 ** Only the changes done within the real-time session are propagated (e.g. only the changes done through the live table)
626 * **Open synchronization** (more complex)
627 ** Depends on the data source (specific change listeners have to be written for each data source)
628 ** Any change done to the data source is propagated (e.g. the live table is updated when one of the pages that is listed is modified outside of the live table)
630 For the scope of this proposal we will focus only on **closed synchronization**.
632 = User Cases =
634 The brand new feature of the Livetable 2.0 is real-time editing.
636 The entries can be directly edited from the Livetable fields. The modified objects are updated on the server, and in all the other instances of the Livetables displaying this data, without needed to reload the page.
638 If two user are editing an entry at the same time, existing conflicts should be resolved automatically.
640 User Case 1:
642 (% class="box" %)
643 (((
644 //user1// and //user2// are viewing the results of the same Livetable
646 * //user1// modify a property of an object directly from the Livetable
647 * the object is modified on the server
648 * //user2// gets its own Livetable instantly updated
649 )))
651 Similarly, if the entry is modified from its edit page, it has to update the opened Livetables too.
653 User Case 2:
655 (% class="box" %)
656 (((
657 //user1// is browsing a Livetable, while user2 is currently editing an object in its edit page
659 * //user1// modify a property in the same object directly from the Livetable
660 * the object is modified on the server
661 * //user2// do not get the update about the modification, as the page is static
662 * //user2// modify a property in the object and save the changes
663 * the object is modified on the server
664 * //user1// gets its Livetable instantly updated, and his modification gets overridden
665 )))
667 Moreover, an object can be displayed in Livetables other than the one of its application. Any Livetable using an object should watch for any updates of this object.
669 User Case 3:
671 (% class="box" %)
672 (((
673 //user1//, user2 and //user3// are viewing the results of different Livetables
675 * //user1// modify a property of an object directly from the Livetable
676 * the object is modified on the server
677 * //user2// have this object on its Livetable, so it gets instantly updated
678 * //user3// does not have this object on its Livetable, so nothing changed for him
679 )))
681 The User Case 3 is more complicated to set up, because we have to watch for the update of any entries of the Livetable, and no longer of the Livetable itself.
683 == Beyond the real-time Livetable ==
685 This project is the first step of creating a real-time editable wiki.
687 The end-goal would be to update in real-time not only the Livetables, but also any occurrence of an object in the wiki.
689 User Case 4:
691 (% class="box" %)
692 (((
693 //user1//, is viewing the results a Livetable, and //user2// is browsing some wiki pages
695 * //user1// modify a property of an object directly from the Livetable
696 * the object is modified on the server
697 * //user2// have that property of the object displayed on its page, and it gets updated
698 )))
700 This is not part of the project, but this could be taken in account, to built code that could extend to this purpose in the future.
702 = The Livetable Implementation =
704 == Logic vs Layout script ==
706 A good feature of the new Livetable would be to support different layouts for displaying the data to the user. The table layout is obviously the first one we need to implement, but we could also add other ones like cards layout or calendar layout when the first one is done.
708 If we want to implement such a feature, we need to think our code in two separated layers:
710 * the layout script
711 ** only display what it receives from the logic script
712 ** listen to user interaction, and notify the logic script appropriately (e.g. the user click on the first column header, it tells the logic part to sort the column 1 in reverse direction)
713 ** there can be different layout scripts: table, cards, calendar, timeline, ...
714 * the logic script
715 ** handle all the logical operations: filtering, sorting, pagination, URL hash, ...
716 ** intermediate between the macro and the layout script
717 ** when it received a change notification from the layout script, it apply the changes and fetch the new data from the server accordingly, then pass the new data to the display layout
718 ** works like an API, can communicate with other XWiki widgets, is easily extendable
720 As the user may want to switch between the Livetable available layouts, the logic script could dynamically import the layout scripts it needs. The available layouts should be specified in the macros.
722 == Editing mode vs Designing mode ==
724 The Livetable should be able to handle two modes:
727 === Edit mode (local) ===
729 The user can modify the data in existing rows and add new rows.
731 The configuration modifications are local, and can be saved with the URL hash. This includes:
733 * switching layout
734 * filtering, sorting, columns visibility
735 * other layout specific configuration
737 The user cannot perform actions locally on the data structure, like adding properties locally (this would make no sense).
739 === Design mode (global) ===
741 The user can still modify the data.
743 The configuration modifications are shared between users in design mode, but are not pushed to the server directly. It allows users in edit mode to not receive the modifications, and be able to locally view / edit the data without being bothered by the structured being changed at the same time.
745 The layout configuration are global, and will be saved as default configuration. This includes:
747 * choosing default layout
748 * choosing default filters, sort, columns visibility (and )other layout specific configuration
749 * adding / deleted properties
750 * changing type of existing properties
752 Users with the appropriate rights can push the new Livetable config to the server. This will update the Livetable struture in edit mode too (or show a notification asking to refresh the Livetable in oreder to see the new structure, so that users do not lose their current local config right away).
754 == The table layout technologies ==
756 For each layout, we might need to use a library to help us displaying the data.
758 In this proposal, we will focus only on the table layout, as it was only the first one required.
760 === Solution 1: Using a table library ===
762 We could use a JavaScript library to create HTML tables from existing data. This would simplify and speed up the development of the new Livetable.
764 However, the library will probably not handle the user interactions around the intended features (sorting, filtering, modifying...) the way we want. We will have to rewrite those functionalities in the way we want them to behave. This might become a problem if there are too many functionalities we have to rewrite. We will end up with a whole library where we only use the displaying part of it, and a lot of functions overriding the rest of the existing functionalities.
766 What we could do to solve these problems is to create a facade / adapter object that uses the library functionalities, and adapt them to our needs. Also, in this way the implementation will stay rather independent from the chosen library, and we will be able to upgrade or change the library in the future by just updating the adapter code. This would be great for maintainability in the future years
768 Several libraries could serve this purpose:
770 ==== [[Tabulator.js>>]] ====
772 pros:
774 * development very active (new release every 2~~3 months)
775 * modular implementation: we can easily overwrite and adapt the existing functionalities
776 * already comes with the intended features: display / editing cells by column type. We can add custom formatters / editores for the types we want.
778 cons:
780 * does not natively support IE11, polyfills needed
781 * not very responsive on small layout (can only hide or collapse columns, but collapsed columns cannot be editable through Tabulator native edit system)
782 * didn't find how to edit cells with dblclick instead of single click
783 * native column reorder is a bit buggy
785 ==== [[Datatables>>]] ====
787 pros:
789 * development still active
790 * lightweight and configurable
791 * very easy to add custom formatters for columns
793 cons:
795 * does not support editable cells by default
796 * not really responsive (has an extension to collapse columns but it looks awful, however we could create our own responsive extension)
797 * column formatters work with outerHTML and not HTML elements, so they would be more limited
798 * if a property does not exists in an entry (i.e the property is not empty but undefined), it window.alert an error by default (we can override that but... why?)
800 Others solutions are available, but they are less configurable, no longer in development, or need a commercial license.
802 === Solution 2: Using a framework ===
804 We could also re-write entirely the Livetable component by ourselves. As opposed to the solution 1, this would imply a little more development time, but greater flexibility and maintainability. Instead of using existing code that we have to adapt, we would already build code that meet our needs in the first place. Furthermore, if the layout script only need to display data and handle user interaction, it might not be too complex to create a table from scratch.
806 In this case, we should use a framework to fasten the development, by automatically binding the HTML table to the JSON object, and helping with event handling.
808 [[Vue.js>>]] would be a great solution, it is versatile and easy to set up. With this framework we would be able separate our Livetable in smaller components that could communicate with each other. We could think of components like table rows, table headers, or column filters. With this technique, the code would become more understandable and maintainable in the future.
810 We will have to add an event adapter for Vuejs so that it can communicate with the rest of the page.
812 We can also use some utility libraries to help us during the development, like libraries for drag-and-drop, custom selects, or menus.
814 == Implementing existing features ==
816 The features of the first version should still be present in the new one.
818 These features include:
820 * Filtering entries
821 * Sorting entries
822 * Pagination
823 * Actions to view, edit or remove the entries
824 * Responsiveness
826 These features has to be implemented differently from the first version, to allow more flexibility.
828 For example, the Livetable of [[>>]] allows users to perform complex filtering and multi-columns sorting. However, theses features are not always displayed to the user, and are hidden under a sub-menu of each feature respectively.
830 It would be nice to find a compromise between the complexity of the feature and its easiness to use for the user, and to have the options visible right away.
832 Notion also allows the user to modify not only the data but also the structure of the table (adding / removing columns, changing type of column, ...). When displaying a Livetable on a page this is not desired, the user should only be able to modify permanently the data. What we could do is let the user perform several operations on the columns only on its local instance of the Livetable, like re-ordering, showing or hiding them.
834 As several display layout would be available (table, cards, ...), we also need to consider if we want these features to be implemented in the same way for each layout. If it's the case, that means less development time, and more coherence between the layouts. However, certain layout could have a better suited implementation that would be more intuitive for the user.
836 === Sorting ===
838 For the moment, we can only sort the table according to one column.
840 Implementing a multi-sort intuitive for the user is quite complicated. Most of existing sortable tables does not implement multi-sorting.
842 For those implemented such a system, they came with different implementations:
844 * keeping the previous sorts order for sub-level sorting (Google Sheets)
845 ** pros: really simple, no wizard or sub-menu needed
846 ** cons: not practical at all for the user experience
847 * using a wizard (Libre Office): we can choose which column correspond to which sort level
848 ** pros: easy to understand and perform
849 ** cons: not easily accessible, not easily readable (the user cannot see with one glance how to column are sorting the table)
850 * using a sub-menu ( we access in a sub-menu a sortable list of the columns used for the multi-sort
851 ** pros: easy to perform, a bit more accessible than using a wizard
852 ** cons: still not accessible / readable right away
854 Each of the described solutions makes compromises, and come with pros and cons.
856 Moreover, in most of the cases, we don't have any indication about which column correspond to which level of sorting.
858 In our final solution, we will need to get the sort-system as accessible and readable as possible.
860 ==== Solution 1 ====
862 This is our custom solution for multi-sorting column.
864 We can add a number next to the triangle indicating the level of sorting:
866 [[image:image-20200511112549-12.png||height="88" width="560"]]
868 (Or with color to be more distinguishable from each other)
870 [[image:image-20200511160945-2.png||height="89" width="561"]]
872 Not all the columns have to sort the data, here only //Title// and //Date// are used (unused columns shows a ghost triangle on hover):
874 When we left-click on the icon (or the whole title):
876 * if the column was already sorting (at any level), it changes the direction of the sort (asc / desc)
877 * else, it became the new level-1 sort
879 When we right-click on the icon, a sub-menu appears to let us choose the level of sort we want:
881 [[image:image-20200511161346-4.png||height="106" width="565"]]
883 * if we click on the current sort level, it changes its direction
884 * else, it overrides the existing sort of the chosen level
886 Only the useful levels are displayed in the sub-menu: an already sorting column will not show lower level sorts, and a non-sorting column will show levels until the lowest level possible in the current configuration.
888 Pros:
890 * easy to visualize how the columns are sorted
891 * quick to modify the different sort levels
893 Cons:
895 * right-clicking might not be intuitive for the user, and he might never know it's possible
896 * the colors of the sort levels, even though they are common, might not looks good in some themes, or might not be wanted by the user (solution: every level stays black)
898 ==== Solution 2 ====
900 This second solution is a variation of the first one. Instead of displaying the direction and the level together, we can display both information separately:
902 [[image:image-20200515112513-1.png||height="87" width="572"]]
904 When we hover a title with no sort on it:
906 [[image:image-20200515112904-2.png||height="82" width="300"]]
908 When we click on the title or the triangle icon: same behavior than the left-click of solution 1
910 When we click on the "+" button: same behavior than the right-click of solution 1
912 Pros:
914 * same pros than solution 1
915 * no more unintuitive right click
916 * as the level is displayed on its own, we no longer need to impose the colors to ease the distinction
918 Cons:
920 * it might not be clear that the plus sign refers to creating a new level of sort
922 (Note: with this solution we could still keep the behavior of the right-click as an alternative way to open the sort level sub-menu.)
924 ==== Solution 3 ====
926 We create a sub-menu above the table where we can specify the order of the columns in the multi-sort (like in Notion). This solution would work on any layout.
928 Pros:
930 * keeps the UI light
931 * works on any layout
933 Cons:
935 * functionality hidden behind a menu: we are adding extra steps to sort the table
936 * we can't know with a glance how the columns are sorted
938 We could still display the triangle icon in the table header to indicate that the column is currently sorting the table. Clicking on it would open the sort menu.
940 === Filtering ===
942 For the moment, the filter only try to see if the rows match the specified text. It is not possible to check for the inequality of a number, nor for the range of a date for example.
944 It would be nice to have more complex filtering options, like: "=", "≠", "≤", "≥", "between", ... Specific operators should be added according to the property type (date, list, ...).
946 Also, for now it's not possible to combine filters in another way than a "AND" logical operator between the columns. A solution could be implemented to be able to combine different columns with a "OR" operator (e.g. name="jean" OR age≥25), or to be able to combine different filters in the same column (e.g. age≤20 OR age≥50) The combination of "AND" and "OR" operators should be kept intuitive though.
948 ==== Solution 1 ====
950 For now, we can only have 1 filter for each column, and we can only match equality.
952 What we can do first is to append the operator before the input, with the ability to change it by clicking on it:
954 [[image:image-20200511114754-15.png||height="195" width="582"]]
956 We can also allow multiple filters by adding an action to create more :
958 [[image:image-20200513100959-1.png||height="200" width="583"]]
960 Clicking "add filter" will automatically focus the created input.
962 When there is no filter, there is only the "add filter" action, so that the UI stays light.
964 From the second filter, a new button is added allowing to choose between the "AND" and the "OR" operators. The logic operator has to stay the same for all filters in the same column. For instance if "AND" is chosen, all the filters of the column will be combined using the "AND" operator.
966 With this current design, there is no other way to change the way columns are combined together. For now, all the columns are combined with the "AND" operator. In any way, the operator should also stay the same between each columns to keep the behavior understandable.
968 Here is a test of the design on large layout with a lot of filters:
970 [[image:image-20200513104407-1.png||height="189" width="788"]]
972 and on narrow layout:
974 [[image:image-20200513104531-2.png||height="194" width="433"]]
976 On the narrow layout, the fields are becoming too small, and the result becomes quite heavy (we can work on the design of the inputs to clean the design a bit more).
978 In addition to that, the filters take much more width than the column content, this is really showing up in number columns like the "Age" one for instance. This might become a problem for large Livetable with a lot of columns.
980 However, on average the user is not going to create 3 filters for each columns, and not filters every columns. As the UI complexity is proportional to the filtering complexity, the UI should stay light most of the time.
982 Pros:
984 * unlimited number of filters for each column
985 * filters directly accessible: the user can quickly filter, and knows instantly what's being displayed
986 * straightforward
988 Cons:
990 * we can only use the "AND" operator between columns with this current design.
991 * the filters can take much more width than the column content
992 * not lightweight at all
994 ==== Solution 2 ====
996 We create a sub-menu above the table where we can specify the filters we want to apply to each columns (as Notion does). This solution would work on any layout.
998 Pros:
1000 * unlimited number of filters for each column
1001 * as opposed to the solution 1, the width of the filters is not a problem anymore
1002 * keeps the UI light
1003 * works on any layout
1005 Cons:
1007 * filters not accessible directly: we are adding an extra step to access the filters
1008 * filters not viewable by default: we can't know with a glance how the columns are filtered
1010 ==== Filtering by Tags ====
1012 There is currently a way to filter by tags, from the outside of the Livetable. As the script layer should behave like an API, it should be easy to communicate with it.
1014 ==== Global search ====
1016 We can also add a global search input above the Livetable, that is going to search if the input text can be found anywhere in the entries of the data source.
1018 Any entries containing the query text in one of their properties will be returned. This filtering system should combine with the normal one, with an "AND" operator.
1020 ==== Filtering operators ====
1022 Filtering a string is not the same that filtering a number, a date, or a list of values. Thus, we should not provide the same actions for all type of filters, but adapt according to the filter type.
1024 When adding a filter on a column, the program has to do 2 things:
1026 * create the operator select, corresponding to the column type
1027 * create the input along with its picker, corresponding to the column type, and the selected operator if needed
1029 There can be different pickers for a same property type. For instance, a "Number" type can be associated to the default number input, or to a custom rating select showing stars. Even if the rating select displays stars to the user, it will return a integer (between 1 and 5) at the end.
1031 Ideally, the operators select should not be aware of anything but the property type. However, as the data source may not come with code written for certain operators for the query creation, and as there is no fallback to a default query creation based on the type of the property, we cannot assume that.
1033 Instead, what we have to do is to disable the operators that are not supported by the data source, indicated in the Livetable JSON configuration object.
1035 For each property type, the default operators should be:
1037 * Text
1038 ** "Equals"
1039 ** "Dos not Equals"
1040 ** "Starts with"
1041 ** "Ends with"
1042 ** "Contains" (default)
1043 ** "Does not Contains"
1044 ** "Before"
1045 ** "After"
1046 ** "Like": allows to perform regex-like filtering for advanced users
1047 ** "Is empty"
1048 ** "Is not empty"
1049 * Number
1050 ** "=" (default), "≠", "<", "≤", ">", "≥"
1051 ** "between": shorthand for "≥ filter1 AND ≤ filter2", displays two inputs instead of one
1052 * Date
1053 ** "Is" / "=" (default)
1054 ** "Is not" / "≠"
1055 ** "Before" / "≤"
1056 ** "After" / "≥"
1057 ** "between": shorthand for "After filter1 AND Before filter2", displays two inputs instead of one
1058 ** "Is empty"
1059 ** "Is not empty"
1060 * List (unique values)
1061 ** "Is" (default)
1062 ** "Is not"
1063 ** "Value In": shorthand for "Is filter OR Is filter OR ...", displays a select
1064 ** "Value not in": shorthand for "Is not filter OR Is not filter OR ...", displays a select
1065 ** "Is empty"
1066 ** "Is not empty"
1067 * List (multiple values)
1068 ** "Contains" (default)
1069 ** "Does not contains"
1070 ** "Values In": shorthand for "Contains filter OR Contains filter OR ...", displays a select
1071 ** "Values Not in": shorthand for "Does not contains filter OR Does not contains filter OR ...", displays a select
1072 ** "Is empty"
1073 ** "Is not empty"
1075 Some notes:
1077 * the filters are case aware: lowercase matches both lowercase and uppercase, while uppercase only matches uppercase
1078 * the operators written in words can take up too much space depending the chosen implementation. This would be the case in the Solution 1.
1080 === Pagination ===
1082 The pagination is already functional and nothing but its design might be changed.
1084 We could also implement an infinite scroll feature that the user could choose instead of the pagination system in the table options. However, this would not be great for every cases, as there might be some content under the Livetable that could not be accessed anymore as the table is continuously expanding. The scroll feature could be done inside the Livetable to fix this issue.
1086 Similarly, we could display a "load more" button at the bottom of the Livetable so that the user manually fetches new entries.
1088 === Data export ===
1090 In the new design, we need to keep the feature allowing the user to export the content of the Livetable to csv format.
1092 This is not the priority, be we could add other formats, like excel or json.
1094 == Implementing new features ==
1096 === Editing cells ===
1098 If the user single-click on a cell, the outline of the cell changes color, showing that the cell is currently focused. A tooltip could be displayed on hover telling to double-click in order to edit. The current focused cell is only visible by the user and is not shared with other viewers of the Livetable.
1100 If the user double-click on a cell, the cell will go into edit mode. The displayer module will be replaced by its corresponding editor module to allow the user updating the value. When the user goes out of focus or the user press "Enter", the changes are kept, and the displayer is used again to display the new value. If the user press "Escape" instead, the cell goes back to it initial value. If the user presses "Tab" while editing a cell, the changes are kept and the next cell is focused in edit mode.
1102 === Adding a new row ===
1104 The user should be able to easily insert a new row in the table.
1106 However, as the Livetable does not display all the properties of the pages, we cannot just insert a new row with all its cells editable, as the created page would be incomplete.
1108 We can create a button above the Livetable to insert a new row in the Livetable.
1110 When the user click on it, the action that follows could be either of:
1112 ==== Solution 1 ====
1114 We simply redirect the user toward the add-new-entry page.
1116 Pros:
1118 * cannot be simpler
1120 Cons:
1122 * we leave the Livetable page to create a new entry
1123 * there are now two buttons with the same effect on the page
1125 ==== Solution 2 ====
1127 We display a wizard in a modal allowing the user to fill all the properties of the new entry. This solution is implemented in Notion.
1129 Pros:
1131 * we do not leave the page
1133 Cons:
1135 * way more complicated than solution 1
1136 * we have to create another wizard doing the same thing than the one in the add-new-entry page
1138 ==== Solution 3 ====
1140 We add a new empty row at the bottom of the Livetable, only if all the properties are displayed to the user.
1142 Pros:
1144 * really intuitive to add new data for the user (coherent with the editing system)
1146 Cons:
1148 * if all properties are not displayed, we cannot use this system
1149 * and using two different systems to add new rows could be confusing for the user
1151 ==== Handling rows in the wrong page ====
1153 When inserting a new row or modifying an existing one, there is a possibility that the row has to be displayed in a different page that the current one, because of the Livetable configuration (sorting, filtering, ...). When this happens, we cannot just move the row to the new page it belongs, as it would be confusing for the user (the row would just disappear with no indication).
1155 To avoid that, we can hold the modified row in the current page with a different state, shown by a different display style. In front of the row, we can display an icon representing an "i" in a circle (for "information"), that explains on hover the current state of the row, and that it will be back to normal on the next user action (change sort, filters, modify another row...).
1157 === Selecting rows and Batch operations ===
1159 This feature has been asked by many users: it would be nice to be able to select rows in the table, and to execute an action for all of them.
1161 The action could be: deleting the entry, changing the rights, modifying value...
1163 This would not be complicated to set up: we only need to add column on the left of the table where we display a checkbox for each row. In the column header, there is a checkbox that allow to quickly check / uncheck all the present rows in the table.
1165 Once we select at least one row, a menu appears (or become enabled) above the table, and let the user perform the desired action.
1167 === Columns operations ===
1169 There are several operations on the columns the user might consider, like reordering or hiding some of them. These action would only affect the user display and would have no effect on the server or the other Livetables.
1171 We can include these changes in the URL hash (automatically or by user action) so that the user can keep the current state of his table, or share it with other people.
1173 Moreover, we can think of a system that allow the user to create a new page based on the current Livetable configuration.
1175 ==== Showing / Hiding columns ====
1177 We can create a sub-menu above the table, showing all the columns the Livetable can display, each column associated with a checkbox allowing the user to toggle its visibility.
1179 If some columns are hidden in the Livetable, we should display an icon in the sub-menu title to indicate that not all the columns are shown. The icon could be an exclamation mark or an eye (or both).
1181 ==== Reordering ====
1183 The easiest way for the user to reorder columns is by drag and dropping them at the place they want.
1185 We could also allow the columns listed in the sub-menu above the table to be order-able.
1187 ==== Resizing the columns ====
1189 This is not the priority but if anyone wants this feature, it could be implemented to the Livetable.
1191 == Other concerns ==
1193 * The new Livetable should be compatible and adaptable with the current styling configuration of the wiki.
1194 * It should also respect the accessibility specifications:
1195 ** be a valid ARIA table
1196 ** be usable with the keyboard only (through tabindex)
1197 * The data received from the server should be cached to alleviate the workload of the server when going back to a previously browsed Livetable page.

Get Connected