Prototype
Introduction
The goal of this document is to describe the prototype and key aspects that have been experimented. It will also list additional aspects that have not yet been experimented and could have an important impact on the architecture.
Language
The based coding language used in the prototype is Typescript.
Typescript has been experimented successfully with tsc and vue-tsc. The support of typescript by vue is nice. The vitejs tool worked well with typescript. The editor used is VSCodium which is displaying all code errors properly and the development productivity is quite high.
Typescript has strong advantages of favoring "api-based" development.
Frameworks & Front-end Technologies
The prototype is using Vue3.
Vue3 is being used and allowed to build a "skin" system. It also allowed to "abstract" the design system (see below). Integration with Typescript is working well. Vue has support to consume web components.One aspect to check are the complexities around upgrades. LiveData is still in Vue2. As time passes, new breaking version of frameworks will happen and it will be difficult to support extensions build on an older version of a framework with a new version of it being used in the core application. It risks breaking compatibility and require to "upgrade" all extensions at the same time.
Design Systems
The prototype is allowing to abstract the Design system. An implementation with Vuetify and DSFR has been made.
In the prototype being worked on, an API has been created to abstract the design system and therefore create the "X design system" which would proxy the favored design system and also allow to create multiple implementation of the design system. In the prototype 3 base components (x-alert, x-btn, x-card) have been used with an implementation with implementation in Vuetify and in DSFR. The result is interesting.
Accessibility
In the prototype being worked on, nothing special has been made yet. It will be very important to set accessibility coding rules early in the project and include testing tools right away.
Packaging and Module Bundling Tools
ViteJS is used.
In the prototype being worked on, initially webpack was used, then rollup to use vitejs (which uses esbuild for compiling). In case of using native Javascript Modules for the development and the delivery, webpack is not the tool to use, as webpack is made to actually do the opposite. ViteJS is much more used by the Vue community (built by them) and in the end was much simpler to put in place. Integration with typescript worked well.
Dependency Injection
In the prototype being worked on, inversify has been used successfully. There are some concerns that the project is not very active. The other frameworks should be tested also. The inversify library has been succesfully used in combination with dynamic loading of extensions and also with static building which makes debugging easier.
The plan is to check other IOC frameworks
Extensions build with Standard Javascript modules
By using vitejs (or rollup) it is possible to build standard javascript modules.
These modules can be loaded dynamically as extensions and provide components to inversify allow to extend the features.
Static bundling can also been turned on to bundle some of the external modules in the main package.
In the prototype being worked on, extensions have been packaged as "standard JS Modules" with an entry point loading components using inversify.
Circular dependencies between extensions has not been experimented. To keep things simple, key APIs have been moved to an "api" module loaded first by all extensions.
The approach is compatible with VSCodium seeing the dependencies between the code (using a mono-repository).
Editor Technologies
No experiment yet in the prototype on a "new editor". However it was possible in the prototype to load the XWiki editor in an IFRAME with all features of the editor available. This could help give us time to build a new editor.
Rendering
Rendering has been done both server-side but also client-side.
- The XWiki wikimodel parser has been posted to JS using TeaVM
- The marked Markdown parser has been used for markdown, adding a macro extension
Client side rendering of Macros has been experimented.
Wikimodel Porting to Javascript
Using TeaVM which is converting bytecode to Javascript (and possibly WebAssembly) it was possible to compile Wikimodel with no modification to the code, using a maven plugin.
https://github.com/ldubost/xwiki-rendering/blob/wikimodel-teavm/xwiki-rendering-wikimodel/pom.xml
Another experiment was done using GWT (which was working). Unsuccessfull experiments have been done using JSWeet and J2CL.
Some learning about the different technologies
| Technology | Tea VM | GWT | J2CL | JSweet |
|---|---|---|---|---|
| Method used | Converting bytecode | Transpiling | Transpiling | Transpiling |
| Successful | Yes | Yes | No | No |
| Size of Code | Around 350k minified | Around 200k | 6,6Mb before optimizations, which fails and generate 0 size code. | |
| Problems | The output is not a standard JS Module and makes use of the "window" object. It was however possible to load the code in a Typescript module to encapsulate it in a Typescript class (which still relying on the window object). It seems however closer to be able to get rid of the window object than GWT. | The output is relying in a window object. The starting of the module requires to load a JS module which does not seem possible to move in a Typescript module. Modifications were necessary to get rid of java.io modules and java.util.regex for the GWT implementaton. This required modifications to the javacc module. The cssparser (which is not needed for wikimodel) needed to be removed. GWT is out of life and supposed to be replaced by J2CL | Compilation worked only in non minified version but tests have not been made to run it. The same code changes as GWT are needed to compile. The same implementation of GWT Regex is used.
| The code compiles to Typescript It's still failing on one type of instruction concerning WikIStyle in IWemConstants (class 'WikiStyle' used before its declaration) On the typescript side, it's using namespaces which are not standard JS Modules. It's possible that with the modifications for GWT and J2CL it could work with JSWeet. It would be nice to use the Typescript outputdirectly but it's unclear if the namespace output can be encapsulated in standard modules. |
| Compilation Speed | Fast 5 seconds (30 seconds maven build using tests) | Slow 16 seconds (40 seconds maven build without tests) | Fast but without the strong optimizations which don't work | Failed in 10 seconds |
| Speed | It seems fairly fast. Measurements are needed. | It seems fairly fast. Measurements are needed. | Could not be tested | Could not be tested |
| Maintainability | As there was no code changes maintainability seems good. It's unclear if it's possible that TeaVM generates standard modules in the future. It should be possible with WebAssembly to encapsulate running the code. However the output is possible bigger in WebAssembly. | GWT is out of life. There are risks involved. | J2CL is sponsored by Google. However it's unclear how it's used. It's based on Closure compiler so there is a little chance to get typescript output. | JSweet seems active but it's unclear who sponsors it. As it generates Typescript intermediate code it seems interesting. However it's unclear if it could in the future generate standard modules. |
| Risks | Future updates could break, as it's dependent on a class library specific to TeaVM and a build tool which is complex. | |||
| Other | Given that the bytecode compatibility seems to more easily go further than transpilation, it could be interesting to test if XDOM can be compiled as currently wikimodel only provides the parser of Wiki Syntax not the HTML rendering. | GWT is not really made to make libraries. | J2CL is supposed to be better for libraries than GWT. | It can output Typescript (but namespace based) |
APIs
Some work was done to create some internal APIs and also a JSON-LD API for XWiki
Internal APIs
Internals APIs have been created for WikiApp, Extensions, Logger, Storage, document, skin, UIX, Macros
See https://git.xwikisas.com/wiki30/prototype2023/-/tree/main/api/src/api?ref_type=heads
There will be a good based to design a more complete and cleaner API.
JSON-LD API
A JSON-LD converter has been done as a REST API in XWiki. It allows to:
- convert simple XWiki documents to JSON-LD
- convert documents based on 2 XWiki Classes (Blog and Movies) to JSON-LD (A conversion mapping is necessary for optimal conversion. A generic conversion could be build but has not yet been done)
On the Cristal side, an experiment of "sheets" for structured documents has been made based on JSON-LD both in view and edit mode.
