Description

All wiki links found in documents are currently registered in a table located in the document's database. We are using it to report links, but more importantly to find backlinks when changing the reference of a document.

The current storage looks like this:

• docId: the local identifier of the document which contains the link
• fullName: the local reference of the document which contains the link
• link: the reference of the target document
• type: the type of the target entity
• attachmentName: the name of target attachment in the document referenced in link

It comes with several limitations:

• the main one is that finding backlinks from other wikis is very slow as we have to request all wikis for links
• it does not have any information about the target locale (it's possible to indicate a specific locate with the page reference syntax, for example)
• the support for page links is a bit fuzzy (a document reference is arbitrarily chosen to represent it, but it might be the wrong one)
• the table only support document as link source and only documents and attachments as link targets, it would be interesting to support more types

Whatever we do, we'll ideally have to keep maintaining the current table and its current behavior for retro compatibility reasons, at least for a while, but it's then not too hard to extract it in a legacy extension.

Database

We need a more generic table than the current one, and also we need to update it on both the source and target wikis.

• sourceWiki: the identifier of the wiki where the source entity is located
• sourceType: the type of the source entity
• sourceDocument: the local reference of the document which contains the link
• sourceLocale: the locale of the document/page content which contain the link
• sourceObject: the reference name of the object which contains the link (null if it's not in an object)
• sourceProperty: the name of the object property which contains the link (null if it's not in an object property)
• targetWiki: the identifier of the wiki where the target entity is located
• targetType: the type of the target entity
• targetDocument: the local reference of the target document (null if it's a page link)
• targetPage: the unlocalized local reference of the target page (null if it's a document link)
• targetAttachment: the name of target attachment (null if it's not an attachment)
• targetLocale: the locale of the document/page where the link is leading to

Pro:

• tables are easy to setup and migrate

Cons:

• data potentially duplicate on two databases
• the support for various types of entities is still a bit hackish and hardcoded because of tables size limits

Solr

Since the goal here is to build and maintain an index, it fits well the main purpose of Solr.

Specific Solr link core

The core could look like this:

• source (string): the unlocalized reference of the resource where the link is located
• sources (strings): the unlocalized references of the resource where the link is located and all its parents to make easier to search or cleanup links from any entity level (per wiki, per space, per document, etc.)
• sourceLocale (string): the locale of the resource where the link is located
• target_<index> (string): the unlocalized reference of the resource where the link is leading to
• targetLocale_<index> (string): the locale of the resource where the link is leading to
• targets (strings): the unlocalized references of the all resources where the links are leading to and all their parents to make easier to search or cleanup links from any entity level (per wiki, per space, per document, etc.)

Pro:

• multiwiki by design, a lot less duplication
• much easier to support any kind of source/target resources, including non-entity links (URLs, mailto, etc.) and even extended with totally unknown types of resources injected by extensions

Cons:

• require a Solr core, always a bit more setup work in the case of a dedicated Solr instance
• impossible to do a database request which includes link criteria among other things

Add links to the existing Solr search core

Since we already index various metadata about all the model entities of a wiki we could as well add links information to it.

We could add the following fields to the search core:

• links (string): the reference of the resources where the various links found in that entity are leading to
• links_extended (strings): contains links plus all the references parents to make easier to search or cleanup links from any entity level (per wiki, per space, per document, etc.)

Example:

Wiki content in document `xwiki:Main.WebHome`:

[[doc:Space.Document]]
[[attach:Space.OtherDocument@Attachment]]
[[page:Page1/Page2]]

Resulting index:

• links:
  • document:xwiki:Space.Document
  • attachment:xwiki:Space.OtherDocument@Attachment
  • page:xwiki:Page1/Page2
• links_extended:
  • document:xwiki:Space.Document
  • attachment:xwiki:Space.OtherDocument@Attachment
  • page:xwiki:Page1/Page2
  • wiki:xwiki
  • space:xwiki:Space
  • document:xwiki:Space.OtherDocument
  • page:xwiki:Page1

Pro:

• multiwiki by design, a lot less duplication
• possible to query backlinks for a whole wiki, space, etc.
• much easier to support any kind of target resources, including non-entity links (URLs, mailto, etc.) and even extended with totally unknown types of resources injected by extensions
• easier to do a query which involve links and other document related matadata
• already existing Solr core

Cons:

• slightly less generic than a dedicated link core