Wiki source code of Notification status storage
Last modified by Thomas Mortagne on 2020/02/10 14:08
Show last authors
author | version | line-number | content |
---|---|---|---|
1 | This page is dedicated to gather information and detail the implementation of Solution 2C described on [[Proposal.NotificationsOptimization]]. | ||
2 | |||
3 | The core idea is to move on user configuration based filtering of notifications from read to write step. This have a big impact on the filering and gathering API and architecture but the way to define notifications types should not be impacted. | ||
4 | |||
5 | = Filtering = | ||
6 | |||
7 | Instead of filters designed to generate SQL queries the new filters should be dynamic filter components which simply indicate directly if a notification should be stored for a specific user(/group ?) or not. This probably means we cannot reuse anything from the current filtering system and instead design a new system from scratch without any "pollution". | ||
8 | |||
9 | = Storage = | ||
10 | |||
11 | We need to store a potentially high volume of very small entries (not much more than a notification and user ids plus a boolean to indicate a read status and probably a few more metadata coming from the notification like the date depending on the engine). So we need an easy to embbed nosql engine (even better one we already have) which fit well this use case. We also need to think about clustering use case. | ||
12 | |||
13 | == The XWiki Hibernate storage == | ||
14 | |||
15 | (/) We already have it | ||
16 | (/) The table already exist | ||
17 | (/) No duplication and possibility to join the notification table and the rest of the DB | ||
18 | (/) Already covers clustering use case | ||
19 | (/) Easy to migrate | ||
20 | (x) Most SQL engines are really not really supposed to be great with this use case and we would need ALL databases engines supported by XWiki to support it well so that's a pretty big constraint | ||
21 | |||
22 | Tests: | ||
23 | |||
24 | 100000000 entries in the event status table | ||
25 | |||
26 | * turns out the perf is actually not bad as long as you setup the right indices. It not the case with the existing table when you are searching the events of a specific user but adding a index on the user reference column made requests very fast | ||
27 | |||
28 | == Solr == | ||
29 | |||
30 | (/) We already embed it | ||
31 | (/) Already covers clustering use case | ||
32 | (x) Not exactly optimized for this use case (to be tested) | ||
33 | (x) Not easy to migrate | ||
34 | |||
35 | == Lucene == | ||
36 | |||
37 | (/) We already embed it | ||
38 | (/) No need for any setup/schema | ||
39 | (x) No clustering support | ||
40 | (x) Not exactly optimized for this use case (to be tested) | ||
41 | (x) Not easy to migrate | ||
42 | |||
43 | == HBase == | ||
44 | |||
45 | (/) Designed for this kind of use case | ||
46 | (/) Remote solution for clustering use cases | ||
47 | (x) We don't have it for other use cases | ||
48 | (?) Embbedable ? | ||
49 | |||
50 | == Not embbedable (for default XWiki Standard) == | ||
51 | |||
52 | * Not in the JVM: Redis, Cassandra, PostgreSQL JSON, ArangoDB, CouchDB | ||
53 | * Not a compatible license: Neo4j, MongoDB | ||
54 | |||
55 | = Gathering = | ||
56 | |||
57 | When gathering users notifications there should not be any filtering other than: | ||
58 | |||
59 | * the user | ||
60 | * the date |