Wiki source code of Notification status storage

Last modified by Thomas Mortagne on 2020/02/10 14:08

Show last authors
1 This page is dedicated to gather information and detail the implementation of Solution 2C described on [[Proposal.NotificationsOptimization]].
2
3 The core idea is to move on user configuration based filtering of notifications from read to write step. This have a big impact on the filering and gathering API and architecture but the way to define notifications types should not be impacted.
4
5 = Filtering =
6
7 Instead of filters designed to generate SQL queries the new filters should be dynamic filter components which simply indicate directly if a notification should be stored for a specific user(/group ?) or not. This probably means we cannot reuse anything from the current filtering system and instead design a new system from scratch without any "pollution".
8
9 = Storage =
10
11 We need to store a potentially high volume of very small entries (not much more than a notification and user ids plus a boolean to indicate a read status and probably a few more metadata coming from the notification like the date depending on the engine). So we need an easy to embbed nosql engine (even better one we already have) which fit well this use case. We also need to think about clustering use case.
12
13 == The XWiki Hibernate storage ==
14
15 (/) We already have it
16 (/) The table already exist
17 (/) No duplication and possibility to join the notification table and the rest of the DB
18 (/) Already covers clustering use case
19 (/) Easy to migrate
20 (x) Most SQL engines are really not really supposed to be great with this use case and we would need ALL databases engines supported by XWiki to support it well so that's a pretty big constraint
21
22 Tests:
23
24 100000000 entries in the event status table
25
26 * turns out the perf is actually not bad as long as you setup the right indices. It not the case with the existing table when you are searching the events of a specific user but adding a index on the user reference column made requests very fast
27
28 == Solr ==
29
30 (/) We already embed it
31 (/) Already covers clustering use case
32 (x) Not exactly optimized for this use case (to be tested)
33 (x) Not easy to migrate
34
35 == Lucene ==
36
37 (/) We already embed it
38 (/) No need for any setup/schema
39 (x) No clustering support
40 (x) Not exactly optimized for this use case (to be tested)
41 (x) Not easy to migrate
42
43 == HBase ==
44
45 (/) Designed for this kind of use case
46 (/) Remote solution for clustering use cases
47 (x) We don't have it for other use cases
48 (?) Embbedable ?
49
50 == Not embbedable (for default XWiki Standard) ==
51
52 * Not in the JVM: Redis, Cassandra, PostgreSQL JSON, ArangoDB, CouchDB
53 * Not a compatible license: Neo4j, MongoDB
54
55 = Gathering =
56
57 When gathering users notifications there should not be any filtering other than:
58
59 * the user
60 * the date

Get Connected