Usability testing #7
Adina MilicaDescription
Useful proposals
Past Scenarios
| # | Target Features | Target Users | Types | Run | Report |
|---|---|---|---|---|---|
7 Jun - 28 Jun 2011
|
|
|
|  | |
2014 |
|
|
|  | Not yet run |
2014 |
|
|
| | Not yet run |
27 May - 12 June 2014 |
|
|
| | |
January 2018 |
|
|
| | |
Dec 2018 - Jan 2019 |
|
|
| | |
| #7 scenario |
|
| | Not yet run |
Test Context
Date of UX testing: 17 January 2025
Time of start: 13:56
Time of finish: 14:55 (about 1h)
Recording: upon request
Instance version: 16.4.6
Tasks document: link
User tested: A.P.
Profile of user: Male, 26 years old, Mid Level Dev Ops Engineer, in a very large company. He uses the keyboard a lot more than his mouse to interact with software. His company’s internal organization & collaboration stack includes:
- Sharepoint - knowledge management & sharing
- Jira - project management
- Teams - meetings & chats
- The microsoft office suite for docs, sheets, etc.
The user has not used XWiki before, does not use Confluence or Notion.
Tech info: Windows, Google Chrome
Testing process: The user has been given a Google Docs with the tasks and their respective scenario (the situation in which they would find themselves if they wanted to check out XWiki).
Throughout the test, I observed the user’s screen and also his reactions. I asked questions after he passed important milestones of the task or at the end of the task.
The questions I asked were focused on:
- establishing how he fully felt about the UX (frustration or satisfaction levels)
- checking if he’s just exploring out of curiosity or he’s lost in the UI,
- offering small extra tasks to check certain hypothesis I had.
Based on what I observed, I wrote notes on his progress, issues and feelings. These notes have been polished, but unchanged in their meaning.
I also added a percentage of completion on each task.
- 100% = no issues or confusion, the user adapted very quickly
- 90% = the user didn’t follow the expected journey, but he still completed the task
- 80% = the user presented higher levels of frustration/confusion than usually, but still completed the task
- 0% = the task wasn’t completed or has been completed after indications
The tester and I performed the test IRL and also entered a Google Meet to be able to record his screen more easily. I cropped the video to only see the instance of the user. I trimmed from the initial video the part in which he creates his instance to hide his personal details.
Prerequisites
Conclusions about creating/validating/entering an XWiki instance:
The user perceived the creation of an instance demo quick.
After entering the instance demo, he said he wouldn’t have read the tutorial and even in this specific case he only skimmed through it.
Tasks
1 Create a page
T11: Create another page called “Dev Ops” ✅100%
Conclusions: While on the recording it may seem like the user wasn’t sure of what to do, he was only curious of what the buttons do. He recognized the usual process immediately.
2 Edit a page
T21: Add this image. ✅100%
Unrelated to the task, but relevant:
When clicking the edit button on a page, it wasn’t clear if the text became immediately editable, as there was no immediate visual sign of focus. Although the main content didn’t become focused immediately, the title did become focused, which was confusing for 1-2 seconds for the user until he scrolled down and realized he can edit the text.
Is this behavior a visual bug?
T22: Resize the image with your mouse to make it smaller. ✅90%
Conclusions: The user felt the need to resize from the upper-right corner and had to search a bit for it.
T23: Center the image on the page. ✅90%
Conclusions: The user initially tried to use the editor toolbar for centering. Seeing that he has nothing there, he accessed the image properties quickly.
Should be ignored from the recording: The user played around with wrapping the text until he saw the T23 center task.
T24:
After the image, make a table with 3 rows and 2 columns. ✅ 100%
Enter in each cell the following texts (do not copy them from here): Main, Wikis, HR, Dev ops, Marketing, Sales. ✅90%
Conclusions:
Context for this task and user profile: User uses keyboard a lot more than mouse.
Moving position issue: Tried 3 times to move his typing position after the image by hitting enter. That didn’t work. Tried to use the down arrow like in word, didn’t work. Gained frustration very quickly.
User created the table easily after he moved.
Moving through table issue: The user tried to moved through table cells using arrow, not mouse. By hitting the down arrow he was sent on the cell in the right, and he expected he would go on the cell below. He didn’t expect this not to work at all.
T25: Merge the cells on the first row. ✅90%
Conclusions:
Keyboard merge cells: The user initially tried to use CTRL + select the cells to merge (he explained that that is how it works in Excel). He expected this to work.
Seeing it didn’t, he searched for other ways and found the cell properties quickly.
T26: Make the first row the header of the table. ✅90%
Conclusions: The user found the setting pretty quickly, but his first instinct was to search for it in the Row settings. In the Table Properties UI he didn’t see the setting immediately.
T27: Link the "Sales" word to this external link. ✅80-90%
Conclusions:
After finishing the task, the user said that it wasn’t as easy as he expected.
He expected that once he selects the text in the cell, he could click right on the selected text, and he’d get the option to link the text immediately.
He searched for it a bit, found it, made the link, but was very surprised, and a bit frustrated that the link action wasn’t where he expected.
He argued that the text is not just any text, but a text in a table cell, so quick actions related to the text should be available in the table related component.
T28: Link the "Dev Ops" text to the Dev Ops page in your wiki. ✅100%
Conclusions:
When getting to the link dialog, the user wasn’t sure what to select: the page icon or the arrow.
He expected a tedious process for this task based on what he experienced before and said the task was easier/quicker than expected.
He traveled to another page and then went back, and the link he added didn’t seem to be preserved. When refreshing the page, the link was there.
T29: You can save and exit the editing mode. ✅100%
Unrelated to the task: In edit mode, the image the user added was looking okay, but in view mode it wouldn’t load.
3 Basic macros + attachments
T31: Before the table, add a box to warn others that this is just an initial version of the homepage. ✅50%
Conclusions:
User didn’t realize he could use the editor toolbar to search for a box.
User saw the “/” functionality and tried to use it to search for this box.
After finding it and seeing the rendered result, user says he wouldn’t have realized to search for “box” if it wasn't in the question. Task formulation flaw?
He says that he might’ve thought to search “/info”, but he thinks that it should’ve been easily available in the toolbar.
Placeholder text: When wanting to write in the box, the user wanted the placeholder text to disappear immediately after clicking and was extremely surprised it didn’t.
Exiting the box: When trying to exit the box, the user tried to use Enter or Shift + Enter or Esc and none worked. He then tried to use arrows to exit, but because the next line of text outside the box was used by the table… the arrows led him into the table. When using the mouse, he didn’t realize there is a line that indicates him to add space after the box.
T32: After that, you are thinking of adding the most important documents that often used in the wiki. Download and add to the current page: doc1 ✅90%
Conclusions:
The completion of the task took more than expected from my POV as test lead.
The user asked if attachments are only for the current page or are global.
It wasn’t obvious for the user from where to attach the file. He expected it to be in the upper side of the page.
After finding the attachments section, he easily uploads the file. He expected the loading bar to disappear once it got to 100%. Because I haven’t explained if attachments are global or only related to the current page, he then checked other pages in the wiki.
T33: Now, link this file after the main wikis table. ✅100%
Conclusions:
The user was a bit confused about how to link the file, not in the sense of how to use the UI, but in relkation to the expectations of the task (does the task want a preview of the file in the page or does it want a simple link). Reformulate task.
After clarifying the goal of the task, he does it very quickly.
T331: I encourage the user to also do an extra task and include the file in the page and see how he would do it. ✅80%
He used the “/” functionality, found the right macro for this.
When entering the macro in the page, an error box with a message appears. The user is confused, deletes the box, chooses the same macro again, he seems like he doesn’t know what to do, then clicks and then double-clicks on the macro on the error box. He uses the configuration modal easily and gets the expected result of the extra task.
He says he didn’t understand why did he have to double-click if the message says only click on the box. User is frustrated. He also said he didn’t expect a full rendering of the document, but just a small Teams-like thumbnail of the document.
4 Installing extensions
T41: Install an app that lets you work easily with tasks. ✅90%
Conclusions:
The terminology “app” made the user immediately look at the Applications Panel and the “More applications” link. He immediately saw the “Install new applications” link, but out of curiosity he first checked out “Create your own”. After being confused about that section he goes on “Install new applications” and found the right extension fairly easy.
Before hitting the Install button (which he saw right away), he checked out Show details and was disappointed at what he found. He says he expected there more important details about the app, not at all technical details.
He then went to the Install button, but check out the options available (Install and Install on farm). He was confused by Install on farm. I didn’t explained what it was until the end of the task and told him to continue how he thinks it would work. He clicked the simple Install. He was confused of why does he have to also click Continue and looked around to if he has to do something. Clicked Continue, still confused. He instinctively looked at the Applications panel to see if the app appears there.
T411: As an extra task, I asked what he would’ve done alternatively if “Install more applications” hadn't been there. ❌Failed
He looked in all sides of the screen for a few seconds, and he says he doesn’t know and has no idea what other place he could’ve looked at.
T42: Install an app that lets you make polls for your team. ✅100%
5 Editing a menu
T51: Add a sub-menu under the navigation bar of your wiki. ❌ Failed because it wouldn’t probably have been done if I didn’t offer indications in the end
Conclusions:
Reformulate task: The user was confused about the expectations of the task. I reformulated the task verbally by saying “you need to do a menu component that links to all the important pages in your wiki so you can enter them from any page in your wiki. It doesn’t really need to be under the navigation bar, but it could be there.” He understands and proceeds to search for something related tot he task.
He looks at the menu icon in navigation (this was done after finding out he could’ve entered the extensions UI from the GA section in previous task). He enters Page Index, thinking he might find something there. He then entered Global Administration and managed to find the right UI. When creating the Menu, he is confused by what has been generated.
He checks out other pages to see if something appeared anywhere on the screen, he searches for it in the breadcrumb, he cannot find it, so he doesn’t understand if he did something wrong or why it doesn’t work.
He enters the menu page again and because of increased levels of frustration, I suggest that scrolls down a bit and checks the fields from there. He completes the task and says he wouldn’t have every scrolled down or paid attention to those fields if he didn’t receive indications.
T52: Edit the contents of one of the side panels. ✅90%
Conclusions:
The user chose to modify the Applications panel. Because of that choice, he thought that he might find a way to modify the panel from The Extensions section in Global Administration. He acknowledged quickly that that section is only for installing extensions and continued his search. He found the right UI fairly quickly. He found it satisfying that the UI uses drag & drop.
6 Adding users
T61: Add another user to your wiki. ✅100%
Conclusions:
He found the right UI quickly, but he argued that he only knew where to go because we previously discussed about Global Administration, and he still wasn’t sure if he found the location of GA that intuitive.
T62: Set rights for this users ✅90%
Conclusions:
The user click Edit on the new user he created. Seeing that there is no UI for setting rights in the user configuration UI, he exited and found the right UI quickly. He was confused by unchecked checkbox state. He asked if it means the same thing with forbidding that right.
