The key to knowledge:
speaking the user's language in the UN Digital Library
Case Study Summary:
The United Nations Digital Library (UNDL) hosts a large repertoire of invaluable digital records that they call Items Details Page (IDP). Such records hold immense knowledge for academics, policy makers, and countries in search of wisdom, truth, and justice, and all at no cost. Despite this great promise, the UNDL does not believe the resource to be accessible to individuals of the general public. Based on a high number of repeated service desk inquiries by general public or non-specialized UNDL users, the UNDL requested a formal review of the usability of their IDP’s.
By triangulating survey data, user eye-tracking sessions, post-test metrics, and more, I confidently identify 36 user problems, summarized into two key findings: general public users have difficulty with the IDP’s labels and terminology and find the IDP’s layout inconsistent and difficult to read. To ensure that the search for knowledge is available to people and organizations of all backgrounds and sizes, I recommend redesigns that make IDP’s easier to read and understand through accessible considerations and user-friendly language.
My role
As a UX researcher, I led the development of our research plan, facilitated and analyzed usability tests, and analyzed findings for redesign recommendations.
Individually, I managed our group project artifacts and timeline, defined research questions and user scenarios, analyzed our findings for key findings, and redesigned a mockup for accessibility compliance and consistent IDP layouts.
Within a team capacity, I served as both a group facilitator, project manager, and sounding board for ideas. By the end of the project, we produced four IDP redesign recommendations and effectively presented our key findings in a final pitch to the client.
Teammates
Liwei Jiang
Kevin Zhang
Shelly Guan
Project Type
UX research
Mixed-methods
Eye tracking usability test
Tools
Figma
FigJam
Tobii Eye Tracker
Matomo Web Analytics
Google Workspace
Timeline
February - May 2025 (2.5 months)
Setting the scene
The UNDL: an underutilized and hard-to-master resource
The United Nations stands as one of the world's most influential forces for peace, security, human rights, and human progress. At the heart of its knowledge sits a vast, open-access repository of historical records, resolutions, meeting minutes, and reports free for anyone in the world to explore through The United Nations Digital Library.
Yet despite its remarkable scope, the library remains largely unknown to the general public, and even those who find it often struggle to navigate without the assistance of the service desk. As the organization began working to modernize its public-facing interface, the UNDL requested a formal usability review of its Items Detail Pages for general or non-specialized users.
Defining the problem
Problem statement
Since its inception in 1998, the UNDL has never undergone formal usability testing, leaving a critical question unanswered: is this wealth of knowledge truly accessible to everyone that they aim to serve?
How might we
How might we propose practical but impactful changes to the existing UNDL website to improve access and usability of the UNDL Items Detail Pages for general or non-specialized users?
The process
To undertake this project, I followed the design thinking framework to empathize with users before defining a problem to design for. User testing was not included in this project as later discussed in this case study.
Learning the undl landscape
A highly visible website with user pain points to address
Kickoff meeting
An initial meeting with UNDL kicked off the research project, with many insights gleaned. Of the many, remarks that stood out included:
Kickoff meeting takeaways
-
Specialized language, even for librarians is an existing pain point
-
Frequently asked questions such as: “Where can I find this?” and “Who is the author?”
-
Some IDP’s have attached while other’s do not. Do users understand this?
-
Users likely aren’t aware that digital records have just yet to be digitized but can be upon request
Matomo web analytics
The UNDL site experiences a great number of daily visitors, predominantly looking for a record, but are bouncing right away.
Matomo serves as UNDL’s dashboard for tracking online visitors. With this data, we begin to see the Items Details Pages as the dominant user touch point (76% of all page views). High bounce rates upon entering a webpage begins to suggest users leaving the website without exploring much further. Additionally, a four-minute average session time leads our team to wonder what this time is spent doing?
Unfortunately, because of the high bot visitors on the website, and high variability in the definition of a “session,” bounce rates and session times was not used from Matomo. However, these initial insights did inform me that lots of people are visiting UNDL to mostly to look at IDP’s but often leave right away before exploring the site further. Such a high bounce rate warrants great concern around information access.
What are general users doing while visiting the United Nations Digital Library Item Detail Page?
Why are they leaving so soon?
Defining the research plan
Triangulating Data to Uncover the Full Story
Research plan
To build an understanding around UNDL and their web analytics insights, a mixed-methods research plan was developed under the following research objective:
Objective:
Understand how general users interpret IDP metadata, language, and resources to navigate through the UNDL.
Research questions:
-
How confident do users feel extracting information from IDPs?
-
Does the terminology match users' natural language?
-
What barriers do users encounter?
-
What do users expect when clicking "download"?
-
Does layout influence how users evaluate metadata?
The following research methods were selected to directly answer our research questions:
Method 1
Matomo Web Analytics
Upon reviewing this tool, “Bot/Unknown” made up 82.5% of unique visitors, leading me to lean on other methods.
Method 1
UNDL Web Experience Survey Analysis
Research Questions Addressed:
1 & 3
Method 1
Eye-Tracking User Testing with Retroactive Think-Aloud (RTA)
Research Questions Addressed:
2, 3, 4, and 5
Method 3
Single Ease Question (SEQ)
Research Questions Addressed:
1
Method 4
System Usability Scale
Research Questions Addressed:
1 & 3
Method 1: Undl Web experience survey
The UNDL Website Experience Survey spotlights the lack of awareness by general users and difficulty with finding records
From the Website Experience survey, 260 individuals responded. With over of survey responders self-identifying as scholars (academics and library-affiliated), and only 33 or 13% as the general public, it’s clear that an awareness of the UNDL by the public is low, and among all individuals, finding records where the greatest pain point (47% or n=121).
Of the respondent groups, people from the private sector (n=12) rated themselves as the most confident using the website (score of 4 out of 5), while the general public had the lowest (n=33, score of 3 out of 5).
Survey and web analytics results are highlighting an issue with general users finding UNDL records.
Why?
Method 2-4: Eye-tracking User Testing with RTA, SEQ & SUS
Eye-tracking user testing surfaces some wins but also 30+ user problems to consider addressing
It was essential to see and hear directly from non-specialized people as they navigate UNDL item detail pages. Eye-tracking was conducted alongside retroactive think aloud (RTA) to physically track how users respond and interact with the website and offer both quantitative and qualitative evidence for this study.
Eye-tracking affords the ability to visually represent individual and aggregate user gazes of websites through heat maps, opacity maps, and gaze plots. Along with RTA after each session, where participants could then vocalize their thoughts and reactions to the website for us to hear first-hand, we could begin to build a case using user quotes, data visualizations of gaze patterns, and traditional usability metrics including success rates, system usability surveys (SUS), and single ease question (SEQ) scores.
8
participants
-
Ages: 18-35
-
All were comfortable with downloading files from a website
-
1/2 were uncomfortable with library databases
-
0 have used the UNDL website
Eight total participants were screened as “general users” (or not specialized in research) to participate. Three scenarios were presented with five total tasks to address our five research questions. Users took anywhere from 20-60 minutes to complete the session, and each person was compensated for their time.
Positive findings highlight SOME areas that general users find useful, understandable, and doable
Positive findings:
-
Users fully used the “Details” section to look for and confirm information
-
Users understood the purpose of the “Subjects” and “Collections” sections
-
When looking to download a document, people were able to do so via the download button & “Access” link.
Despite these celebrations, there were problems to address
Using a rainbow spreadsheet, I identified 36 issues from user test sessions. Each issue was given a severity rating (i.g, how serious this issue was and how urgent it was to troubleshoot) between 1 and 4. Issues with a higher severity rating (scores of 3 or 4) and occurring across more than half of our participants were prioritized for further evaluation and potential recommendations.
36
problems identified
User test sessions indicate multiple areas for improvement.
Which are most urgent to address and how?
Research Synthesis: Findings & Recommendations
The IDP is navigable...but the path towards knowledge is not clear for general users
User Success Rates
Users can complete most tasks, but some were more difficult than others
System Usability Scale (SUS)
UNDL IDP falls below acceptable industry standards, highlighting a great need for redesign
The UNDL was also benchmarked against industry standards using the System Usability Scale.
With a score 46.4, which is well below the benchmark of 68, this places it in the not acceptable range. While users are generally able to complete tasks, the experience is inefficient, inconsistent, and requires more effort than expected. This result emphasizes the need for a redesign for all users.
Ultimately, seven themes emerged from our problems list. These themes were further consolidated into two major re-occuring themes:
Ultimately, seven themes emerged from our problems list. These themes were further consolidated into two major re-occuring themes:
Key Finding 1:
General Public Users have difficulty with the IDP’s labels and terminology.
Key Finding 2:
UNDL Web Experience Survey Analysis
Key finding 1:
General Public Users have difficulty with the IDP’s labels and terminology.
During our usability test, we often saw participants struggle with understanding a lot of the UNDL’s labels and terminology. Some of the language on the IDP reflects the UNDL’s internal and bibliographic database logic rather than how general users naturally think about and describe these concepts.
Supportive Findings 1 Labels & Terminology
UN acronyms may confuse those who expect an author name, according to user quotes and opacity maps
One of the most frequently asked questions the UNDL receives is “Who is the author?” When participants are asked to identify an IDP author that was “UNCTAD”, participants were not confident and had to confirm if the author was truly an acronym by double checking the details section.
Supportive Finding 2: Labels & Terminology
Gaze plots, quotes, and SEQ scores show that users are not confident they found what they’re looking for, even if they are correct
It was essential to see and hear directly from non-specialized people as they navigate UNDL item detail pages. Eye-tracking was conducted alongside retroactive think aloud (RTA) to physically track how users respond and interact with the website and offer both quantitative and qualitative evidence for this study.
When tasked with finding a linked record to an IDP via the “meeting record,” all eight participants stated they did not feel confident they were in the right place, despite five of them successfully navigating to the correct meeting record.
Our participants rated this task was rated an average score of 1.9 out of 7 in difficulty, indicating it was difficult and created low confidence in our participants.
This is another example of how unclear labels left users unable to feel confident, even when they had found the right result.
Recommendation: Labels & Terminology
Relabel to words users are familiar with and provide tooltips for specialized terms
To address our findings, IDP redesigns are created with the goal of maximizing the usability improvements of small but mighty changes. This ensures that UNDL doesn’t expend too many resources to enact change while ensuring users don’t have to completely readjust to a new website. Lastly, features proposed are based on language and tools that most general web-browsing users should be familiar with.
Such simple fixes include updating terminology that adds clarity and distinction to the functionality of features on the IDP. Additionally, in cases where terminology cannot change due to reasons such as institutional tradition, tooltips with definition and context would greatly benefit all users in seeking knowledge.
Changing “formats” to “citation formats” to match general user expectations
Providing tooltips to provide additional context and support for unfamiliar labels such as “symbol”
During the retrospective think aloud, participants consistently expressed feeling very lost and frustrated with the task. Most users stated they could not find anything recognizable, like a file name or the title of the document, and did not think the label “meeting record” would lead to the relevant document they needed.
Key finding 2:
Users find the IDP layout inconsistent and difficult to read
As seen in the video below, when participants transition between two different IDP layouts, they spend time trying to understand what changed and how that impacts their ability to find a record.
Supportive Finding 1: Inconsistent layout and difficult to read
Heat maps and user quotes indicate that users get confused when IDP layouts change
Upon arriving on an IDP without a download button, most users kept looking for "title", "download", and the "details" section to orient themselves and find where a potential record could be. Doing so, they were clearly aware that the layout had changed for some reason but had no inkling as to why. In particular, they noticed when the title and download tab went missing.
Supportive Finding 2: Inconsistent layout and difficult to read
Heat maps and user quotes indicate that users get confused when IDP layouts change
On another hand, users found the IDP difficult to take in because it legibility. Half of our participants remarked on the frustration of trying to read the IDP because of small, condensed text.
Based on these remarks, we dug into the code and found that much of the descriptive text on the IDP was far too small in font size and too narrow in font weight.
Referring to WCAG's accessibility guidelines, we found that minimal fonts on the IDP should be 14px and font weights at least 400px or regular, unless font sizes are much bigger.
On a positive note though, details title texts are readable as well as color contrast, assuming font weights again are increased to regular or higher.
Recommendation 1: Consistent layouts
Maintaining consistent IDP layouts through disabled states and tooltips
Given our findings, to address the layout challenge, we recommend keeping the layout the same for every IDP but adding in disabled states for actions that users cannot take for certain IDPs.
An additional tooltip with an explanation and instructions on what to do next would give users more context and guidance for moving forward. The second tooltip line was based on our best guess of what UNDL would want to communicate, however UNDL should definitely think through the final message they'd want to convey if moving forward with this tooltip.
Recommendation 2: legible text
Standardizing font sizing to meet WCAG accessibility compliance
For our second recommendation, we've increased the font size of all descriptive text to 14px and regular font weight. To keep the information hierarchy of the text consistent, increase the font size of section headers were also adjusted.
Final Delivery
In conclusion
My pursuit for understanding barriers to knowledge access for UNDL IDP users resulted in a greater appreciation for always centering the language and experience of users. Things as simple as words, font size, and inconsistency can prevent people from fully accessing and leveraging the trove of resources that The United Nations Digital Library offers.
This message, along with the research findings and recommendations, was conveyed in a final report deck to the UNDL team. To support their long-term website redesign initiative, digital copies of our deck, problems lists and highlighted videos of user sessions were shared for their own keeping as well.
A positive and enlightening moment for the United Nations Digital Library
UNDL expressed immense gratitude for our findings, calling it “fantastic.” With the evidence provided, they were surprised but completely understood the common theme around general user experiences being challenged by library and UN jargon. In particular, one client highlighted how something as simple as increasing font size could make “such a difference” in making the IDP more accessible for their users, especially based on a participant pool aged 35 and under. Another expressed,
“Thank you for putting us in this mindset to speak a different (general user) language.”
Lessons learned: the power of language and a variety of evidence
Language is powerful. It powers knowledge and inspiration, but when highly specialized, can also confuse and exclude others. By creating a habit in intentionally considering all potential users, using user-friendly terminology in my future work can ensure consistently inclusive experiences for all.
Lastly, having a vast variety in data and evidence was incredibly helpful in conveying our findings to UNDL. By packaging user quotes, with quantitative stats as well as benchmarking standards, a compelling story and arguments were effectively and welcomely received by UNDL.
Although I won’t be with UNDL to implement our design recommendations or to see the impact on their users, my meetings with The UNDL leaves me confident that they will continue this mission of serving all knowledge-seekers.
Next Steps: Continuing the mission
Energized and inspired by these research findings, UNDL asked for this presentation to be shared with their stakeholders and larger team.
Beyond continuing to share these research findings, if the UNDL were to continue our partnership, I would recommend a second round of user testing with our redesign recommendations as well as A/B testing. Doing so would allow us to accurately measure the impact of our results and iterate based on findings. A/B testing on the impact of tooltips and updated terminology in particular could be tracked against customer service tickets and click rates on information icons.