Turing Certs - User Experience and Usability Evaluation (Product Research & Analysis)
My Role
UX Researcher, Project Management
Team
5 User Research members
1 Supervisor
4 Company Stakeholders
(CEO, PM, Designer, Engineer)
Skills
UX Evaluation, User Research, Survey, Usability Test, Competitor Analysis
Duration
Feb-Aug, 2022 (6 months)
Overview
Goal
This project aims to incorporate six key research and evaluation methods to analyze TuringCert's product features, interaction design, and usability. Based on this research we provided recommendations for product optimization.
Impact
Successfully provided design recommendations to the company, resulting in concrete improvements and optimizations
For recipients: Updates included certificate management, enhanced download options, and better support features
For issuers: Improvements involve interface adjustments for more efficient data uploads and optimized management of certificate issuance records
Contribution
Collaborated to conduct usability testing and research and analysis process, providing design recommendations to the company for product improvements.
Worked closely with the team to execute user methodologies, fostering collaboration and ensuring valuable insights
Assisted PM and designer in implementing 5+ product feature optimizations based on proposed solutions
Click the button to access the full version of the analysis report
Introduction
TuringCerts provides a one-stop solution for issuing, managing, and verifying digital credentials on blockchain, ensuring security, transparency, and convenience for all users.
Goal
We utilized six user experience methods to gain insights into user behaviors and identify potential issues, providing specific and actionable design recommendations.
Research
Interaction Map
We started by using the product and creating an interaction map to quickly identify the main functions and find interaction issues, such as dead ends and inconsistencies.
User Research
We conducted user research to understand how recipients obtain and utilize digital certificates and perceive them.
RQ1: How do users manage and use certificates?
RQ2: What is the meaning of certificates to users?
RQ3: What concerns do users have about digital certificates?
We analyzed the transcripts by converting them into notes, using a bottom-up approach to identify common patterns (concerns and behaviors).
Based on the affinity diagram, we created a spectrum with nine variables based on logical conflict. The interviewees were positioned on this.
Creating personas based on the user research:
Competitor Analysis
We explored similar products on the market to identify features worth referencing for TuringCerts.
Findings
Key Finding 1 - Understanding past receiving experiences
How certificates are managed and used
Certificates can be categorized by receiving time and type
Recipients often prefer to assign names to their certificates
Users tend to download multiple certificates at once when needed
Meaning of certificates
Proof of abilities - Certificates are widely recognized as evidence of one's skills
Sense of ceremony - Certificates evoke memories
Key Finding 2 - Thoughts and attitudes toward blockchain digital certificate
Concerns about digital certificates
Provide a comprehensive explanation of blockchain risk management and applications to address concerns and strengthen trust.
Trustworthiness of online certificates and blockchain
Regardless of trust in blockchain products, users generally show a high willingness to use digital certificates.
Key Finding 3 - Feature Optimization Suggestions
Seek competitive products and suggest improvements
Validate
Heuristic Evaluation
We evaluated the product's usability and provided specific improvement suggestions.
Usability Test
We gathered insights from potential issuers, identified usability issues, and conducted interviews to provide design recommendations.
UQ2: Can the user understand how to add recipients' information and successfully issue certificates?
Task A-04 had the lowest completion rate (40%), with only two participants completing it. Tasks related to certificate data entry (Task A-03 and Task A-04) scored the lowest overall (4.8 and 3.6 out of 8).
Our average SUS score is 68.75, which indicates it is slightly above the standard average of 68.
Insight & Suggestion
Key Insight 1 - Managing issued certificates
Managing issued certificates is unclear and may cause confusion
Problem
Certificate Image Mismatch - The thumbnail differs from the actual certificate.
Action Functionality Not Clear Enough - The "Preview" button is also not noticeable when the user takes action for the certificate.
Suggestion
Provide an interactive preview feature in the user interface, allowing recipients to view the full certificate in real-time.
Improve the prominence of buttons related to certificate operations.
Key Insight 2 - Certificate information input and upload
The certificate creation process is inflexible and not as expected
Problem
Users are unsure how to use a CSV file to input recipient information for batch certificate issuance.
Suggestion
Provide visual guidelines with step-by-step illustrations for creating and uploading CSV files.
Highlights from Testing
Highlight - 1
Observation
Issuers will want to confirm whether the recipient received the notification email in the expected format.
Feature Optimization Added
Issuers can send a test email to preview certificate content.
Highlight - 2
Observation
Issuers need real-time assistance during the process (e.g., a chatbot or a hyperlink page), with specific Q&A for each step provided.
Feature Optimization Added
Issuers can gain support via various formats (e.g. tutorial videos, user manuals, or FAQs)
Takeaways & Fragments of the Design Progress
Alignment, Communication, and Patience
This project aims to apply research theories in practice through collaboration between industry and academia, reinforcing the link between theory and practice.
During the research, the biggest challenge was the long periods of remote communication (COVID-19 pedantic), making teamwork and alignment especially crucial. Throughout the process, I learned how to use various tools to bridge distance barriers and improve collaboration efficiency.
