AI SURVEY ANALYSIS
KEY DETAILS
My Role: UX Researcher
Timeline: January 3, 2024 - January 30, 2024
Tools: AI, Visual Studio Code, Google Sheets
PROJECT CONTEXT
137,414 California public site users provided feedback on their experience using TaxSys by rating the product on a 5-point satisfaction scale. Of those 137,414 respondents, 71,351(~52%) also provided a written response explaining their ratings. The TaxSys team tasked me with analyzing and synthesizing this data, requesting a turnaround time of 1 month.
GOALS
The primary goal was to determine user satisfaction levels and extract actionable insights to enhance the TaxSys experience. The results of this study would also be shared with a GSG client in California who requested data on public user perceptions of our service.
METHOD & PROCESS
When I was presented with the data I quickly realized that the analysis and synthesis process would take months (or longer) to complete on my own. Understanding the challenge I was being presented with, I reached out to the company’s AI Lead for assistance. The AI lead was able to develop code that integrated with artificial intelligence, allowing us to analyze the large volume of feedback rapidly. AI provided me with a broad sentiment analysis and statistical extraction, breaking down the feedback and making it easier to analyze. To avoid error, I supplemented this process with my own manual verification, reviewing direct user quotes and counting the number of users who selected each rating option on the 5-point scale.
ANALYSIS & SYNTHESIS
AI and manual analysis revealed that 78.3% of participants rated their experience as '5' (Excellent), with a mean rating of approximately 4.7.
Sentiment analysis identified six key themes across all ratings:
-
Fees & Transparency
-
Customer Service
-
Dissatisfaction with Taxes
-
UX/UI & Technical Issues
-
Features
-
Security, Trust, Privacy.
These themes were also manually verified and supported through direct quotes from survey respondents.
FINDINGS
Users expressed concerns about high fees, customer service responsiveness, and technical issues with the website. Positive feedback highlighted the ease of use and availability of multiple payment options.
IMPACT & REFLECTIONS
The user feedback study conducted for TaxSys has provided critical insights that are essential for enhancing the company's offerings. The study revealed a high level of user satisfaction, with 78.3% of participants rating their experience as excellent. However, it also identified key areas for improvement, such as enhancing customer service responsiveness and resolving technical issues with the site. By addressing these insights, the company has significantly enhanced the user experience, leading to increased satisfaction and loyalty. This proactive approach not only refined our product but also strengthened the company's competitive position by ensuring its services continue to align closely with user needs and expectations. Additionally, this report was shared with California clients who use TaxSys, offering them valuable insights into public user perceptions of the service and underscoring the benefits of partnering with our company.