Activision Support
Survey Tool System
Led the creation of a survey tool that refined and simplified the user experience. By auditing, deconstructing, and rebuilding our survey system, we were able to significantly optimize the data being collected, helping our Data Analyst team gather more impactful insights, and improving overall efficiency to their workflow process. The new system resulted in increased volume in surveys taken, increased engagement once the user chose to take the survey, and improved customer service satisfaction ratings after they finished.
my role
Product Design Lead
Responsibilities: Stakeholder Interviews, Research, UX Strategy, Cross-Functional Review Sessions, Competitive Tool Analysis, User Flows, Wireframes, Visual Design, Prototypes, A/B Testing, Design System Components, Functional Specs
team
1 Lead Designer, 1 Associate Product Manager, 1 Project Manager, 1 Developer, 1 Software Engineer
Q4 -2020 _ Q1 2022
TOOLS
Adobe XD, Salesforce, AEM
PLATFORM
Responsive Web
01
background
Activision is a leader in the gaming space and after the launch of their free-to-play title - Call of Duty: Warzone - traffic to the support site increased by the millions! Negative thoughts about COD: Warzone affected customer satisfaction ratings, which was related to the actual game and not the customer service. The Activision Support overall business performance is largely impacted by their customer satisfaction surveys (CSAT), and so as the usability expert, I was assigned the ambiguous task of revamping the survey system. Having the challenge of limited user-testing resources, I saw this as an opportunity to gather more precise qualitative user data to improve their service experiences and embarked on a mission to deconstruct and create an efficient survey system to positively affect business KPI’s.
— The Problem
The rushed launch of the new COD: Warzone game increased negative game performance for players, resulting in an overwhelming and unbalanced perspective of Activision Support’s service ratings.
— The Solution
Capture focused customer service insights from our surveys.
Q4 -2020 _OCT
02
taking a step back
I was curious, revamping the survey system, but why? In order to understand the full scope of the project and define the problem, I conducted 1 on 1 interviews with the Director of Support and Manager of Operations to understand vision, values, goals, expectations, and success factors. These insights helped me define the problem:
“How do we improve support experiences and increase CSAT scores?”
The solution? By capturing focused customer service insights from our surveys revamping the survey system, but now with a better understanding of the problem. This statement helped me write up some goals and next steps that would inform the audit process.
GOALS:
Overarching:
Deliver customer service focused insights to improve player experiences and improve analytics workflow process.
As a user, I would like to…
Experience better support based on my feedback
Decrease level of effort when taking the survey
Be incentivized to provide my feedback
(later removed from scope)
As a business, I would like to…
Validate or set new target for metrics for CSAT (maintain or increase from 5.7)
Increase survey exposure
Increase volume of survey submissions
03
my process: audit+research+ideation+review+iterate+test
Q4 -2020 _NOV
Audit
Starting from scratch, I had to schedule MULTIPLE sessions to audit the process and gauge how, when, where, and why our surveys were being used with cross-functional partners:
Content | Product Managers | Analytics | Project Managers | Front & Back-end Engineers | CS Agents
Research
My research was pretty broad. I had a lot of questions to consider based on my cross-functional sessions. I started off by having some general survey best practices solutions based on UXR knowledge, but I wanted to get a refresh of what marks a successful survey.
Tediously pouring over spreadsheets of survey responses to see any type of pain-point patterns
Taking a close look at all the surveys provided and matching them to good ol’ Nielson Norman Group best practices and gathering as much survey data from academic sources, while reviewing multiple articles about best use cases for customer satisfaction metrics (CSAT, NPS, CES).
Competitive and comparative analysis of survey tools and what their capabilities were compared to our current tool
Ideation
Research then paved the way for incubating some recommendations. The main learning and overall recommendation of this process was… Consolidation.
Consolidating the amount of cross-platform surveys, survey questions, survey tools, survey metric ratings, but at the same time being purposely focused while increasing more contextual targeted web surveys.
Below is a snippet of the old survey audit.
Analyzing the design with my research lens, I wireframed and quickly prototyped some initial ideas to get our stakeholders in the mindset of how these solutions could take place.
Review
I created a quick deck and piled my research, insights, and designs into a format that could help get buy in from our stakeholders. It was taken pretty well, with some questions on refinements with different cross-functional teams and more discovery with designs.
+
Iterate
The decision was made to move forward with the ideas, with refinements. Next steps taken were collaborating with a Product Manager and setting up working sessions to come up with more solutions for content, frequency, and designs.
Q4 -2020 _Q2 2021
00
intermission*cue jazz elevator music
This story was backlogged for Q1 and Q2 due to “hosing down fires”
Constraints on content strategy sessions to nail down experience metrics definitions*
Aggressive timeline to launch
Higher priority initiatives injected into timeline due to title launch
*At this point, I collaborated with a product manager on cross-functional survey content sessions. We would partner up with the content/UX writer to hammer out how we would structure the questions and go through numerous review sessions with stakeholders. This went on up until development. This was a challenge due to the nature of the service business and how success was measure by individual teams.
Q2: 2021 _JUN
04
survey tool vetting
Stakeholders saw value in my proposal to consider other survey tools and so I conducted a full-scale survey tool analysis and broke down the pricing and benefits in another presentation. I was able to get stakeholder buy-in and I partnered with a couple of Product Managers and interviewed at least 5 different survey tool vendors to understand the value of the tool and how it could relate to our business.
My selection was GetFeedback, a tool that incorporated ease of data collection, flexible contextual surveys, and was much cheaper than our current tool. Some design trade-offs included full customization of the survey design.
We did another round of presentations covering the opportunities and weaknesses of each preferred new tool, our current tools, and building the tool from scratch.
Ultimately, the stakeholders decided to pick building our own tool. It wasn’t my recommended option, but it was the next best between current system, new tool, and building our own. Some of the decision stemmed from a priority to have a well branded and targeted experience (they loved my initial prototype).
Q3: 2021 _JUL
05
iterate+review+test
Throughout this project I went through many design iterations to help give context to the proposals and also during tool vetting. Refinements were made but there wasn’t as much alignment, so I took to some quick user A/B testing. We didn’t have any dedicated user testing tools, but had a subset of users we could tap into. Thanks to the power of Google surveys and Discord for distribution, we landed on the highest result. After many months of deliberation, the business finally chose the designs I recommended.
Iterate
Review
Test
Web Experience
Survey Screens
Article Experience
Survey Screens
Q3-Q4: 2021 _SEP-NOV
06
develop+launch
Worked intensely on handoff with the technical product managers, developers, and engineers to construct a back-end system that captured data that was fed from AEM CMS Components to an overarching repository that would be sent to Adobe Analytics for analysis. Ensuring front-end worked with easy authoring of content implementation, AEM components were built in a way to accommodate agnostic structure, and have flexibility for content to change using AEM’s publish system and for the different locale’s available. I helped with design QA and then we were able to launch! A little off schedule, the 2 most visible surveys for content and overall site experience.
The Results
+9%
Overall increase in CSAT score
+26%
survey take rate per month
+50%
saved on survey budget
Q4-Q1: 2022 _DEC-Jan
07
outcomes
Launching the 2 web surveys helped provide a much higher engagement rate and overall validated improvements we were set to making for newer initiatives. The main legacy tool was sunset, and allowed the team to have less work potential updates to the survey content. Half of the budget was saved due to this project and overall accomplished our goal of providing customer service focused insights to improve support experiences.
learnings
Timing is way out of my control and in the future plan in more time for iterations and testing
I didn’t have much requirement writing experience, but asking the experts in product management helped me get over some of that hump
Facilitating stakeholders and cross-functional teams is hard, but can be helped with asking focused, but open ended questions