We have been receiving a growing number of JIRA tickets about improving the user experience of the existing survey invitation process within Survey Designer. Therefore, the need of redesigning the entire invitation process has been prioritised during Q1 2022.
Gathering requirements:
We have been interviewing representative users of Survey Designer with a focus on their workflow during the setup of the email invitation. The goal of qualitative research is to gather insights that drive decision-making, especially when the measurement is unfeasible or impossible. Our aim is not to collect statistically significant findings but to uncover patterns and latent needs during the interaction with the system. This is the synthesis of our findings and a selection of the most relevant quotes, addressing these themes
Users are stressed during the phase of email invitation due to working under time constraints or managing long respondent contact lists with several conditions
Testing the email seems to be the only way to quality check layout and content, giving confidence to the user of an optimal execution
Saving draft emails allows the users to review, improve, check and internally test their invitation.
There is noticeable different workflow between Market Research and Voice of Customers.
Mapping out the existing user journey
Mapping out the stories of Voice of Customers and Market Research
The goals:
To provide better and smoother user experience of the invitation process for different lines of business.
To remove complexities, such as creating invitations for continuous respondents upload and creating schedules for different time zones.
To increase confidence to users by improving the previewing & testing capabilities.
To eliminate the need of using Professional Authoring (the legacy survey building tool).
Design workshop
A 2-day design workshop was held in the Oslo office. Participants includes service and support, front-ends, back-ends, product designers, and project managers. Based on the insights we gained from the user interviews and the JIRA tickets, several ideas were being explored.
Through users research, we discovered that our users are sending the survey invitations to different target groups with different look and feel. The email management in Survey Designer today is a flat list of all the emails sorted by date created, due to the absence of a hierarchy, it is very difficult for the users to manage.
The core idea we picked to move forward is a campaign concept, that can be used to manage all aspects of the survey invitation process.
What is a campaign?
A Campaign consists of the following aspects:
Invitation email or SMS definition
Email or SMS reminders (reminders are optional and possible to create multiple reminders)
Filter criteria & conditions to define target audience
Schedule settings + a graphical timeline
Delivery reporting (ideally with the ability to see aggregated figures for the entire campaign as well as the results of specific invitation batches)
Though Invitation SMS is a popular request, due to limited time frame and resources, it is out of the scope at this phase.
Mapping out the campaign workflow
Design Validations
7 participants were invited for the usability tests. 2 of them are internal stakeholders and the rest are participants who participated the users interviews. Participants were asked to perform a list of tasks using a hide fidelity clickable prototype.
Significant amount of user feedback was collected during the usability tests.
The solution: invitation campaign
After uploading respondents to the system, users would be guided to the campaign page.
The campaign edit page
The activity of email invitation to a survey is perceived as one of the most stressful steps in the user’s journey.
They may find themselves collecting millions of data points and the action of sending the invite is potentially addressed to a very high number of respondents. Any mistake or oversight may harm not only the outcome of the research but expose the users themselves to difficult situations to manage.
The goal of this page is to give users an overview of the important assets and their properties of an a campaign. By validating the important properties (the green checkmarks), giving confidence to the user of an optimal execution.
Defining the target audience
Sending the survey invite to a very high number of respondents is considered one of the stressful moments during the journey. Allowing users to verify and preview the respondents while building the condition criteria, provides users sense of security.
Designing the email message
Instead of letting users starting from a blank email, we will be providing a default look and feel that users can work from as a starting point (and this should work well in commonly used desktop and web email clients + mobile Apps).
There also have been considerations of introducing a reusable email template concept and a more intuitive email builder or wysiwyg editor. Due to the limited time frame and resources, it will be addressed at the later stage.
Scheduling an email
In the context of Voice of Customers, users have the need to include respondents who are being continuously uploaded to the system. An automation was designed to cater for those use-cases. It allows users to automatically include new respondents that meet the target audience criteria. For example, if we are sending a survey to people who have made a purchase 3 days ago. They will be automatically uploaded to the system through FTP as new respondents. This step is handled in the respondent upload page, which is out of the scope of this project. By enabling the automation, the new respondents will be automatically included in the campaign.
Setting up an reminder
Reminder email reminds people to take the survey. It shares the same target audience and the same look & feel with the invitation email. When creating a reminder, it creates a duplicate of the invitation email with extra reminder settings, e.g. to send the reminder 7 days after the invitation to people who did not start or complete the survey, users are also able to specify which days of the week the reminders should be sent if the users have set up the automated invitation.
Campaign statistics
Once a campaign has been launched and has started collecting data, users will get a dashboard of the campaign statistics, which includes statistics of how the survey is performing on a campaign level as well as the email deliver statistics. A list of email deliveries helps user monitor and manage the email deliver history. A list of all the messages within the campaign allows users to review and make changes to the email messages without changing any of the settings.
