Qualitative data in CRO: Expert tips and advice 

Data is absolutely essential for a successful CRO campaign. Often, it is all too easy to get drawn into using only quantitative data to analyse your A/B tests and areas of weakness on your website. Focusing on quantitative data alone, however, could mean you miss out on crucial, social insights in your results.

When you are working up close with the material you are testing, it’s sometimes easy to overlook details that may be causing first-time users and potential customers problems when completing your desired conversion actions. Gathering qualitative data can reveal areas where user experience is lacking.  

Easy Ways to Gather Qualitative Data for CRO 

Session Replay 

Many CRO and web analytics tools have session replay features which allow you to take a step back and look at your site from a different perspective; the user perspective. 

With session replay you can identify exactly where users are landing, where they face frustration points and where they exit your site. Where quantitative tools report on bounce rates and engagement time, watching your users navigate your website can show you exactly where they hesitate, helping you answer the question “why?” and identify patterns in behaviour.

Some good times to utilise session replay are:

• After a feature release
• During and after a high impact A/B test
• When deciding on your testing strategy roadmap

Once you’ve found areas of weakness in your session replay you can use quantitative data from the same pages to ideate on ways to improve user experience such as through A/B testing or implementing new features. 

Surveys 

Surveys are excellent for gathering data to aid with high level CRO testing. Users are usually more than willing to share feedback in a survey, especially if incentivised with something like a discount or gift card. A benefit of surveys is that they can be placed almost anywhere on your website from the home page to site exit and so are a powerful tool to target users in areas you’re looking at improving. 

Some common questions for long-form responses are:

• Did you find what you were looking for?
• Did you encounter any issues using the site? 
• Do you have any suggestions on how we can improve? 

After setting a survey live on your site, you can check in on the data periodically, using the responses to aid ideation workshops and prioritisation of your testing plan to match user needs.

Additional questions can be included to gather data on where leads are coming from outside of trackable marketing channels and social media. Limiting responses to a scale, 1 to 10 or similar, can reduce the time needed to analyse data collected and provide a quantitative baseline for comparing notes after changes are made. 

Post purchase, for both software and ecommerce, surveys can be a valuable opportunity for customers to suggest improvements or review products and services after the initial conversion event has occurred. Such feedback data can then be used as testimonials, encouraging trust signals and a higher site conversion rate. 

User Testing 

Focus groups are one of the most powerful tools for CRO – even if getting a focus group together can be a daunting task. User testing during focus group sessions is the best chance to directly observe how your target users interact with your website or software, removing unintentional bias that can occur from internal usability testing. Journeys that appear seamless and intuitive to your team may not translate well to prospective customers, so user testing is imperative during periods of redesign or re-platforming. 

Your focus groups don’t need to be large to be effective, one well conducted session with five participants can provide enough feedback to facilitate multiple A/B tests. Larger sessions are still beneficial but over 15 – 20 participants can lead to diminishing returns and limit the one-on-one observation time you need for quality data collection. 

When you set tasks you set for your focus group, keep these clear and concise. Three to five tasks is typically a good number. Avoid over-explanation to allow for natural browsing and user behaviour. Also, make sure question-and-answer sessions happen after users have completed tasks as the focus of user testing should be observation.

Standard user testing should include:

• One or two specific tasks
• A broad task
• A completion of the full funnel

Leveraging Both Quantitative and Qualitative Data in CRO

Qualitative data gives you pivotal context to your CRO testing results and reasoning behind any poor conversion rates. It’s easy to think of your strategy as being data driven with web analytics tools alone but without qualitative insights, you won’t get the full picture of test results and baselines.

Your CRO strategy should leverage both types of data – qualitative data paired with the relevant quantitative data – to fully understand how to improve your conversion rate and user experience effectively. Integrating a qualitative approach into your CRO roadmap with a balance of user testing, surveys and session recordings will yield the best conversion improvement results.