Six pitfalls to avoid when designing a customer experience survey
The most common mistakes in customer experience survey design are collecting data with no decision behind it, chasing a perfect questionnaire, ignoring the operational data you already hold, getting the balance between scores and comments wrong, asking at the wrong moment and never testing the survey on a phone. Each one has a simple fix, and the most useful is to write down what you will do differently depending on the answers before you write a single question. A transactional survey should take a customer between one and three minutes to complete.
On this page
Key takeaways
- A good survey starts from the decision it informs, and a question that no decision depends on does not belong in it.
- Keep it to one to three minutes, with one main score, one optional comment on why and the touchpoint passed in from your own system.
- Send it when the experience ends, and complete it yourself on a phone before it goes out.
A customer experience survey exists to tell you what to change, and the mistakes below are the ones that keep coming back across the Qualtrics programmes I have built and rescued, whichever survey tool is in use. I call them the six survey pitfalls, and each comes with the fix and one tip. The first four are the ones I return to most often, and the last two are about when and where the survey reaches people.
1–3 min How long a transactional survey should take to complete, so it can be answered in the moment it is asked.
6 pitfalls The first four come up most often, and the last two are about when and where the survey reaches people.
10 checks The points to tick before a survey goes out.
Where does each pitfall happen?
Every survey goes through the same six stages, and each pitfall shows up at one of them. Read the strip from left to right, use the controls above it to go one stage at a time, or select a pin to jump to its pitfall.
- Decide what the survey is for. Pitfall 1, collecting data without purpose, starts here, before any question exists: nobody has said which decision the answers will inform.
- Design the questions. Two pitfalls sit here. Pitfall 2, striving for perfection, is a questionnaire that keeps growing while the launch slips. Pitfall 4, imbalance between quantitative and qualitative questions, is a mix with too many closed questions or too many open ones.
- Send it at the right moment. Pitfall 5, asking at the wrong moment, is a survey that goes out before the experience has finished, or so long afterwards that the customer is working from memory.
- The customer answers. Pitfall 6, never testing on a phone, shows up here, when the invitation is opened on a phone and the survey was only ever checked on a laptop.
- Analyse the answers. Pitfall 3, ignoring operational data, is reading the survey on its own, without the CRM or web analytics data that explains why people answered as they did.
- Act on what you learn. None of the six pitfalls belongs to this stage, but it is where the fix for pitfall 1 pays off: customers are shown how their input was used and what it changed.
Collecting data without purpose
The pitfall. Gathering data without a clear objective leads to wasted effort and answers nobody uses, because the survey goes out for its own sake and no decision is waiting on the result.
Do this instead. Define what decisions the data will inform and how it will drive action, and keep only the questions that serve those decisions. If nobody can say what will happen when a score drops, the question does not belong in the survey. Then run a feedback loop that shows customers how their input was used and what it changed, which builds trust and makes them more willing to answer next time. The closed-loop CSAT guide covers how to build that loop.
Tip. Always tell customers why their feedback matters and how it will be used, in the invitation itself. It primes them and lifts response rates.
Striving for perfection
The pitfall. Trying to collect every possible data point produces a long and cumbersome survey, and a launch that keeps slipping while the questionnaire is polished.
Do this instead. Put speed and relevance ahead of perfection. Launch a minimal viable survey, learn from the first responses and improve it in rounds, because a concise, well-focused survey gets better response rates and quicker answers you can act on than an exhaustive voice-of-customer programme that arrives late. The same logic runs through the guide on whether to build the whole thing or start with an MVP.
Tip. Put yourself in your customer's shoes. A transactional customer experience survey should take between one and three minutes to complete.
Ignoring operational data
The pitfall. Treating the survey as the only source of information and not joining it to operational data such as the CRM or web analytics, which can segment customers and explain why they answered as they did.
Do this instead. Use what the organisation already holds about the customer, meaning their interactions, history and demographics, to segment the audience and tailor the questions, so you are not asking people things you already know. The result is a richer, more contextual picture of the customer journey than the survey can give on its own.
Tip. At a minimum, always record the touchpoint a response relates to, such as a webinar, a car purchase or an in-shop visit. Pass it into the survey from the system that sent the invitation, so the customer is never asked to tell you which one they mean.
Imbalance between quantitative and qualitative questions
The pitfall. Too many closed questions limit the context you get back, while too many open questions add effort for the customer and make the answers inconsistent to analyse.
Do this instead. Use closed questions for clear, comparable measures and open questions for detail and context, and pair each main score with one open question that asks why. The score tells you where to look and the comment tells you what to fix.
Tip. Make the open question optional. Nobody likes being forced to type, and forced comments invite snarky answers.
Asking at the wrong moment
The pitfall. Sending the survey before the experience has finished, so the customer is judging something half done, or so long afterwards that they are reconstructing it from memory.
Do this instead. Pick the moment the experience actually ends, such as the ticket being closed, the order arriving or the session finishing, and send the survey from that event. Write the trigger down so it is a decision someone made.
Tip. Finished is often later for the customer than it is for your team, so check what the end looks like with someone who deals with customers every day.
Never testing on a phone
The pitfall. Building and checking the survey on a laptop only, when many customers will open the invitation on a phone.
Do this instead. Open the survey on a real phone, on a mobile connection, and complete it from start to finish before launch. Look closely at long rating grids, the open-text box and the buttons, which are the parts most likely to be awkward on a small screen.
Tip. Ask someone who was not involved in building the survey to complete it on their phone and tell you how long it took. That also checks the one-to-three-minute target.
What does a good customer experience survey look like?
A survey that avoids the six pitfalls is usually small. It is sent when the experience ends, it tells customers why they are being asked, and it carries one main rating and one optional comment on why. The touchpoint and customer details come in from the system that sent the invitation.
In the survey
- Why the customer is being asked, in the invitation
- One main rating
- One optional comment on why
From your own system
- The touchpoint the response relates to
- The customer details you already hold
- The moment the experience ends, as the trigger
Everything beyond that has to earn its place by informing a named decision.
Is your survey ready to send?
Check a draft against these ten points before it goes out, and fix anything you cannot tick.
Your ticks are saved in this browser only. Nothing is sent anywhere.
What do these pitfalls look like on real projects?
Three projects on this site show what changes when these problems are fixed.
For an Australian nonprofit, survey responses used to reach the CRM through a weekly manual export and import, which meant a one-week delay before new contacts received a survey and another before their answers reached the contact record. Now each response lands on the right record as soon as it arrives, across 120k+ yearly contacts, so the answer sits next to everything else the organisation knows about that person.
Read the case study Case study · Reading the answers A custom research platform in under a weekA US research organisation ran a recurring survey across nearly 9,000 towns, and the raw exports had thousands of columns and took hours of manual work before anyone could read a result. A repeatable data transformation model, part of a platform delivered in under a week, now turns each export into analysis-ready data in minutes. What I take from it is to plan how the answers will be read when you design the questions.
Read the case study Case study · Acting on scores Vendor CSAT from 2.71 to 4.5On the vendor-managed projects I stabilised for a professional services organisation, average project CSAT rose from 2.71 to 4.5 within two months. Two of the six changes behind it were daily measurement of project KPIs and weekly feedback loops with the vendor's management.
Read the case studyA score only matters when someone acts on it. How to build that loop for your own customers is covered in the closed-loop CSAT guide.
Questions people ask
How long should a customer experience survey be?
Short enough to finish in the moment it is asked. For a transactional customer experience survey, that means between one and three minutes. If yours runs longer, cut the questions that do not inform a decision, launch the short version first and improve it from the responses.
How many questions should a CSAT survey have?
There is no fixed number, because what matters is the one to three minutes it takes to complete. In practice that usually means one rating question and one optional open question asking why, with a few more only if each informs a decision. The touchpoint should come from your system and not be asked of the customer.
When should a customer experience survey be sent?
At the moment the experience ends, such as when a ticket is closed, an order arrives or a session finishes. Too early and the customer is judging something unfinished, and too late they are working from memory. Write the trigger down so it is decided in advance.
Should open-text questions be required?
No. Make them optional, because forcing people to type frustrates them and invites snarky answers. Pair each main score with one open question that asks why, so the people who want to explain have a place to do it.
Do I need a CRM to run a customer experience survey?
No. You do need to know which touchpoint each response relates to, such as a webinar, a purchase or a visit to a shop, and recording that is the minimum. A CRM or web analytics data adds customer history and demographics to the answers, which lets you segment and tailor the questions, and it is worth adding once the first round has run.
More resources
A note from me
Designing or rescuing a customer experience survey?
If you would like a second pair of eyes on a survey before it goes out, send me a message or book a short call.