Survey Says: How a Simple 2-Minute Survey Can Improve a Web Project’s Effectiveness

A couple of years ago I was confronted with a website I had always wanted to re-develop. I knew the organization well – a local fitness center – and I was all too aware that their existing website was complete garbage. Nobody liked the current site. Not the staff, not people in the community, and not their clientele. And so when I was approached with the opportunity of re-doing everything, it was as if I were a moth and somebody switched on a thousand light bulbs. I was attracted to everything – and everything beckoned me – screaming out for my attention. I wanted to change it all.

And then it hit me. I was directionless.

I was directionless not in the sense that I didn’t know what to do. I was directionless because I was looking at too many directions. I wanted to focus my energies on everything, and it left me dry and lightheaded. I needed guidance.

The staff of the fitness center was helpful. They provided input on new features and updates, but in the end, they were just as clueless as I was.

The only obvious place to look for answers was the clientele – the people who visited the fitness center. But how do you access these people? The answer was easy. A survey.

Surveys may be blunt instruments, but they’re excellent at gathering lots of feedback quickly, and that’s exactly what I needed. Thankfully, the staff had marketing data available that greatly helped in crafting the survey, and we were able to collect responses via email, the current website, and in person.

The people who answered the survey are the ones who gave me the direction I sought. Their replies to questions and indications of preference told me not what I wanted, but what they – the ultimate end user – wanted. I was able to breathe a sigh of relief in looking at their answers, because I knew I didn’t need to go chasing after all those light bulbs.

You can’t just build any old survey

Surveys are simple in principle. They ask a series of questions, ranging from multiple choice to essay boxes, and the respondent fills in their answer.

But surveys are not effective in principle. Effective surveys tell you things you didn’t know. Ineffective surveys tell you things you already knew, thus making them useless. So avoiding useless, ineffective surveys is essential – or else you risk wasting everyone’s time.

Taking stock of what you already know is a good first step. If there’s an existing website that you’re redesigning, as was my case, then there’s going to be (there better be) data surrounding that presence, which could include analytics, previous surveys, marketing research, and tons of other good stuff. Sifting through this information, no matter how poor, will help you understand what’s missing. Think of it like a jigsaw puzzle. The existing data are the corner and edge pieces. Sure, those are the easiest parts of the puzzle, but without them, you’d have a hard time completing the picture.

Here are some other good sources of existing data:

  • Website statistics
  • Old surveys
  • User testing results
  • Form submission data
  • AdSense or AdWords performance reports
  • Staff knowledge
  • Marketing research
  • Comment or complaint boxes

Asking the right questions

When you know what you don’t know, as odd as it may sound, then you’re ready to start building a list of questions you want to ask end users. There’s no cut and dry formula for figuring out the best questions to ask – your intuition is going to drive what you put down on your list.

Your intuition is also going to be influenced heavily by what you discover from research. You’ll likely find anomalies or odd trends that need explaining. Or you could find old questions that have been asked before that need to be asked again. And then your experience with similar and past projects will help fill in the rest. What did you learn from these projects that might benefit this one?

In fact, a good practice to begin following is to maintain a sort of diary for your completed projects. In this diary you would notate what went wrong, what went right, and what you took away from the whole process. As you accrue more and more entries in this diary, you can begin to develop a mental or written checklist of questions that need to be addressed in every web project.

Collecting answers

Collecting responses to a survey is tricky. There are different audiences you need to take into account or else your results will have skewed results.

For the fitness center, there were two distinct sets of audiences that needed to be considered. The first group included those who checked the current website often, and the second group included those who did not visit the website. Why is it important to make this distinction? Because the first group is going to have a different perspective than the second group. They’ve used the website more often and know its strengths and weaknesses. The second group probably doesn’t know much about the website, and they’re going to have a unique perspective.

There are other distinctions in the audiences that may need to be identified as well. Are there subsets of savvy and non-savvy computer users? Are there different geographic areas in which audiences reside? Are there multiple physical locations that the end user visits (such as multiple fitness center locations)?

This is all essential to the collection strategy in that you’ll need to devise multiple collection methods. In the case of a physical client location, there may need to be a paper version of the survey that can be handed out. Otherwise, you’ll miss a segment of the audience (those who don’t use the website, non-savvy computer users, those who visit a specific location, etc).

Here’s what I recommend when thinking about collection methods:

  • Online survey using SurveyMonkey, which could be delivered via the current website or through email (establish new links for each media you test).
  • Paper survey that staff could hand out on location.
  • On-site computer survey that patrons could complete.
  • Direct mail surveys.
  • Telephone surveys.

Making sense of it all

After casting your line out there and getting a few bites, the toughest part of this whole process then begins: interpreting the results.

Your survey might be completely quantitative, where doing a statistical analysis would make sense, or it could be completely qualitative, in which cause you’ll have to sit down and pore over the data. But you’ll more than likely have a mixed survey of both quantitative and qualitative responses.

In most of my surveys, I use quantitative questions to discern demographics and other data that I can use to categorize and better understand the respondent. The qualitative questions, such as comment boxes, is where I focus the majority of my attention.

The key is to look for patterns. Are there consistent patterns in what people are saying? For example, does the audience, regardless of the demographics, desire the ability to download an application from the homepage? These patterns are going to be the most important areas to hone in on.

My recommendation is to read through all of the responses, whether in electronic or printed format, and then let it sit for a while in your mind. After a couple of days, revisit the responses and begin compiling pattern data in a spreadsheet application. Rank the importance of these items based on the number of times respondents mentioned them. Then go back over the surveys and pick out any outlier data (e.g. creative input that no other respondents shared) and notate them in a sort of wish list document.

Making time for surveys

Setting aside the time and resources for an initial user survey can seem daunting – even to me. I don’t include the process in all of my projects.

However, collecting end user feedback can save time down road when you hit snags in deciding which features to incorporate into the website. And even better, the feedback users provide will ultimately create a better experience once the site is launched, which in turn, will result in a more effective website for your client.

If you have a client with easily identifiable groups of end users to survey, push them toward adapting the process for the project. Yes, there should be an added cost in your proposal – a survey, while it may seem simple, is really no easy task – but also highlight the benefits of understanding the people who will be using the website. These benefits should speak for themselves, but some clients need a little handholding. Here are a few examples:

Understanding end user behavior will help us…

  • Design a website that eliminates needless clicks and pages, which will lower the barriers to completing a transaction.
  • Provide the information and features users demand, which will lower the amount of phone calls and emails to support.
  • Brainstorm new ideas for the website that could lead to new income opportunities.
  • Cut the fat from the website to make it a lean, simple presence that clearly communicates the organization’s message.

Those are some generic benefits, but you get the point. It’s time to stop throwing darts in the dark – successful websites depend on user feedback, and the survey is one of the best tools for accomplishing that.

Have a question or comment about this post? Drop me a line!