April 30, 2001
Better, Cheaper, Faster...
How to Conduct Successful Online Surveys
Whether you’re gauging client satisfaction, tracking association member needs or simply getting feedback on your new Web site, formal feedback mechanisms such as regular telephone surveys or self-mailers can be a costly and time-consuming proposition.
Consider the following chart that we recently used to compare the post-development costs of delivering a five-minute survey and one reminder message to 400 members and prospective members of the Legal Marketing Association's Mid-Atlantic Chapter (LMA):
|Survey Method||Time||Cost||Est. Response
The evidence is compelling. Compared to traditional surveying methods, conducing surveys on the Web clearly saves time and money and increases overall response rates. There is no printing, envelope stuffing, mailing or paying for the return of responses. Invitations are sent immediately, responses are tabulated automatically and results are immediately available on the Web. Once the survey is developed, the only significant expense is the cost of the application. Are you sold yet?
Eight Simple Steps
Following is an eight-step action plan for conducting a successful Web-based survey that Sugarcrest recently designed and used to develop the 2001 program curriculum for the Mid-Atlantic Chapter of LMA. Although its purpose was to assess association member needs and interests, the lessons learned from the LMA survey could be applied to a broad range of online surveying objectives.
To conduct the survey, we used a Web application offered by Zoomerang which cost $199 for one year of unlimited access. Zoomerang's current rates are $350/year (1,000 survey responses); $200 for each additional increment of 1,000 survey responses. If you decide to use the Web, there are several solutions in addition to Zoomerang—most of which offer enough functionality to accomplish your objectives. For a review of several Web-based survey products click here.
1. Plan for Success. The first and most important step in the development of your survey is to create a clearly defined plan. Whether you are working alone or with a group, set aside time to think about what you want to accomplish with your survey. As you begin, remember that the vast majority of online surveys should require no more than five minutes—and preferably less—to complete. Save the detailed questioning for face-to-face meetings. For the LMA survey, we focused on listing the objectives that we felt we could achieve with one, five-minute survey. Our final goal was:
To identify the training and development issues most important to our members so that we could offer a broad range of rich and valuable training programs that address their needs and that have a positive impact on their employers’ bottom lines.
We also developed the following additional objectives:
- To ascertain the interest level among LMA members in various educational programs.
- To understand the type of LMA member (job position, firm type, firm size, etc.) that has attended past programs and that plans to attend future programs. We figured cross-tabulating the results with answers to other questions might help us tailor our program offerings.
- To ascertain the number of programs members have attended in the past year. This would allow us to cross-tabulate and perhaps give more weight to the answers given by participants who are active in the LMA.
- To ascertain LMA member satisfaction with past programs. This would allow us to set a benchmark and accurately gauge the success of next year’s programs.
- To ascertain LMA members’ planned participation in future programs. This would allow us to cross-tabulate and perhaps give more weight to participants who plan to be active in the LMA in the next several months.
2. Identify Participants.
Once you identify your objectives, create a pool of participants (e.g., key clients, colleagues, association members, etc.) that best represent your target audience.
3. Obsess About the Questions. Next, invite your most experienced colleagues or coworkers to participate in a brainstorming session to create a list of potential questions for your survey. Memorialize the results of the session and distribute them to the group for its comments. Thereafter, the results will form the basis for your online survey.
As you began to develop the survey questions, try to keep them simple and the style and grammar consistent and clear. You can use "rating" or "scale" questions to clarify responses and you should also use open-ended questions to give participants an opportunity to freely express their own interests and concerns. By generating both quantitative and qualitative responses you will have the best chance of balancing your desires to gauge current needs and to establish a benchmark from which to measure feedback in future surveys.
Once you have written most of the questions, check to make sure that they are flowing in a logical progression. And, as described below, make sure to test your questions before going live.
4. Select the Right Delivery Method.
Once you select the Web-based delivery method, you have the option of sending the survey via e-mail or of posting the survey on the Web and inviting participants to visit the survey URL. Imbedding the survey in e-mail is often complicated, because it requires you to format the survey for several different e-mail packages. It also gives you little control over the physical appearance of the survey. In addition, with the growing concern among businesses and law firms about computer viruses, you risk limiting your response rate if you send the survey as an e-mail attachment. As a result, it is simpler to have an application service provider such as Zoomerang host your survey on its Web site. You will then send an e-mail invitation to participants and include a link to the survey URL within the body of the message. This option gives you much more control over the survey’s appearance as you can choose from among several attractive layouts and add your organization’s logo to the survey.
5. Test Your Survey Once you develop your survey and select a delivery method, you should run a test to make sure each step in the process is functioning properly. Try e-mailing the invitation and the survey link to a test group of close colleagues or co-workers. Make sure you ask them to give you candid feedback. This review will lead to several small changes that might take minutes to incorporate but that will likely save your participants hours of frustration and increase your overall response rate.
6. Follow these Tips for Getting a Big Response. Even if you design and deliver the best survey in the world, you can’t guarantee that you will get the response rate you’re after. Like anything else, it comes down to paying attention to details—some of the most important of which we list below:
Send your invitation at the right time. The timing of your invitation can have a significant impact on the number of responses you receive. Most survey experts agree that Friday, Saturday and Sunday are not good times. Instead, they recommend that you consider sending your e-mail invitation on a Monday night so that it is there when the participants receive their e-mail Tuesday morning. If you do this, you can expect to receive about 50% of your responses within one day and the vast majority of the rest within four days. Our experience confirms these statistics.
Show your appreciation and clearly explain the purpose of your survey. Your participants are very busy people and they’re doing you a favor by responding to your survey. As a result, in your initial invitation and in all other communications with them, make a point of thanking them and telling them how much you value their feedback. In addition, make sure you clearly explain the purpose of the survey (e.g., “the purposes of this survey are to assess your level of satisfaction with our performance on the XYZ matter and to learn how we can improve our services in the future.”)
Offer an incentive. Offering incentives to encourage response is not always necessary—especially when the survey addresses issues that interest the participants. However, if you do decide to provide an incentive, direct incentives such as discounted fees, free passes to a firm seminar or cash awards are preferred over indirect incentives such as raffles or contests.
For example, while we were confident that the 2001 LMA program offering would garner strong interest, we also had the dual goal of increasing our membership in 2001. As a result, we decided to offer each participant who completed the survey before the deadline, a free companion ticket for one Mid-Atlantic LMA Chapter program in 2001. This incentive awarded participants directly and encouraged them to help us in our membership drive by inviting a friend to one of our 2001 programs.
Be up front about estimated survey completion time. Since your participants’ time is very valuable, it behooves you to properly set their expectations about the amount of time it would take them to complete your survey. Consider building in a buffer by overstating the estimated time by 20% and clearly post this time estimate in both the invitation and in the survey instructions. Take these steps and a majority of your participants will be pleasantly surprised about the time it took them to complete the survey. You will build a degree of trust and credibility with your participants so that the next time you ask for their feedback they will be more likely to participate.
Provide clear directions about how to complete the survey and submit the responses.
While many Web-based surveys are fairly self-explanatory, take time to make sure your instructions are clear. Also, make sure that once participants complete the survey they knew how to submit it to you. Finally, make sure that after their submission, participants are linked to a confirmation/thank you page. It is important to have clear closure rather than risk having participants wondering if their survey was successfully submitted and, as a result, questioning whether they should have spent the time in the first place.
Set a deadline. Incentives with corresponding deadlines help establish urgency and increase your response rate. This is especially true if your survey participants are busy clients. As a result, consider making a limited time offer (e.g. free firm seminar on an important topic, tickets with your attorneys to social and/or sporting events, etc.), to all of those participants who complete the survey by a given date.
- Send reminders. While you might expect to receive most of your responses within the first few days of your initial invitation, it doesn’t hurt to send a reminder seven days before the deadline. This reminder can generate a significant number of new responses and turn an already successful response rate into a blowout.
7. Analyze the Results. Once the survey is complete and all of the results are automatically recorded online, you need to turn the raw survey results into valuable information for decision-making. Fortunately, if you select the right survey application and design a survey with both qualitative and quantitative questions, this step will be relatively easy. You will be able to cross-tabulate quantitative responses and review a number of thoughtful written responses. As a result, you will spot several trends as well as concerns and suggestions that are shared by many of the participants.
8. Present the Results. In most instances, you will want to draft and distribute to others a report summarizing the key conclusions supported by the data. A good online application will allow you to post the report online so that the appropriate managers, colleagues or co-workers can view the results. In any event, your objective should be to present the results in a format that is easy to understand and that effectively conveys your research findings.
Conclusion. As we continue to study the solutions provided by technology and the Internet, we see several opportunities to seamlessly combine our real world needs with helpful virtual tools. In some areas—such as understanding client or member needs and concerns—you can radically redefine the speed and cost of getting quality of results. We hope that by sharing our experience, you will achieve similar results.
This article includes excerpts from an article written by Felice Wagner in the March 2001 issue of Strategies: The Journal of Legal Marketing.
Felice Wagner, a former practicing attorney, is CEO of Sugarcrest Development Group, Inc., a D.C. firm that gives seminars and training programs throughout the country on business development and client loyalty. She can be reached at (202) 462-7046 or firstname.lastname@example.org. Want to see how you measure up as a rainmaker? Take the Rainmaker Reality Check today!