Science of Surveys

As graduate student on the Snapshot Wisconsin project, part of my role is to help the team better understand their volunteers and conduct research that will assist with program improvement.  One way I do this is by surveying trail camera hosts when they enter the program and after they have been participating in Snapshot Wisconsin for one year.

Developing a survey takes more work than you might expect!  Some things, like age or occupation, are relatively easy to measure.  However, abstract concepts like satisfaction or attitudes are much more difficult to capture in a survey.  These abstract concepts must be measured in more indirect ways, and typically social scientists develop a number of survey questions or items to measure a concept.

For example, let’s say I wanted to measure someone’s job satisfaction through a survey.  You could ask, “How happy are you overall with your job?” (Rate 1-5).

Surveys ready to be delivered to Snapshot trail camera hosts

In order to capture more aspects of job satisfaction, it would be better to ask: “How happy are you with each of the following parts of your job?  Autonomy, work load, salary, coworker relations, etc.” (Rate each 1-5).

Bear with me while I get theoretical for a moment…

Imagine you have a whole universe of survey items you could ask someone about job satisfaction.  If you choose just one question to ask them, that question is not likely to be a good representation of their job satisfaction as a whole.  However, if you ask them multiple questions, you get a much better representation of their job satisfaction.

Let me use an analogy.  If I want to know all the different species of mammals found in a particular county and I put out just one trail camera in that county, it isn’t likely to be sufficient.  I put out a whole bunch of cameras across the county, I’d get a much more accurate count.

Often, I get this question from people who take surveys: Why do some of these survey questions seem so similar to one another?  Can’t you ask this with just one question?

The answer is: if we are asking about an abstract concept in a survey, assessing it indirectly though multiple questions is the best way to go if we want valid scientific results.

Through email and the internet it is so easy to deliver surveys and if you are like me, you get a survey in your inbox from some business or organization just about every week.  Hopefully this sheds a little light on what goes on behind the scenes before you get that “new mail” notification.

For those of you who have completed a Snapshot Wisconsin survey, your responses are truly valued.  We are learning a lot; see here for some early results and keep your eyes on the blog for more. If you are interested in learning more about the science behind surveys, let me know in the comments!