Some college professors leave a lasting impression because of their deep knowledge of the subject matter and their passion for it.
So when I needed an expert opinion last week on social science theory and methods, I turned to Dr. Robert Griffin, one of my former professors who teaches journalism and media studies at Marquette University.
“Dr. Bob,” as my fellow classmates and I used to call him (and still do), has a vast knowledge of survey research in mass communications. He approaches his work with the precision of a social scientist and the curiosity of a journalist.
“You’ve raised some really good issues regarding how Native Americans and some other groups who are smaller portions of the U.S. population have their voices and opinions heard through national surveys,” Griffin wrote in an email message on Thursday.
He was referring to my blog post on Tuesday in which I questioned why a demographic study on Twitter usage did not include Native Americans. The Pew Research Center’s Internet and American Life Project conducted the study. I contacted a project leader at the Pew center and he shared with me an explanation as to why the report did not include Native Americans. I included his comments in my Tuesday blog post, titled “Tired of being left out.”
“What the Pew folks told you was, unfortunately, the straight story for national, general surveys,” Griffin said.
Griffin said that in one sense, the opinions of Native Americans are being heard in national surveys because their individual opinions count the same as every other respondent’s opinion in the sample results. It is when the survey results are divided by ethnicity/race of the respondent that Native Americans and other small sample-size groups “tend to be in the shadows.”
He agreed with Aaron Smith, senior researcher at Pew’s Internet project, that the margin of error in the Twitter study for the subsample of Native Americans, which was plus or minus 26 percent, was too large for the responses to be included. For example, if a result was that 60 percent of Native American respondents favored a certain policy, that margin of error could be anywhere between 34 percent and 86 percent.
“The chance for misrepresentation of Native Americans would be unacceptable” in such a case, he said.
However, he said, that still leaves “the real and important issue of how Native Americans’ voices are heard as a group.”
I agree, and that’s an area I plan to explore.
Griffin said he knows of “no magic potion” to address it outside of conducting a “large and very costly overall sample” of the general population that would proportionately yield an acceptable error margin for Native Americans.
He did offer ideas about alternatives.
“[T]here are techniques such as stratified sampling which, when planned from the get-go, are used to get roughly equal numbers of respondents from various groups, regardless of how large or small the group is in the population. You wind up with enough people from each group to fairly represent the opinions of the members of each group for cross-group comparisons. These techniques are used when the study is specifically designed to compare the views of one group to other groups, especially when one or more groups are fairly small in number. The problems are that this is also a costly procedure and that you tend to lose most of the ability to aggregate these folks’ opinions across groups to make more general statements about the overall population. That kind of approach might be the only solution, outside of sample surveys designed to get opinions just of Native Americans. Whether Internet surveys will help is an open matter, since their success depends on respondents having Internet access and upon using a proper probability sampling technique to eliminate selection biases. There are also some statistical techniques which could increase the precision of the results, but I’m not sure they would help sufficiently. Perhaps some statisticians might be able to shed some other light on other ways to improve the process.”
Griffin encouraged me to follow this line of questioning on surveying Native Americans.
“[Y]our points are very important, and I hope you continue to raise them,” he said.
Thanks, Dr. Bob. I will.
Follow me on Twitter @karenmichel.