In the recent Wisconsin budget saga, pundits and politicos on both sides of the debate have attempted to reinforce their positions with public opinion numbers and statistics. Rachael Maddow has led off numerous shows in the past couple weeks with the latest poll from Wisconsin, and Nate Silver has critiqued Rasmussen’s early poll on Walker and unions. This week, the Wisconsin Policy Research Institute added another statewide poll to the mix. Partisans may and have cherry picked numbers out of it like “65 percent think Governor Walker should compromise” or that a majority disapprove of Senate Democrats’ leaving the state. Fortunately, there is more information within the poll release than partisan positioning.
Two aspects of WPRI’s release make it notable over previous polls on the subject; they included two questions in an “experiment to examine responses to different ways of framing a key element of the budget repair bill, specifically collective bargaining, and they released the full crosstabs of their data, revealing the beliefs of different demographic groups. Question framing and wording are often cited as reasons to dismiss poll results when they are unagreeable, though rarely is there a direct comparative sample to see if such dismissals are correct.
The first question in the experiment asked if respondents strongly or somewhat favor or somewhat or strongly oppose the following Walker-friendly statement (let’s call it statement A); “Limiting most public employees' ability to negotiate over non-wage issues in order to prevent local union affiliates from obstructing the budgeting process for local governments.” The results found a statistical tie: 47 percent were strongly or somewhat in favor of the statement, while 50 percent somewhat or strongly oppose the statement.
And yet, when the other half of the sample is asked about favoring or opposing the second union friendly statement (statement B), “Stripping most public employees of their right to collectively bargain over benefits and working conditions as part of a ploy to eliminate public employee unions altogether,” there is a 25-point swing in the favor/oppose margin! Only 32 percent favor the second statement while 58 percent oppose it.
An interesting demographic aspect is the shift in opinion among the college educated between the two statements. 40 percent of those with an undergraduate or higher degree favor the first statement, statement A. When we turn to statement B, where overall favorability fell by 15 points, the favorability of those with college or graduate degrees marginally increased (42 percent). The best educated were more inclined to favor a statement of “stripping … rights to collective bargaining” that is supposed to be engender sympathy for the unions than the Walker-friendly statement. These results suggest that the college educated are more impervious to the phraseology of poll questions.
Gallup discovered a similar occurrence regarding the wording and results in their union questions. On February 21st, they found that only 33 percent of Americans favored the Wisconsin bill “that would take away some of the collective bargaining rights of most public unions, including the state teachers’ union,” and 61 percent opposed it. Less than two weeks later (March 3-6), a question on the same subject found 49 percent favored “changing state laws to limit the bargaining power of state employee unions,” and 45 percent opposed. The 34-point swing between the two Gallup questions is even larger than the swing between the two WPRI experiment questions.
The essential difference in both sets of questions is the wording “collective bargaining rights”; statements with those words in some order suggest clear support for unions, where as questions omitting the words “collectively” or “right” show mixed opinion. Gallup observes that their “differing results likely reflect Americans' sensitivity to nuances in how the debate can be framed. They may also indicate the high and low boundaries of support for setting new limits on collective bargaining.”
Question wording matters, not just the percentages that follow them. So, too, does looking beyond the state level responses to see how different groups react. The certain take away from these polls reaffirms the old adage popularized by Mark Twain; there are “lies, damned lies, and statistics.”
And yet, when the other half of the sample is asked about favoring or opposing the second union friendly statement (statement B), “Stripping most public employees of their right to collectively bargain over benefits and working conditions as part of a ploy to eliminate public employee unions altogether,” there is a 25-point swing in the favor/oppose margin! Only 32 percent favor the second statement while 58 percent oppose it.
An interesting demographic aspect is the shift in opinion among the college educated between the two statements. 40 percent of those with an undergraduate or higher degree favor the first statement, statement A. When we turn to statement B, where overall favorability fell by 15 points, the favorability of those with college or graduate degrees marginally increased (42 percent). The best educated were more inclined to favor a statement of “stripping … rights to collective bargaining” that is supposed to be engender sympathy for the unions than the Walker-friendly statement. These results suggest that the college educated are more impervious to the phraseology of poll questions.
Gallup discovered a similar occurrence regarding the wording and results in their union questions. On February 21st, they found that only 33 percent of Americans favored the Wisconsin bill “that would take away some of the collective bargaining rights of most public unions, including the state teachers’ union,” and 61 percent opposed it. Less than two weeks later (March 3-6), a question on the same subject found 49 percent favored “changing state laws to limit the bargaining power of state employee unions,” and 45 percent opposed. The 34-point swing between the two Gallup questions is even larger than the swing between the two WPRI experiment questions.
The essential difference in both sets of questions is the wording “collective bargaining rights”; statements with those words in some order suggest clear support for unions, where as questions omitting the words “collectively” or “right” show mixed opinion. Gallup observes that their “differing results likely reflect Americans' sensitivity to nuances in how the debate can be framed. They may also indicate the high and low boundaries of support for setting new limits on collective bargaining.”
Question wording matters, not just the percentages that follow them. So, too, does looking beyond the state level responses to see how different groups react. The certain take away from these polls reaffirms the old adage popularized by Mark Twain; there are “lies, damned lies, and statistics.”
No comments:
Post a Comment