'Hey, kids, let's do a poll!'

"Hey, kids, let's put on a show!" Mickey Rooney would say to Judy Garland and others as part of the classic old film formula that was a box office bonanza.

It started with "Babes in Arms," about teens who save their vaudevillian families from ruin by putting on a show. Three more musicals with the same theme followed and made Garland a star.

"Amateurs rule" was the message.

These days, the theme seems to be, "Hey kids, let's do a poll!"

Yesterday it was St. Norbert College providing the fodder for political spin across the state, while offering up numbers that meant almost nothing.

Today it's UW-Milwaukee, with its political poll that uses an interesting methodology.

One problem is that the survey was stretched out to 15 days. If a poll is supposed to be a snapshot of opinion, it's best done in 3-4 days. The UWM poll -- and the St. Norbert poll, which stretched over 10 days -- are blurry photos at best.

But this is what should set off alarms:

In the survey of 471 Wisconsin residents who intended to vote, 44 percent indicated they would vote for Doyle, while 33 percent said they would choose Green, if the gubernatorial election were held today. But 21 percent had not made up their minds, and 107 respondents said they probably would not vote.

It is customary, when doing a poll to try to find out what's happening in an election campaign, to ask people who actually intend to vote.

If 107 people said they probably would not vote, they should have been excluded from the survey to begin with.

When almost 20% of your sample consists of non-voters, it will skew all of the results. The survey used likely voters in the governor's race matchup, which showed Doyle 11% ahead, and in asking about the amendment to ban gay marriage and civil unions, which was favored 53-37. But the other questions used the whole sample. For example, 22% said they don't know whether they like the job Doyle is doing as governor. Those are people who aren't paying attention. and probably are among the non-voters.

The problem is that the news media treat all polls much the same, although there is a wide range in their accuracy and credibility. And the campaigns add to the confusion, twisting them to their advantage.

The most accurate polls, of course, are the ones you never hear or read about, that are done by professional political pollsters for the campaigns themselves. They are used for strategic purposes, not publicity. To campaigns, the horse race numbers are less important than the issues and messages they test, to find out how to move voters in their direction. When you see a poll that's leaked by a campaign, pay attention to whether the campaign has released the entire poll or just one or two questions that make their candidate look good. To be able to judge its credibility, you need to see the context, methodology and questions they asked.

Next best are those done by professional polling firms for media outlets. You'll see many of those done for national media outlets, but few in Wisconsin. In those cases the numbers are mostly reliable. One variable is the analysis -- does it come from an objective polling firm, or from a a political reporter for the media outlet which sponsored the poll?

A friend in the business says: "National media polls are usually pretty good because the sponsors have the resources to pay to do them right. (Often they combine with other news outlets to share the cost like NY Times-CBS, ABC-WashPost, or NBC-Wall St. Journal.) But local media polls can be on varying quality because they don't have the resources to spend and it is a business where it costs more to do it right: like using trained/experienced interviewers, calling back to get representative samples instead of whomever is home, interviewing large enough samples. Unlike some other states, Wisconsin does not have a history of a reliable statewide media poll and that's why these university polls gain so much attention."

Other tips from a longtime pollster:

Be wary of people who say they are doing it without a client (like Strategic Vision, whom the GOP will cite all year). If they are doing it right, it is a pretty costly proposition to do it for publicity purposes. Either someone is sponsoring it in some indirect way, or they are cutting costs.

Be wary of polls like Zogby that take internet volunteers--the cheapest of all polls.

Xoff's bottom line: Don't believe anything you hear, and only half of what you see. And, as my grandfather used to say in bygone days, "Never bet against the Yankees, Notre Dame or the Democrats."