Why polling may not be reliable

Speaking of polls . . . their days may be number — at least how they are conducted now. Jay Leve, founder of SurveyUSA, said that polling depends on a set of assumptions: “You’re at home; you have a phone; your phone has a hard-coded area code and exchange which means I know where you are; æ.æ.æ. you’re waiting for your phone to ring; when it rings you’ll answer it; it’s OK for me to interrupt you; you’re happy to talk to me; whatever you’re doing is less important than talking to me; and I won’t take no for an answer — I’m going to keep calling back until you talk to me.” But, of course, those assumptions are becoming more and more disconnected from reality in our busy, increasingly cell phone-oriented society.

25 Comments

  1. JWink
    Posted August 12, 2009 at 6:28 am | Permalink

    If only people with hard-wired telephones are being polled and not cell-phone users … it would appear to give a bias to polling results.

  2. Rage
    Posted August 12, 2009 at 7:19 am | Permalink

    There are plenty of reasons why polling might be unreliable. That’s why you don’t look at one or two polls. And even then, I reminded of (ahem) the assured re-election of Governor Hayden, and, more recently, the certain victory of Howard Dean in Iowa (Yeah!). In latter instance the polls wildly shifted in about the last week.

    On the other hand, when Dubya’s numbers hit the deck, they stayed there.

    So polls have some use but caveat emptor.

  3. XXX
    Posted August 12, 2009 at 8:08 am | Permalink

    As the Cons used to be so fond of saying, polls don’t matter.
    {sarcasm off}

  4. littlejohn
    Posted August 12, 2009 at 8:15 am | Permalink

    Of course they are unreliable. They show resistance to the Plans of President Obama in many cases.

    That little thing being said, I have long believed that that polls are basically unreliable. They us (inmy opinion) too few of numbers, they are easily skewed by whom they are asking (see above), they are easily skewed by the way the questions are phrased —intentionally or otherwise—- and sometimes reflect the news of that day. They are only a snapshot That being said, continued polling by various organizations can be useful in showing trends in the mind of the electorate, but I don;t really think any one poll is much useful about anything

  5. DorisKing
    Posted August 12, 2009 at 8:20 am | Permalink

    So will the chorus of hosanas cease whenever Obama’s poll numbers drop a point?

  6. Posted August 12, 2009 at 8:26 am | Permalink

    I am a conservative and I always have and always will say polls don’t matter. Not only have I never been called, I don’t know of anyone that has.

    In addition, polls in themselves can be biased. If you want a certain outcome, it doesn’t take a genius to ask the questions or not ask the questions to make sway the voter to answer the way you wish. Such as who would you rather have as the president, Obama or Clinton when the person answering the question would rather have neither.

    Then there are the mathematics of statistics. Have you ever taken that class? It doesn’t take much to use the formula needed to make the polls come out in your favor.

    I have written computer prorams that forced the outcome needed. Hmmm? Perhaps that is how they proved global warming, you suppose?

  7. Regular
    Posted August 12, 2009 at 8:30 am | Permalink

    I don’t have a cell phone, don’t want the bill and I’m not much of a conversationalist in a car or while shopping.

    I get a few ethnic-based phone calls looking for opinions. I suppose since I’m in a predominately African-American neighborhood, they must think I fit some sort of demographic.

    Most of the time, I answer polls – other times I say no and hang up – rude, crude and socially unacceptible, but it conserves time.

  8. littlejohn
    Posted August 12, 2009 at 8:30 am | Permalink

    “So will the chorus of hosanas cease whenever Obama’s poll numbers drop a point?”

    Probably not. But I don;t lend my voice to the crowd.

  9. GMC70
    Posted August 12, 2009 at 9:02 am | Permalink

    Why polling may not be reliable

    =====

    Was it ever?

    Most polls were really about reflecting public opinion. They are instead about attempting to shape public opinion by carefully choosing the question and it’s form or the demographic to be questioned. They are primarily out to reaffirm or promote a preexisting point of view, not to “discover” the “public’s” views.

    They also reinforce the misperception that there is a single “public” or “American people.” One of my pet peeve phrases is “the American people” want X or Y. Folks, there is no such thing as THE American people; there never was. There are lots of varied interests and points of view. To say that any single perspective reflects “THE” American people is inherently faulty.

    And I’ve said before and will again – any leader who consults polls prior to making decisions isn’t fit to lead. Period.

  10. DorisKing
    Posted August 12, 2009 at 9:39 am | Permalink

    Never mind that they are accurate predictors about 99 percent of the time.

  11. Monkeyhawk
    Posted August 12, 2009 at 10:05 am | Permalink

    “littlejohn,” “TheBorgHunter,” and “GMC70″ whine how polls are “unreliable” when majority opinion disagrees with them.

    Unlike “littlejohn,” “TheBorgHunter,” and “GMC70″ I have actually worked closely with research and polling companies. We use it all the time in advertising and marketing and broadcasting and I’ve worked a few political campaigns.

    The huge percentage of public opinion research has nothing to do with politics. And it would be in no pollster’s best interest to develop skewed numbers.

    Now, a propagandist such as I can take that research and cherry pick results.

    Bill O’Reilly may have more eyeballs looking at the screen than Keith Olbermann, but B-O’s eyese are old and poor and stupid and they’re likely closed as his audience nods off into their drool cups. “Countdown’s” numbers dominate advertisers’ target demographics and earn the hour more bucks.

    Any one poll and the result of any one question is essentially meaningless. You have to look at the internals.

  12. Pedant
    Posted August 12, 2009 at 11:18 am | Permalink

    DorisKing
    Posted August 12, 2009 at 9:39 am | Permalink
    Never mind that they are accurate predictors about 99 percent of the time.

    Not sure it’s that high, not for the polls commissioned for newspapers or the public interest. They don’t pay enough. Can polling lead to that level of accuracy? No doubt it can, absolutely.

    Just as in finance (where valuable knowledge gained is rarely shared publicly), there are far better polls conducted by politicians’ staffs and that are not shared with the public. For example, look at Karl Rove and his last-minute prediction that John McCain would be elected president last November. I am sure Rove knew better (in fact, I’d bet he knew exactly what the final numbers would be, something approaching ±100,000 votes in the national aggregate).

    Americans who are fighting uphill battles love to disparage newspaper and/or public polls, but the fact remains that every politician and every political activist watches some poll — probably their own or one they privately commission — like a hawk.

    Polls have been, are currently, and always will be at the heart of American democracy.

  13. okobserver
    Posted August 12, 2009 at 12:18 pm | Permalink

    MSNBC and CNN’s primetime news programs have suffered an incredible nosedive in ratings since Barack Obama was elected president, Newsmax has learned.

    Ratings for “Countdown With Keith Olbermann” on MSNBC at 8 p.m. have plunged a dizzying 42 percent since October, shortly before the election. CNN’s 8 p.m. show, now being hosted by Roland Martin, has seen a 49 percent plunge over that time period, while “The O’Reilly Factor” on Fox News has dipped only 15 percent, according to Nielsen Media Research.

    Bill O’Reilly’s 8 p.m. show remains the top-rated cable news program, averaging 2,650,000 million households per night in April, more than the combined totals for Olbermann’s program (938,000) and CNN’s 8 p.m. offering (613,000) — with more than a million households to spare.

    In the key demographic of viewers 25 to 54 years old, Olbermann has lost 53 percent of his average nightly audience, a precipitous plunge. Rachel Maddow, who follows him at 9 p.m. on MSNBC, has lost an astounding 65 percent of her 25-to-54 audience since October, and her 819,000 households in April compare poorly to Sean Hannity’s 1,953,000 households on Fox.
    ————————-
    Monkey you are so full of hot air I’m surprised you can stay earth bound.

    I have purchased plenty of advertising by looking at the Aubitron ratings and I know that looking at the numbers above I would put my ad dollars on Oreilly.

    Wishing it were so doesn’t make it so.

    Personally I have never thought polls were an indications of public sentiments. The number of dissatisfied voters who are showing up at town hall meetings do.

  14. GMC70
    Posted August 12, 2009 at 12:18 pm | Permalink

    Unlike “littlejohn,” “TheBorgHunter,” and “GMC70? I have actually worked closely with research and polling companies.

    ===

    Of course you have. MH, with all due respect (and that’s not much) if you told me the sky was blue I’d have to check it myself. It all depends on what you want of your poll; and political polls – the ones routinely reported – are meant to drive opinion, not measure it. That’s true on all sides.

    However, this is more troubling:

    Polls have been, are currently, and always will be at the heart of American democracy.

    Perhaps; but they are a large part of what is wrong. Any leader who checks the polls to tell him what course to take isn’t fit to lead.

  15. Pedant
    Posted August 12, 2009 at 12:40 pm | Permalink

    GMC70
    Posted August 12, 2009 at 12:18 pm | Permalink
    However, this is more troubling:

    Polls have been, are currently, and always will be at the heart of American democracy.

    Perhaps; but they are a large part of what is wrong. Any leader who checks the polls to tell him what course to take isn’t fit to lead.

    You are describing a fork in the road, a choice by an elected politician between course A and course B. That poll results are the sole criteria in such a decision is not something I wrote, and it’s not what I believe.

    I suspect you know that, too.

    That said, any politician who plans on standing for re-election will check his or her own polls, always. Their subsequent decision may fly in the face of the poll and even their own re-election, but my point is that they’ll check it anyway.

    In that way polls are at the heart of American democracy.

  16. Monkeyhawk
    Posted August 12, 2009 at 12:46 pm | Permalink

    “okobserver” crows –

    “I have purchased plenty of advertising by looking at the Aubitron ratings and I know that looking at the numbers above I would put my ad dollars on Oreilly.”

    Yeah, well. You’ve purchased some ads. All I’ve ever done is sell advertising, produce advertising, research markets, develop strategies and tactics and marketing plans while working with highly-trained and well-paid media analysts, negotiators, and buyers.

    But you’ve seen an “Aubitron” book!

    Trumps me.

    I’ve been out of the radio business for a while so maybe you can refresh my memory about the differences and application of such things as:

    AQH Persons,
    Cume Persons,
    Rating (Cume and AQH),
    Share, GIs, GRPs,
    Cost Per Rating Point,
    CPM,
    Exclusive Cume,
    Net Reach,
    Frequency,
    GI,
    TSL,
    Metro,
    TSA,
    and DMA.

    It’s all covered in that “Aubitron” book you looked at.

    Thanks for your help and expertise!

  17. Posted August 12, 2009 at 1:04 pm | Permalink

    OK’s observations add a nice touch to explaining the amount of false B.S. that has been injected into the health reform debate.

  18. okobserver
    Posted August 12, 2009 at 2:52 pm | Permalink

    Monkey your stupid statement about about “Bill O’Reilly may have more eyeballs looking at the screen than Keith Olbermann, but B-O’s eyese are old and poor and stupid and they’re likely closed as his audience nods off into their drool cups. “Countdown’s” numbers dominate advertisers’ target demographics and earn the hour more bucks.”

    Showed one of two things. You are incredibly stupid or didn’t know what you were saying. I merely pointed this out to you. Not touting my expertise merely debunking yours.

    Read what was written.

  19. okobserver
    Posted August 12, 2009 at 2:54 pm | Permalink

    DavidB you are becoming as inconsequential as monkey.

  20. cosmos_originally
    Posted August 12, 2009 at 2:58 pm | Permalink

    DavidB posted August 12, 2009 at 1:04 pm

    OK’s observations add a nice touch to explaining the amount of false B.S. that has been injected into the health reform debate.
    —————-

    Definitely! And so do her responses to other posters.

  21. Monkeyhawk
    Posted August 12, 2009 at 3:01 pm | Permalink

    No, “okobserver” –

    You most certainly were touting your expertise.

    I read what was written.

    You wrote:

    “I have purchased plenty of advertising by looking at the Aubitron ratings and I know…,” didn’t you?

    Perhaps you and “GMC70″ should hook up this evening so you can lick each other’s… wounds.

  22. Monkeyhawk
    Posted August 12, 2009 at 3:08 pm | Permalink

    But nothing quite comes close to Business Investors’ Daily declaring this week that Stephen Hawking would have been long ago murdered with Zyklon B by Queen Elizabeth’s Goon squads if he lived in the UK.

  23. okobserver
    Posted August 12, 2009 at 3:33 pm | Permalink

    Monkey you outright lied about the price of advertising on Olberman over Oreilly. I proved you wrong with the statictics. You started telling me what a good salesman you were. Well if shading the truth makes you a good salesman – then you are terrific.

    You and David need to get together and realize that outright lying will get you in trouble everytime. BTW invite Cosmos he needs a life.

    gotta go

  24. Monkeyhawk
    Posted August 12, 2009 at 4:09 pm | Permalink

    “okobserver” contributes –

    “gotta go”

    What?!

    No tutorial on how you use “Aubitron” data for media purchases?

    I’m stunned and disappointed.

    You claim you proved I lied, but you don’t know what you’re talking about.

    It’s painfully obvious you don’t know what you’re talking about. It reminds me of when “Regular” “went all Cronkite about how mountaintop-removal coal mining was “…a geological impossibility.

    Or how “GMC70″ alleges there’s nothing to the concept of “color of law” setting apart law enforcement officers from civilians.

    How ’bout when “Boxlock20″ waxed eloquent about the Presidency of Thomas Jefferson between “January 20, 1777 and January 20, 1781?”

    Sorry, “okobserver,” but as a group the CONs who participate in this forum CONsistently show their CONfusion with reality. And CONsistently try to CON their ways into imaginary expertise that exposes their CONstant ignorance.

    Then they post:

    “gotta go”

    …and CON themselves into believing they’ve somehow won the field.

    Except those rare times I participate in “Regular”-endorsed “play” in this forum, I try to discuss with individual posters on the thoughts and ideas they post.

    I bring facts, evidence, sites, logic, reason and, I hope, some good humor to most discussions I bother with on WE Blog.

    I tend to stay away from commenting on issues of which I know nothing or not much. I think the new arena in Wichita is pretty mis-guided, but I really don’t have a dog in that fight.

    (If they bring in “Disney’s Texas Chainsaw Massacre — ON ICE! however, I might become a customer.)

    I wouldn’t dare try to impress you with my expertise in the janitorial-services-byconvicts industry. I concede you probably have insights I’d never think of. (I’d never think of, for example, rating my employees by how many teardrop tattoos they have on their faces. Good call!)

  25. okobserver
    Posted August 12, 2009 at 6:08 pm | Permalink

    MH I really have no expertise in the janitorial business so I will let you have the edge on that one. No matter how much garbage you put in your last post it doesn’t matter. You made a bold face lied and I proved you wrong with the numbers.

    The demographics favor Oreilly not Olberman. Plain and simple.

    Had to go and now just want to go. Nothing happening here.