Seeking information about the accuracy of the most recently touted poll

The New York Times has given pride of place on its website to a story claiming that a “New Poll Finds That Young Americans Are Leaning Left.” According to the poll, the 17-24 year old Demographic hews to the liberal side on every single issue but for the War, as to which they have a happy optimism that should shame Harry Reid:

Young Americans are more likely than the general public to favor a government-run universal health care insurance system, an open-door policy on immigration and the legalization of gay marriage, according to a New York Times/CBS News/MTV poll. The poll also found that they are more likely to say the war in Iraq is heading to a successful conclusion.


They have continued a long-term drift away from the Republican Party. And although they are just as worried as the general population about the outlook for the country and think their generation is likely to be worse off than that of their parents, they retain a belief that their votes can make a difference, the poll found.

More than half of Americans ages 17 to 29 — 54 percent — say they intend to vote for a Democrat for president in 2008. They share with the public at large a negative view of President Bush, who has a 28 percent approval rating with this group, and of the Republican Party. They hold a markedly more positive view of Democrats than they do of Republicans.

It certainly sounds devastating for the future of the Republican party. I’m wondering, though, about the poll’s accuracy.  The article states that the pollsters talked on the phone to only 657 young people across America.  Based on this sampling, they claim statistical accuracy of plus or minus four percentage points.  My question: Is polling so accurate that 657 young people across America, out of a demographic population that must number in the millions, can be used to provide hard data about what most American young people believe?

I have to admit to being suspicious of polls, since it so often turns out that the collecting methodology was suspect or biased.  (Both Best of the Web, on the WSJ’s Opinion Page, and Cheat-Seeking Missiles are exceptionally good at digging into polls and exposing the underlying problems with data collection.)  In this case, the NYTimes, where I found the above article, doesn’t provide a link to the underlying poll. I’m rather cynically wondering here if we’ll discover that most of those polled are urban youngsters (that is, youngsters who live in areas that are more liberal in any event), or if repeated experience has shown that young adults who answer polls over the phone trend differently from other young adults.  And, getting back to my original point, how accurately do 657 people reflect the attitudes of millions?

Clearly, I’m operating here from a basis of some pretty heavy duty ignorance about polls in general and this poll in specific.  Any information you can provide would be very helpful.


15 Responses

  1. Hello Bookworm,

    Well, I confess don’t know much about polls either in the sense that I don’t know what would make an accurate sampling of a cohort group. However, I do know that how you phrase the question is of major importance. In fact, a dishonest pollster can make polls say anything they want it to say.

    For instance:

    Question#1: Do you believe ripping children from their parents are wrong?

    Question#2: Do you think that people should have a fair shot at making a decent living?

    Question#3: Do you think that a woman should have the same freedom men have to choose whatever doctor she wants if the doctor can save her life?

    Question#4: Do you believe that invading another country that hasn’t done anything to you and doesn’t threaten you is wrong?

    If you answered “Yes” to all the above questions, your answer could be interpreted as being 1) against deportation of illegal immigrants 2) amnesty for illegal immigrants 3) being pro-abortion and 4)against the war in Iraq.

  2. Is polling so accurate that 657 young people across America, out of a demographic population that must number in the millions, can be used to provide hard data about what most American young people believe?

    No Way.

    Half of the pollsters and 99% of those analyzing polls don’t even have degrees in psychology. Wait, did I say half, do half of the pollsters even have degrees in psychology? You know, pollsters, those making up the questions.

    4% statistical error doesn’t really matter when your questions are skewed concerning only one thing. It is causality that matters in polls, not percentage errors because humans are not dictated by percentage errors. Our behaviors are not modified or whatever by thinking over percentage errors. For example, when people are driving, they don’t think of the percentage error for getting into a crash if they are using a phone or whatever. People answer the way they do because of perception, psychology, group mechanics, etc. Not percentages.

    Statistical percentage analysis is a mathematical field, not a human field. They try to make the mathematical error percentages seem like “human truth”, but it isn’t book.

    A pretty well designed poll based upon human psychology and demographics would be this, I think. Take about 100 out of each state university across the country. Take 75 out of every technical engineering college, 2 years, sampled equally geographically. Then take about a generalized poll of maybe 25-50% of the workers of Sonic, Chickfillet, MacDonalds, BurgerKing, etc and get their views and ages and education level. Then take the same percentage, 25%-50% for beginning engineering/business/advertisement, etc firms. Interns basically.

    Divide down the polls through education, geography, college, job, etc.

    Then the analysts can get some good results for you, Book. 700? Seven hundred is not enough to correlate 5 variables, let alone the hundreds for young people out there. Every 10 person you grab randomly, there are 4 statistically outliers there, Book. 2 on the high, 2 on the high, the 6 is the middle which is what you want. Statistically error with 10 though is very high, because you can sometimes get 5 on the high, 1 on the low, and 4 in the middle. HUGE skew. The reason why the 700 poll is crap is because it is trying to handle 200-1000 variables with only 700. Can’t be done. Not if you want an accurate analysis of the psychology and beliefs of young people, Book. 700 is fine if you want 4% statistical error over not much of anything important.

    Most political polls don’t really make much sense until you realize that 80% percent of blacks are Democrats and they skew the results by themselves. So every sample of 500 blacks means there’s only 100 Republicans, skewing the results. This actually produces a trend you can see, if you didn’t also only have 500 whites. Or 1000 whites, or 1500 whites. It still skews it because with a poll of only a thousand, to correlate blacks being polled to their percentage of the population means a higher percentage error on the opinion of the black community. Skews one way or another, I say.

    Put together, that skews results, but you wouldn’t realize it until you broke down the demographics. It is actually more accurate to see the Strong Agree/Disagree numbers for whites, separate from blacks. There’s a noticeable difference in opinion on certain subject matters. It makes no sense to combine them and say “here’s 61% approval”. That means nothing. Without at least knowing the divisions between the young folks poll, Book, you won’t get much of anything but you can lie about it a lot. That suffices for the media. Truth to power, with the exception of their power.

  3. 2 on high, 2 on low, 6 in middle. Normal bell curve for random sampling. Correction.

  4. Actually, there’s a link to a PDF with the poll in a left block of the page I read. Their immigration conclusions are questionable, as discussed at the link.

  5. Young people have always skewed liberal. This is not news. I thought Carter was a good president when I was 18. Young people are very inexperienced and idealistic, which precludes their being able to put poll questions into context. They are also much more susceptible to answering in the way they think others of their peer group will.

  6. Maybe ,but I doubt the above, however “out of the mouth of babes and sucklings,has thou ordained strength.” Hey, just a minute, on second thought,the pollsters are part of a world wide communist led of a beast conspiracy to bring down the Repubs.Yeah that’s it !

  7. Personal anecdote: about six months ago I got a call from a woman claiming to be doing an AP poll. Her Central American accent was so heavy that at times I had to ask her to spell out some of the words in the questions. (“Judge Boosh” was a frequent topic.)

    Most of the questions were so sloppily worded as to be meaningless. My favorite example: Do I approve of the way Judge Boosh has handled education, the environment, and the economy — yes or no? It would be hard to find anyone who would answer yes to that question, and there were no followups to determine _why_ I might disapprove. Do I think he’s not doing enough to protect the environment or do I think he’s caving to eco-cultists? Do I think he’s letting atheists indoctrinate our kids or do I think he’s turning schools into Bible camps? Never asked.

    Ever since then I’ve decided polls reflect only the opinions of those who commissioned the poll. No other meaningful conclusions can be drawn from them.

  8. […] turns out that the collecting methodology was suspect or biased. (Both Best of the Web, on the WSJ’s Opinion Page, and Cheat-Seeking Missiles are exceptionally good at digging into polls and exposing the […]

  9. Polls and statistics can be used by left and right to spin and spin to tell any story they wish.You know it an I know it. Again Twain,I believe said, there are lies,damned lies and then there are statistics.

    ps Also, stats are like a beautiful – – – – – in a – – – – – – ,what you see can be very interesting,but what is covered up is quite vital ! In other words you don’t get to see the whole truth of the matter.

  10. Polls with a sample of under one thousand should always be considered suspect. Even though the larger margin of error is noted, I dont believe it actually conveys the lowered accuracy. I dont give any creedence to a poll with a sample less than a thousand, or with a margin of error greater than 3%.

    And as has been pointed out above, how the questions are given, who the poll is aimed at, the group paying for the poll, and many other factors make modern polling not a lot different from tea leaves, chicken entrails, or any other method of taking the auspices.

    Polls have been given a place all out of proportion to their utility or accuracy, and are in fact just propaganda masquerading as science undergirding objective news reporting. But just the fact that polls now ARE the news should tell us something.

  11. 657 is a quite adequate sampling size. The commenters are correct, however, that (a) the sample has to actually be representative (if done by a reputable pollster, it probably is), and (b) the questions have to be unbiased (I’d have to see the questions).

    I have absolutely no reason to disbelieve this particular poll, however. People are almost always more liberal when young and learn better as they grow older (look at you, Bookworm). Also, these kids are the victims of their educational system, which is absolutely dominated by liberals. No wonder they end up parroting liberal positions.

  12. Don’s going to have some problems with the pdf poll then. Because there is not much statistical difference between the so called Adult Only poll and the Youngling Poll in some of the most liberal questions.

    46. Which of these comes closest to your view? Abortion should be generally available to those who want it, abortion should be available but under stricter limits then it is now, abortion should not be permitted?
    Available Available but stricter Not permitted DK/NA
    5/18-23/07 All adults 39 37 21 2

    Look at the abortion one.

    39 available or adults, 37 for Young Folks.

    Look at global warming question down the list. 52-54, adults to young.

    The polls aren’t even consistent with the popular belief that as you age you become more conservative.

    There may be no reason by inductive logic to disbelieve, but there are plenty of reasons through deductive logic to disbelieve.

  13. Young people are left of center:




  14. As Winston Churchill pointed out, everyone when young is liberal – then they grow up.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: