David Dworin Online

Archive for the 'Polling' category

Phones Increasingly Unwired

May 15, 2007 11:28 am

Data from the latest National Health Interview Survey suggests that the land line is becoming a thing of the past. Yahoo news has a summary:

One in four people aged 18 to 24 had only cell phones, as did 29 percent of those aged 25 to 29, the study showed. The percentages declined with age after that, with 2 percent of those 65 or over having only cell phones.

The group most likely to have only a cell phone are young people with low incomes. Studies on the subject still conclude that the cell-only population isn’t large enough to skew broad polls, but it certainly has an affect on segmentation, especially towards these target groups.

Political Future Watch

April 12, 2007 6:54 pm

Slate has announced a guide to political futures markets for the 2008 US Presidential Election. Political futures generally predict outcomes better than polls for a number of reason. Slate is going to track the big prediction markets along the big races and report along.

For a snapshot, the Iowa Electronic Market, the Big Daddy of prediction markets, shows Clinton and Obama neck and neck for the Dems, and the Field Candidate (anyone but McCain, Romney, and Guiliani) taking the republicans. Guiliani leads among those candidates with their own futures. The dems are also predicted to win by a slim margin.

Also, I haven’t investigated the differences between contracts in the markets, but the wide price differences between the Iowa Electronic Market and Intrade seem like an arbitrage opportunity for those who can trade in both.

Millenials: Selfish or Selfless

March 8, 2007 9:52 pm

A discussion of Millenials in the Christian Science Monitor:

But Twenge and others are wildly mistaken about the Millennial generation – those born since the early 1980s. No matter what teens say on surveys, there is scant evidence that they act more selfishly. In fact, the trends in youth behavior support the opposite conclusion – that Millennials have much greater regard for one another, their parents, and the community than Generation Xers or baby boomers had at the same phase of life.

Some notes:

  • This is an important lesson about surveys and revealed preferences. Survey questions only work if you have reason to believe people will respond honestly (why questions about race don’t work) and the questions you’re asking actually measure what you want (which is much more frequent). The crime data shows how people are actually behaving, not how they answer silly questions about their personality.
  • Why are there always sky-is-falling predictions about the next generation, especially when in general, things keep getting better? Cut the kids some slack, as much as you try to say that you were different, you were just like them when you were young.
  • More specifically, whats with the boomers and trying to put all the negative attributes of their generation (selfishness, antipathy towards parents, sex and drug use) onto the newest generation, despite the fact that all the evidence says Millenials are totally unlike the boomers (similar to how boomers weren’t like their parents).

Sociologist Len Saxes Visits Rosners Domain

February 27, 2007 11:51 am

Rosner’s Domain has an interview with another Jewish demographer (I noted his talk with Ira Sheskin here). This time, he questions

There is no question that intermarriage is changing the face of American Jewry (and Jewish communities throughout the Diaspora). But the evidence suggests that intermarriage is not the cause of the problem. The underlying issue is that many Jews do not grow up experiencing a rich Jewish home life and their Jewish education is ineffective and, too often, distancing. A colleague who is a Jew-by-choice recently joked that it was easy for him to embrace Judaism - he didn’t have the baggage that born Jews did of a having had to “suffer” in Hebrew school as children.

Dr. Saxe heads up the Steinhardt Social Research Institute, which conducted the meta-analysis counting project I mentioned here. As far as I can tell, the institute is fairly new and hasn’t really put out much research yet, though other scholars at Brandeis working on similar matters certainly have.

One thing to note is that Dr. Saxe extrapolates a lot from his Boston Community Survey, despite the fact that there’s no evidence that trends in Boston apply nationally or globally, and there are a number of reasons to expect otherwise. Nevertheless, I tend to agree that the problem isn’t intermarriage, no matter how the numbers line up, but a lack of engagement on the part of the Jewish community.

How Many Jews Are There?

February 10, 2007 1:51 am

A new study from the Steinhardt Social Research Institute says there could be over seven million Jews. Why are there so many different numbers, and why are Jews so hard to count.

One of the biggest reasons is that it’s hard to determine just who is a Jew. People identify as Jewish on religious, ethnic, and cultural grounds, while others identify by birth, and still others were clearly born to Jews but do not personally identify as Jewish. This makes picking out who is Jewish in a survey extremely difficult. You have to ask the right questions, the right ways, and then figure out which answers mean someone is a Jew. Changes in how you define a Jew can cause swings of literally millions of people.

But once you come up with a definition for who’s Jewish and who isn’t, you’ve still got trouble counting people. That’s because no matter how you define it, there just aren’t that many Jews in the population. Think of a jar with 1000 marbles in it. Of those, somewhere between 975 and 985 are yellow, while the rest are green. The green marbles, between 15 and 25, represent the size of the Jewish population. Now imagine that you have to count them by taking a small sample, say 100 marbles, out of the jar. If the jar were half green and half yellow, this would work pretty well, but with only a small number of green marbles, your sample of 100 could give you one, two, or three, each of which would give you radically different estimate of the number of green marbles in the whole jar. Counting Jews works the same way - the statistical sampling methods used in normal social science break down when trying to reach a very broad but very small group.

Further, Jews are less likely than the population at large to respond to random digit dial phone surveys. In other words, when you try to take the marbles out of the jar and count them, some marbles won’t let you, and the green marbles are less likely to let you. Given the huge swings a few marbles can give you, this makes them even more difficult to count. Because Jews won’t talk to the survey interviewers, we don’t really know how much less likely they are to answer the phone, so we don’t really know how to weight things. We also don’t know the characteristics that make one person more likely to answer than another.

Each of these caveats makes counting Jews extremely difficult, and leads to the controversy around the numbers given by Jewish demographers. So what are the different techniques that are used? Random digit dial surveys, like the massive National Jewish Population Survey use a weighting technique, similar to what political pollsters use, to try and bring the numbers in line with what they know to be true about the population as a whole. These can be controversial, whether for counting Jews or voters, as they start to tinker with the underlying science of public opinion research in ways that not all social scientists or statisticians agree. The NJPS also needs an extremely large sample to cover the whole country, and this means it’s prohibitively expensive to conduct accurately.

Another technique, used by Dr. Ira Sheskin at the University of Miami, takes a collection of community studies and combines them all together to get a national account. Since most Jews live in urban areas, Dr. Sheskin’s community studies, of which he has done many, are more likely to target those numbers where Jews are concentrated. He also over-samples in “core” Jewish areas, increasing his response rate. However, this method has its own shortfalls. For instance, many elderly Jews are “snow birds,” with summer homes in the Midwest or Northeast and winter residences in southern areas like Florida and Arizona, making it likely that they’ll be double counted. The same thing happens with younger Jews who may go to college in one city while claiming residence in another. And finally, it ignores the “long tail” of Jewish communities, smaller cities that still have Jewish populations.

Researchers at Brandeis’s Steinhardt Social Research Institute used a third method. They took publicly available survey data, rather than data specific studies on the Jewish community, and conducted a “meta-analysis,” trying to merge it all together to come up with a good accounting. In other words, instead of taking 100 marbles out of the jar, they found twenty people who had each taken 100 marbles out of the jar for different reasons, and used their counting. Not only does this give them more samples, but it also allowed them to save money for expensive polls with large samples that are required for the other two techniques. Unfortunately, there are a number of risks here. Different studies use different samples, questions, methodology, making it complex to determine how to weight any specific data point, and restricting the available data to what those other studies are interested in. Nevertheless, it provides an innovative, and significantly more cost effective, technique for gathering data not only about how many Jews there are, but information about them as well.

So which of these methods works the best, and how many Jews are there? Because of the problems described above, there’s no way to really know. The best answer is that it’s somewhere between four and eight million, and that whatever number you pick is probably off by a million. And like with most things Jewish, there’s going to be a healthy debate around the issue so that any two Jewish demographers will probably give you three different numbers.

Dr. Ira Sheskin in Rosner’s Domain

January 31, 2007 2:53 pm

Rosner’s domain interviewed Jewish demography expert Dr. Ira Sheskin:

Where the lowering of the numbers will have an impact is on small Jewish communities. A Jewish community of 10,000 which now supports, say, 3 synagogues, a JCC, a kosher butcher, and several Jewish agencies and organizations, may very well lose some of this infrastructure if, say 20 years from now, the population is down to 5,000. So the impact of the lowering of the Jewish population will be at the local level more than at the national level.

Many resources in the Jewish community are being applied now by Jewish Federations and Foundations throughout the country to assure the Jewish future. Sheldon Adelson just started a foundation which will provide $200-$250 million per year in grants to Jewish communities. The challenge is to devise programs and services that will provide a quality of life within Jewish communities that will keep people wanting to be Jewish.

Dr. Sheskin also addresses intermarriage later on, though he tacks differently than i would. For those interested in Jewish demographics, most research studies are available for free online from the Jewish Databank. There, you can find the summary reports of most community and national studies, as well as the raw data for many of them, including the National Jewish Population Survey.

Cell Phones Change the Polling Landscape

January 23, 2007 8:17 am

Mystery Pollster Mark Blumenthal cites a National Center for Health Statistics study concerning the growth in cell-phone only households, a concern for pollsters who can’t reach them, as well as a Pew Study which finds that this group is distinct from the rest of the population:

A new study of the issue finds that cell-only Americans – an estimated 7%-9% of the general public – are significantly different in many ways from those reachable on a landline. They are younger, less affluent, less likely to be married or to own their home, and more liberal on many political questions.

It doesn’t look like the study controlled for age in the base numbers, and aren’t younger people generally less affluent, less likely to be married or own their own home, and more liberal on many political questions? What the study really says is that young people aren’t reachable by pollsters because they are in cell phone only households. Don’t worry, though, young people don’t vote!  When they did control for age, most of the differences look like they’re for things that correlate strongly with affluence and urbanization, which makes intuitive sense.

Subscribe