Anyone Can Publish a Poll. Here Is How to Tell If You Should Believe It.

Last week, a firm nobody had ever heard of published election polls for races in Los Angeles, Wisconsin, and Nevada. The numbers looked legitimate, the formatting was spot-on, and social media picked them up almost immediately. Even a campaign shared the results before anyone stopped to ask a simple question: who exactly are these people?

As it turns out, they were nobody. The organization, calling itself Median Strategies, had a website barely a week old. When researchers and reporters started digging into their methodology, the firm admitted the polls were complete fabrications—part of a “social experiment” to see how easily unverified data could slip into our political conversations. Then, just as quickly as it appeared, the website vanished.

The experiment worked, and that should be a wake-up call for anyone evaluating or publishing election survey results.

Why this matters beyond elections

At Magellan Strategies, we no longer manage political campaign polling projects, so we could easily just watch this from the sidelines. But we won’t. When fake polls circulate, the damage isn’t limited to one election; it erodes residents’ trust in legitimate surveys from their local cities, school districts, special districts, and public policy organizations. Public organizations rely on accurate, reliable survey data to make informed decisions on behalf of residents and voters. When people stop participating because they’re skeptical of the data, the whole community loses out.

Unfortunately, with AI, it’s never been easier to make fake polling numbers look legitimate and professional. However, it’s actually quite simple for the media and political observers to learn if a poll is legitimate, provided you know which questions to ask. (I wish more Colorado media organizations had survey/polling publishing standards)

How to identify an illegitimate survey

The reason polling aggregators caught on to the Median Strategies fraud was simple: the firm wouldn’t answer basic questions. In professional public opinion research, we document and publish a survey’s methodology and results because that’s how you verify the work. Transparency about the survey is paramount. So, before you believe the findings of a survey, learn the basics.

A credible opinion survey firm will document who conducted the project and whether an individual or organization paid for it. You should also be able to identify a survey firm’s employees and their history of publishing survey results. Additionally, understand the survey’s margin of error, the sample size (how many people actually responded), and the demographic profile of those respondents. (the percentage by gender, age ranges (18 to 34, 35 to 44, etc.), party affiliation, race, education level, income, and region)  

As we mentioned, knowing the survey data collection methodology is incredibly important. Were the respondents invited to participate because they are a member of a paid/incentivized panel? Were the respondents interviewed by a real person at a call center? Or our preferred data collection method: inviting a random sample of registered voters by sending an MMS text (a text with an image)? Or, heaven forbid, a mail/paper survey that is filled out by hand and mailed back to a data collection center. (This method is guaranteed to oversample old, white, wealthy homeowners and undersample younger individuals, populations of color, renters, and those with lower incomes.)   

Every survey data collection method has its pros and cons, and a transparent firm will provide this information. You may also want to know when the survey was fielded, because in some instances, opinion can change quickly. A recent example of this is the Wisconsin Democratic Primary Election for Governor, where support for candidate David Crowley surged in the final two weeks. Finally, the polling firm should provide a “topline” document showing every question asked and the order in which they were asked. If the full survey release does not include this information, the survey should not be reported and should be dismissed as a poor attempt to influence public opinion. 

How Magellan Strategies holds ourselves to this standard

For us, transparency isn’t just a checklist we add at the end; it’s how we’ve done business for over 20 years. That’s why public organizations, including governments, school districts, special districts, and public policy organizations trust Magellan Strategies with their most important survey projects. To meet a true transparency standard, we believe a reputable survey research firm must include the following survey “deliverables” when publishing results.  

1. Toplines: A topline document showing every question in the order they were asked to respondents.

2. Crosstabs: A set of crosstabbed data showing responses to each question by major demographic/voter population subgroups such as age, gender, and party affiliation.

3. Verbatim Responses: If the survey included open-ended verbatim questions, the polling organization should be able to publish those so the general public can do their own keyword research or AI theme aggregation exercises. 

We work this way so that anyone – a council member, reporter, legislator, political observer, or a skeptical resident can see exactly how we managed the survey project and compiled the opinion data. We don’t ask you to just take our word for it; we show our work so you can check it yourself.

The takeaway for public organizations

If you’re hiring a research firm, hold them to this same standard. And if you’re looking at survey numbers during a public meeting or a ballot debate, ask for the information outlined in this post.  

Accurate, professionally collected public opinion data helps public organizations listen to the communities they serve. Fabricated data does the opposite, and last week’s episode shows how little effort fabrication requires. The difference between the two is not the formatting of the published data. It is whether the people behind it are willing to show you how the work was done.

At Magellan Strategies, we will always adhere to a transparent survey publishing policy.