On November 19, Twitter reinstated Donald Trump on to its platform after the company’s self-designated “Chief Twit,” billionaire Elon Musk, had tweeted, “The people have spoken.” This was based on results from a Twitter poll that Musk had posted asking whether he should “Reinstate former President Trump,” to which 51.8% of respondents had apparently answered “Yes.” So did such a poll have much scientific merit or were such results essentially “polling” your leg, so to speak, and potentially “polling” open the door for even more unscientific polls on Twitter in the future? And is this how Musk is going to decide whether to reactivate Twitter accounts that have been previously banned for spreading Covid-19, vaccine, or other health-related disinformation? Well, there are five major reasons why Twitter polls like Musk’s wouldn’t stand up to any type of real scientific scrutiny.
Before we get to these five reasons, let’s take a look at the main thing that Musk seemed to be touting about the poll: the size of its responses. Yes, at first glance, Musk’s poll did seem rather large, garnering 15,085,458 votes according to the following tweet:
At one point, Musk claimed that his poll was getting one million votes per hour. But just because someone says, “I’ve got a big poll,” doesn’t mean that you should necessarily trust what comes out of it. In other words, the 7.8 million votes of “Yes” does not guarantee that “The people have spoken” and “Vox Populi, Vox Dei,” which is Latin for “the voice of the people is the voice of God,” as Musk asserted on November 19:
Vox may be “voice” in Latin, but you shouldn’t let just any voices carry. It’s difficult to tell how many of these voices may have actually been “Vox bots” or “Vox the same person voting over and over again,” which could end up being “Vox garbage.” This brings us to the first big unscientific problem with Musk’s poll:
1. It’s not clear how many individual humans actually voted.
You know the saying, “vote early, vote often?” Well, the risk with any voting or polling system is ballot stuffing, which is not a Thanksgiving dish but the practice of casting more votes than the the number of people who can legitimately vote. Nothing about a Twitter poll seems to prevent such a possibility. A bot may be able to log a vote or even multiple votes on a Twitter poll. At the same time, a single person could set up multiple Twitter accounts to register multiple votes on such a poll. Truly scientific polls will have safeguards that can verify whether someone voting is an actual human being and restrict that person’s ability to vote only once. Twitter polls won’t be able to achieve such standards as long as you can vote completely anonymously and establish anonymous accounts on the social media platform.
2. Musk didn’t specify the characteristics of the respondents and the non-responders.
With any poll, the question is whether the results truly represent what the entire population of interest (in this case Twitter users) believes or instead reflects the thoughts of only a particularly segment of the population. The latter situation could result in some major biases. For example, choosing a Justin Bieber concert to determine what percentage of people have heard of Bieber would be kind of biased in the Biebs favor. Therefore, you’ve got to determine whether the sample polled is truly representative of the overall population.
One common way of determining how representative your sample might be is to report the relevant characteristics (e.g., age, sex, political affiliation, socioeconomic status, and botitiude) of those who responded to the poll versus those who did not and determine how similar versus different they are. The bigger the difference, the more non-representative and potentially biased the responses may be. Did Musk voice any of these characteristics? Umm, vox no.
3. Musk did not provide much time for people to respond.
The poll appeared to open on a Friday (November 18) and close on a Saturday (November 19). So if you happen to have had anything else going on in your life during that one-day periods besides being on Twitter, you could have easily missed the poll or perhaps filed it away as “I’ll respond later after my bout of diarrhea ends” or something like that. Giving people not much more than a day to respond likely favored those folks who happened to be on Twitter during that time period, had strong enough motivation to respond quickly, and believed that Musk would listen to them. This, in turn, could have introduced significant biases into the results. If Musk had truly wanted a broader sample of people’s opinions, could he have kept the poll open longer? After all, whether Trump should be on Twitter wasn’t exactly an urgent DEFCOM 1 matter.
4. There was no transparency about how the poll was administered or promoted or how the votes were verified and counted.
The $44 billion deal that gave Musk control of Twitter basically gave him control of, well, Twitter. That means that he can readily change who works at Twitter, such as lay off half its workforce, or how Twitter’s functions work, such as changing Twitter verification policies so that anyone able to pay $8 a month can get a blue verification check-mark. Heck that latter change even let a seemingly “verified” yet fake Eli Lilly and Company Twitter account claim that insulin will be free, as I covered recently for Forbes. With so many people gone from the company so quickly, who knows how accurate the polling Twitter functions may be right now. So, before you trust any polls, make sure that the methods used to solicit and count responses are clearly documented, legit, and verifiable by an independent third party. For example, you wouldn’t trust someone who told you, “I surveyed a million people and they all said you stink,” would you?
5. Musk did not discuss the limitations of his poll.
One of the most important parts of any presentation or publication describing a real scientific study is the “Limitations” section. This is where the study authors describe the weaknesses of their study and how such weaknesses may affect interpretation of the results. Clearly, no study or no poll is perfect. All have their limitations. Yet, Musk didn’t clearly express such limitations.
Despite these mega-limitations of his Twitter poll, Musk ostensibly let the poll decide whether to allow the MAGA-leader back on his social media platform for the first time since Trump had been banned for inciting violence during the January 6,2021, insurrection and storming of the U.S. Capitol building. That was after Musk had promised on October 28 that, “Twitter will be forming a content moderation council with widely diverse viewpoints. No major content decisions or account reinstatements will happen before that council convenes.” Trump has tweeted since his account went back live again, though. When asked whether he’ll return to Twitter, Trump answered, “I don’t see any reason for it.” But it remains to be seen how many Scaramuccis or heads of lettuce will pass before Trump is back to his old tweeting ways.
Regardless of what you feel about the former U.S. President and current Mar-A-Lago resident Trump being back on Twitter, you’ve gotta worry about basing significant decisions on a highly unscientific and easily manipulatable method like a Twitter poll. A Twitter poll is not replacement for a real scientific poll. And it certainly is not a replacement for real scientific evidence. In other words, a Twitter poll won’t do when the stakes are high.