Flawed Statistical Thinking, the IRS, and Loooooooong Odds

Let’s say there’s a casino where you can’t choose to play. Everyone is forced to participate in the gamble and instead of winning money, the “winners” get punched in the face. A man plays at the dice table 30 hours in a row and never gets punched in the face once. Lucky jerk. Then he says that the casino should be under different management. Within 15 minutes of this statement, he rolls snake eyes (the lucky face-punching number) three times.

Should we consider the possibility that his punches were perhaps the result of his statement about management? They could just be random. Randomness does work that way sometimes.

What if we then discovered that another table in that casino admitted to using loaded dice. Fans of the casino might argue that just because the dice were loaded at one table doesn’t mean they’re loaded at our poor sap’s table. But I think that most of us would take the triple face-punching and the loaded dice story, put them together, and conclude that management conspired to have their critic punched.

I float this analogy in service of recent events.

A few days ago, Nate Silver published a post noting the flawed statistical thinking from Peggy Noonan regarding IRS audits of Romney supporters. Go read it for yourself, but the long story short is that Silver is (rightfully) irritated by people who hold up a few anecdotal examples as proof of a conspiracy. However I suspect in his haste to make a larger valid point, Silver got the odds wrong on of one of the cases Noonan holds up.

In summary, Frank VanderSloot donated a substantial amount of money to groups in support of Mitt Romney. He was mentioned by name by President Obama’s campaign website in an attack that must have seemed normal to lots of people but seems mildly creepy to me. During the next four months, he had to deal with 3 audits: one individual audit, one audit by the Department of Labor and one audit of his business.

In a move that is very much unlike him and that I can only attribute to a careless reading of the events or “blogging-while-buzzed” (full disclosure: that is me right now) Silver has drastically miscalculated this case. Silver looks at the odds of a single audit and maintains (as any reasonable statistician would) that the odds of a single audit of a single individual do not rise to the level of conspiracy.

One of the things I love about randomness is how it builds. Flip a coin and call the result. Get it right and no one thinks you’re anything special. Same thing if you call the second flip correctly. By the third flip, people are getting irritated, waiting for you to be wrong. By the 10th flip, they want to change the coin because they suspect it’s rigged. Every subsequent flip drives the odds higher in a way that becomes almost impossible to comprehend. By the 34th flip, the odds of calling them all right far outweigh the population of the planet.

What we’re talking about here isn’t a single audit (which is what Silver based his math on). We’re looking at three different audits within four months of being mentioned by the President’s team as a “very bad man”.

What are the odds of that?

Fortunately for us, those odds are very easy to count. The odds of someone making over $1 million per year being audited is 12%. Let’s assume it’s similar for medium sized business. Then the odds of being audited AND your business being audited in the same year drops down to 1.4%. That’s certainly lower, but not so low that we should be overly suspicious. Hey… this kind of thing just happens.

But then add in the Department of Labor audit. The Department of Labor  “conducts more than 3,000 audits each year“. For the sake of statistical generosity, let’s call that 4,000. There are 30 million businesses in the United States, so the chance of an individual business being targeted randomly is pretty low, under 1%.

Now let’s add all those odd together. The chance of having all three of these audits in one year is 1 in 520,833 or generously rounded up to 0.002%. This is the kind of thing we look at askance and say “Huh. That’s really kind of weird.”

This would look weird even if the IRS hadn’t admitted to rigging the game (albeit in a different context). Given the admissions they’ve made so far, I think any serious mathematically minded individual should look at the odds and say “That’s odd.”

I feel that, given the circumstances, the burden of proof should lay at the feet of the auditors. I want them to prove this wasn’t politically motivated because, given that they’ve admitted to so far, this particular case is extremely weird.


  1. PMB Fan says:

    As always, I enjoyed the analysis, and I’m glad to see you back blogging again.

    However, you want the auditors to prove a negative?

  2. @Crowesq says:

    Burden of Proof isn’t really the same as “Proving a negative”. When accused of discrimination, administrative agencies have to provide “some” proof that it was a reasonable action based on something (anything) other than discrimination. It is not an extraordinary burden, especially if the audits are, as they claim, coincidence.

    There is a reason disproportionate impact is accepted as a pretty good indicator of discrimination. It’s open to refutation, but it puts the alleged discriminator on defensive. Essential when an individual or group is confronted by an Agency of great power & resources.

  3. politicalmath says:

    Audits don’t happen in a vacuum. What would satisfy me from the IRS side is to have the internal communication of the auditors in each case made public and have the auditor testify under oath.

    It would be particularly helpful if we found out that the process for selecting Vandersloot’s tax return for audit was a process that started before he was mentioned by the president’s campaign. They should also point out red flags in his return that they used as signifiers to trigger the audits.

    This isn’t like proving there isn’t a unicorn anywhere in the world, it’s more like convincing me I don’t have cancer.

  4. Ben says:

    I’m sympathetic to the idea that VanderSloot was targeted, but I don’t see how this set of probabilities establishes anything. First, 1 in 520,833 means that there are probably on the order of 30 people who were so audited. So it’s not like this was a freak occurrence — it happens to many people every year. The question is whether it was odd for it to happen specifically to one of the people Obama named, and it seems like the answer to that question is all about mood affiliation. If you distrust Obama, then the odds Obama chose to name one of these 30 people who were randomly audited is exceedingly low. But if you trust Obama, he doesn’t just single out random people — he singles out bad people who are much, much more likely to be doing bad things than the average person. In that case, the question becomes what is the probability Obama *wouldn’t* name any of those 30 people, who were likely doing something bad to get audited 3 different ways.

    Again, I don’t buy the “trust Obama” explanation, but not because of the math.

  5. Will says:

    The odds of each of the 3 audits you have calculated are independent. I am skeptical that that is a fair assumption. I don’t know enough about audit triggers and processes to say for sure, but suspusion in a single area of a person’s finances or business might likely trigger further audits, i.e. a single audit is positively correlated with further audits. For instance, it may be the case that while high income persons ($1M+) have a 12% chance of audit, high income business owners have a higher risk of personal audit and high income business owners who are being audited may have a even higher risk of a business audit.

  6. politicalmath says:


    While I’m not a lawyer, my understanding is that the DOL and IRS audits should be independent odds since they come from different departments (the IRS being a part of the Treasury Department). The business and income audits being independent is a bit of an open question. I haven’t seen any data on that although I did go looking for it… if you see anything let me know.

    However, we do know that very high income individuals (+$1 million) have a 12% audit rate and that very high income businesses have a 20-30% audit rate. However, “high income” businesses are considered to be multi-billion dollar businesses. Vandersloot’s business would be considered either the high end of small or the low end of medium. It’s hard to get business audit data for this rage but we do know that the audit numbers are lower for this range of business than they are for the high income businesses.

    However, the real kicker is this: More important than the average audit rate is the existence of red flags in his personal and business taxes. The fact that he had not been audited in previous years (either individually or his business) indicates that these flags probably did not exist. This is reinforced by the fact that he suffered no fines when the audits completed. Given all these variables, I think an inquiry into why he was audited is perfectly justifiable.

    @Ben – Yes, these things do happen. Very occasionally. Which is why I’m not saying this is proof. Hell, if it weren’t for the fact that we already know the IRS was involved in political corruption, I would chalk it up to “freak occurrence”. But with the admitted corruption in an election year in a time frame just after he was singled out by the president’s campaign… like I said, the burden of proof shifts to the IRS and Department of Labor to prove they *didn’t* target him for his political actions.

  7. Ike says:

    Shorter version:

    They can’t prove they didn’t target VanderSloot, but they can prove that paperwork on his case pre-dates his presidential notoriety.

  8. shori zand says:

    Hello… DropshipDragon offers dropping for level of quality, affordable items direct supply by china manufacturer to the buyers. Stunning for eBay suppliers and webmasters similar! …

Leave a Reply

Your email address will not be published. Required fields are marked *