Data Visualization Training in DC – June 29th 2015

If you’re in or near Washington, DC, I’d like to invite you to attend my full day data visualization training hosted by the Leadership Institute.

This session is cheap. $40 for the whole day and they give you food. They give you food! You could probably bring some big ziplock bags, rent a food cart for the day and make out ahead. Think of it as an investment.

You can read the formal description that they let me put up on their site but I wanted to give a more casual explanation here.

Who is this for?

This is the hardest part of the training for me. Who will be there? Programmers? Graphic designers? Journalists? Nobel Prize winning economists? Gender theory majors? I’m not making any promises, I just don’t know. So I’m trying to talk about all the things needed to make great data visualizations starting on the ground floor.

This is not visualization theory training. This is not sitting in a room listening to the sultry tones of my melodious voice all day.

I’m going to give a 1 hour presentation on how to do some stuff that you need to be able to do to make cool data visualizations, then there will be a lab, an hour of time to walk through the the stuff I just showed you. My goal is for everyone who attends to actually build 2 data visualizations during the course of the day.

Does this mean technical people will get nothing out of this? I sure hope not. I’ll be presenting on algorithms that I use, yes. But to make sure this is accessible, I’m also providing to the class a set of Excel helpers that I personally use to build my own visualizations. I’m also building a set of web-based tools to help get your visualizations started. I’ll provide all that code to the class so, if you’re technically minded, you can pull it apart and see how it works. And I’m happy to answer questions on it.

If you attend, you will get:

  • To see my glorious face
  • All the files, presentations, labs, and helper scripts that I present in the class.
  • Membership in a Google Group where we can discuss visualization and you can get help with your visualization projects in the weeks after the class is over.

Things you will NOT need

  • a dinosaur-powered rocket ship
  • a mathematics background
  • programming experience
  • design experience

Things you WILL need

  • clothes, probably
  • a laptop with Microsoft Excel
  • some kind of data manipulation software (Photoshop, Paint.Net, Gimp)
  • the willingness to try something new

I’m excited about this event. You will never find training like this for $40… you’re basically stealing it.

This is like those Groupons that you could get when Groupon was just starting out and they totally ripped off their partners and you could get a dozen cupcakes from that tiny little cupcake store for $3 or whatever and 5000 people would buy them and put the little cupcake store out of business.

I’m the cupcake store. You’re the person who kind of feels a little guilty for taking advantage of this great offer, but, hey, it’s a great deal and you’re not going to pass it up.

Fetal Pain And Neuroscience

This is a piece that I put up on Tumblr last year during the Texas fetal pain legislation. I didn’t know at the time that Tumblr is terrible for blog posts so, as we once again get into the questions of fetal pain due to the impending federal legislation, I thought it would be a good piece to bring to blog form as the discussion renews.

Note: The conversation below is neither a political nor a moral discussion. This is strictly about what nueroscience tells us about the experience of pain and how that sense of pain develops in a fetus.

<Begin>

Back in June 2013, a Texas GOP congressman said something about how fetuses feel pleasure and pain and was roundly mocked. The congressman was a former OB/GYN, so we know he had some experience and training with pre-born infants and I wondered how accurate his statement was.

So I turned to my brother, a published neuroscientist currently going through med school. He actually might object to being called a neuroscientist since he doesn’t have a PhD, but he did years of graduate research exclusively in the field that has resulted in several peer-reviewed publications, so I feel pretty comfortable with the title.

Anyway, I asked him about it and we had a big discussion that I felt really informed my understanding of the topic. I meant to compile it into a blog post, but the mockery faded and so did my interest.

Then Salon decided they wanted to throw on their “It’s science, bitches!” baseball caps and prove that “their side” is using science while “the other side” is using nonsensical pseudoscience for dummies.

This is a thing I’ve noticed too many people like to do: Simplify the science to a point where they are conveying no valuable information or understanding (check), find a technical voice willing to confirm their bias (in this case, a second-trimester abortion provider who, I’m sure, has no dog in this fight), and generally act like second graders who found that the science textbook phrased something in such a way that they can titter to themselves and feel intellectual about their ignorance.

It’s juvenile, it’s arrogant, and people deserve to understand the complexities of the matter a little better. Journalism is supposed to do this. Actually… decent, intellectually curious human beings are supposed to do this, whether or not they work in journalism. But journalists are supposed to get paid for it and they have failed miserably to present the science here, instead giving us a pre-baked conclusion backed up by an abortion provider while yelling “SCIENCE”.

So here is my discussion with an actual neuroscientist who actually wants people to understand things that matter. Keep in mind, this discussion stems from the original congressman’s statement, not from the Salon piece.

Me: You know something about embryology, is the congressman’s statement here about fetal pleasure correct?

Bro: It’s highly inaccurate. It’s straight up wrong because of his definitions.

Me: Inform me.

Bro: Let’s back up a bit and define our terms, the words “feel,” “pleasure,” and “pain”.

“Feel” can be simply mean a response to environmental stimuli not necessarily requiring a level of processing indicative of intelligent life. For example you poke microorganisms and they can recoil because they don’t like getting poked. It “felt” that poke.

“Pain” suggests a different and higher level of processing capable of suffering in response to environmental stimuli. This is a much more complex neurological process and also much harder to cleanly define.

Me: OK. For context, the political argument on the right regarding abortion (at the moment) is “a fetus of a certain gestational age can feel pain”. So what would you suggest is a good benchmark for measuring pain (in the sense that humans or higher level organisms experience it)?

Bro: That gets way trickier. Again, our terms aren’t clearly defined in a way that I, as a scientist, can really give a “yes” or “no” answer. I don’t know whether you mean “at what point do we know something is feeling pain?” or “What level of processing is required to feel pain?” or even “What level of neurological pain processing is close enough to what we call ‘suffering’ that it would matter from a moral point of view?” 

Keep in mind that while “pain” and “response to stimuli” are commonly grouped together, they are not necessarily bound together. You can feel pain but be unable to respond to it. For example, there are cases of anesthesia awareness where a patient is fully non-responsive but also fully conscious and capable of pain. In a situation like that, the only way we know pain happened is because the patient reported it after the fact. So a lack of response to stimuli is no guarantee of a lack of pain. (for more on anesthesia awareness: http://en.wikipedia.org/wiki/Anesthesia_awareness )

If I were making a pro-life argument, I would take a similar stance as the guy in the article, although I would absolutely not use the word pleasure. Pleasure comes from a far more complex form of brain processing than pain. Take that statement out of the discussion completely.

The main issue (it seems) is that at some point a developing human is capable of processing pain. There is no difference in terms of brain processing 2 minutes before a child is born and 2 minutes after. The question then becomes when is the fetus (or pre-born infant or whatever we’re using these days) capable of processing pain. The answer is that we don’t really know. But here are the neurological boundaries for the discussion that might give us some groundwork for trying to answer that question.

Me: Hooray for philosophy.

Bro: By the way, I know this is from a neuro heavy perspective of embryology… but that’s what I got.

First, there’s response to stimuli. A fetus responds to stimuli around 15-20 weeks. We could imagine how we would feel in response to the stimuli so we might transpose our imagined experience to that of the fetus. If they respond in the same way we think we would respond, we think they feel the same way.

That’s as a result of things called “mirror neurons” that help us to enjoy watching football games cause we can imagine ourselves catch the football. It improves things like empathy/sympathy. But just because a goldfish recoils from being poked with a sharp object doesn’t mean that it processes that poking in such a way that it leaves emotional scarring for years to come. We might be emotionally scarred, but the goldfish won’t be. It feels, i.e. responds, but it doesn’t process.

Second, there’s pain. This means that it responds to *and* processes pain stimuli in a way that we do, but that doesn’t mean that it processes the experience of pain. By that I mean that we have higher processing capable of recalling, discussing, analyzing, philosophizing, considering (etc.) experiences. We can do that but a frog wouldn’t do that.

Third is the mental recall and analysis of pain. We might try to rescue someone who is in pain, because we know how pain feels and it sucks and you empathize with someone and we don’t want them to experience it. We see pain, recognize it, empathize, and respond. We’re processing the experience of pain in a recollective and complex fashion.

So that’s 3. 1. Feeling, 2. Processing, and 3. Complex processing of pain

Me: So, if I understand correctly, 1 (feeling) isn’t something we would normally care about. And the fetus does that pretty early on. 2 (processing) is something we care more about, but there’s a lot of gray area. Fetuses *probably* have that in the 20ish week range, but it’s hard to say. 3 (complex processing) is something we care deeply about but it’s pretty much impossible to say whether a fetus or even a baby under 1 month has that in a way we would recognize.

Bro: Mostly right. Processing is actually something that might not really be happening until like… a year old. It almost certainly doesn’t happen prior to birth.

With 3 you can definitely suffer. With 2 you can maybe suffer. With 1… well, if suffering occurs with that we might as well not even eat plants.

The pain line of argumentation is useless for someone who wants to focus on conception. If you want to use pain as a moral sticking point and you want to do it grounded in science, you should focus on either the grey area of 15-25 weeks or sometime around like 6 months or a year after birth. Is anyone making the case abortion should be legal at 56 weeks? If they base the right to life on pain processing, they’re going to have to give up on 38 week abortions too. At least if they want to be intellectually coherent and base their argument on science. Is that a big concern on both sides?

Me: Not usually.

Bro: OK. Well, back to the neuroscience. I’m not boring you, am I.

Me: Nope, continue.

Bro: OK, the brain is kind of like computing. A single neural connection doesn’t really mean a whole lot just like if you have one transistor that doesn’t really do a whole lot. Neurons are building blocks. When you have 100 billion, then you can build something pretty awesome. But it’s hard to attribute meaning to individual neurons.

The brain is divided by function into gyruses and sulci. Gyruses are the ridges and sulci are the folds or wrinkles. Different gyruses have different functions.

Fetuses younger than 24 weeks have almost no gyruses. Fetuses older than 35 weeks have virtually all their gyruses. The prefrontal cortex is what controls the executive function, where the mirror neurons are found, that lets us sympathize, empathize, philosophize, understand society. That’s the last to develop. We’ve got the largest pre-frontal cortex, monkeys have a good sized one and on down the evolutionary chain. Most mammals have one, while invertebrates don’t

The neurons located in a gyrus have the function that we expect it to have, so when the pre-central gyrus first forms, there we can find the motor neurons that control voluntary movement. Before the brain folds to make that ridge, there might be some of those neurons and the infant can move a little bit, but once it folds then where the neurons are ceases to be gray area where that gray matter is (or if it is present).

We can say “There’s the precentral gyrus! Well, he’s definitely processing muscle movement and that’s where it’s happening.” But the gyrus is a folding to making space for neurons that are multiplying in order to make room for them, not a magic appearing of neurons for the first time. So the folding is happening at 5 months, but the infant can start purposeful movement at 18 weeks (or right around 4 months).

Does that all help?

Me: I think so. From a medical/neurological stand point there are a couple key “dates” that we could use to determine what might be a helpful guideline for the “pain” discussion. If we want to say that newborns are people, we would have to track that concept back to at least 35 weeks.

And then there is a gray area back to about 24 weeks and then another one back to about 18 weeks based on neurological development and “what it means” in terms of pain processing.

Bro: That’s about right. 18 weeks, 24-25 weeks, 35 weeks (which is before birth, but not by a lot), and then the next major milestone to be months after birth. Those are estimates because some fetuses can be 1-2 weeks ahead or behind schedule. So if we’re going to track back to 24 weeks, we probably need to back up another 2 to cover the range of fetal development.

So to sum up, the fetus can respond to stimuli at like 8-9 weeks (recoil and movement originating in the spinal cord), it can’t process responses until 18 weeks (purposive movement originating from brain), 24-25 weeks you’ve got significant differentiation going on, and by 35 weeks we can actually identify the part of the brain where experiences are processed. We’re talking not just stimuli, but actual complex and higher thinking.

That’s why calling it “pleasure” or “pain” at 15 weeks is a little bit misleading, because the processing of the stimuli for the fetus at 15 weeks is very very different from how we would understand pleasure or pain. The neural processing might be closer to the response a fish would have to being poked. It would swim away.

Please keep in mind, it’s not a scientists job to make moral decisions. It’s our job to figure out how something works, not to say what it means.

I could say that the neural processing of a grown pig or a cow is capable of the same degree of executive functioning as an infant of say… 6 months, but that doesn’t mean we should stop eating meat or that we should start eating babies. A good scientist will not say “This is what happens, so therefore we have a moral imperative to do that.” He or she will merely describe the phenomena as it is observed. So don’t interpret this to mean “there is no moral issue with a 14 week abortion because they don’t process pain” or “we have a greater moral obligation to a 3 year old monkey than to a newborn infant because the monkey processes pain in a more complex way”. Pain is obviously only one component of the moral discussion here.

IRS Tea Party Document Disclosure in One Chart

In response to a Congressional Ways and Mean committee request, the IRS has provided 13,000 pages of documents related to the Tea Party targeting scandal. That sounds like a lot, but only out of context.

This is because, according to the IRS themselves, they have 65 million documents related to recent Tea Party targeting scandal. The difference between 13,000 and 65 million is wonderfully hilarious so I thought there should be a chart for it.

IRSDocs

Flawed Statistical Thinking, the IRS, and Loooooooong Odds

Let’s say there’s a casino where you can’t choose to play. Everyone is forced to participate in the gamble and instead of winning money, the “winners” get punched in the face. A man plays at the dice table 30 hours in a row and never gets punched in the face once. Lucky jerk. Then he says that the casino should be under different management. Within 15 minutes of this statement, he rolls snake eyes (the lucky face-punching number) three times.

Should we consider the possibility that his punches were perhaps the result of his statement about management? They could just be random. Randomness does work that way sometimes.

What if we then discovered that another table in that casino admitted to using loaded dice. Fans of the casino might argue that just because the dice were loaded at one table doesn’t mean they’re loaded at our poor sap’s table. But I think that most of us would take the triple face-punching and the loaded dice story, put them together, and conclude that management conspired to have their critic punched.

I float this analogy in service of recent events.

A few days ago, Nate Silver published a post noting the flawed statistical thinking from Peggy Noonan regarding IRS audits of Romney supporters. Go read it for yourself, but the long story short is that Silver is (rightfully) irritated by people who hold up a few anecdotal examples as proof of a conspiracy. However I suspect in his haste to make a larger valid point, Silver got the odds wrong on of one of the cases Noonan holds up.

In summary, Frank VanderSloot donated a substantial amount of money to groups in support of Mitt Romney. He was mentioned by name by President Obama’s campaign website in an attack that must have seemed normal to lots of people but seems mildly creepy to me. During the next four months, he had to deal with 3 audits: one individual audit, one audit by the Department of Labor and one audit of his business.

In a move that is very much unlike him and that I can only attribute to a careless reading of the events or “blogging-while-buzzed” (full disclosure: that is me right now) Silver has drastically miscalculated this case. Silver looks at the odds of a single audit and maintains (as any reasonable statistician would) that the odds of a single audit of a single individual do not rise to the level of conspiracy.

One of the things I love about randomness is how it builds. Flip a coin and call the result. Get it right and no one thinks you’re anything special. Same thing if you call the second flip correctly. By the third flip, people are getting irritated, waiting for you to be wrong. By the 10th flip, they want to change the coin because they suspect it’s rigged. Every subsequent flip drives the odds higher in a way that becomes almost impossible to comprehend. By the 34th flip, the odds of calling them all right far outweigh the population of the planet.

What we’re talking about here isn’t a single audit (which is what Silver based his math on). We’re looking at three different audits within four months of being mentioned by the President’s team as a “very bad man”.

What are the odds of that?

Fortunately for us, those odds are very easy to count. The odds of someone making over $1 million per year being audited is 12%. Let’s assume it’s similar for medium sized business. Then the odds of being audited AND your business being audited in the same year drops down to 1.4%. That’s certainly lower, but not so low that we should be overly suspicious. Hey… this kind of thing just happens.

But then add in the Department of Labor audit. The Department of Labor  “conducts more than 3,000 audits each year“. For the sake of statistical generosity, let’s call that 4,000. There are 30 million businesses in the United States, so the chance of an individual business being targeted randomly is pretty low, under 1%.

Now let’s add all those odd together. The chance of having all three of these audits in one year is 1 in 520,833 or generously rounded up to 0.002%. This is the kind of thing we look at askance and say “Huh. That’s really kind of weird.”

This would look weird even if the IRS hadn’t admitted to rigging the game (albeit in a different context). Given the admissions they’ve made so far, I think any serious mathematically minded individual should look at the odds and say “That’s odd.”

I feel that, given the circumstances, the burden of proof should lay at the feet of the auditors. I want them to prove this wasn’t politically motivated because, given that they’ve admitted to so far, this particular case is extremely weird.

This Is How We Disappear

(Skip to point 5 for my personal post-mortem on the Romney campaign)

I haven’t posted in 6 months and my Twitter presence (where I’ve usually done most of my interaction) has slowed to a crawl. I missed the last #BLSFriday (my monthly data-dig into the BLS employment data) and I haven’t made a decent chart in ages. Several people have asked what happened and I hate to just disappear without any explanation, so I wanted to put something up here.

I love being able to make a contribution to the political discussion. I love digging into data and asking questions that too much of the data community prefers to ignore. I love the people I’ve met and become friends with and my chances to speak and educate. But for a variety of reasons, I’ve had to pull back. I don’t like to just disappear and leave people in the dark (we miss you @Cubachi!) so I wanted to elaborate here.

1) I moved to the west coast

I didn’t think it would make a big difference, but moving into a time zone just 1 hour further from DC has really limited my Twitter engagement. By the time I get on at night, a lot of east-coast people I like to interact with have called it a night. I still try to check in frequently (and @stephenkruiser and @politicsofamy make the evenings pretty awesome) but it’s not the same interactions that I loved.

2) My new job requires my personality

Some of you know what my new job is, but for people who don’t, I’ll just say that it requires my personality. Whereas all my previous work relied on my ability to deliver a good product, this job requires that I put my face on my work in a big way. I’ve never been super-secret about my identity, but the nature of my new job requires that I keep my name and personality squarely in the professional sphere.

3) Baby + 2 year old

We just had our 2nd kid and our 2 year old is a delightful little time suck. As much as I love digging into data, building charts, making videos and arguing with the internet, I like spending time with my kids more.

4) This is a hobby

My political data work was fun, educational, engaging, and some of the best stuff I’ve ever made. I have gotten job offers by the dozen. But I have a career in which I make money. It’s not a huge amount of money but… well, let’s just say this conversation actually happened (although it is paraphrased):

Fox News: “Hi, this is (so and so) with Fox News. We’ve seen your stuff and we love it. What do you think about doing a regular piece for (show X).”

Me: Sound great. So… compensation… I was thinking [2/3 my going rate as a programmer].

FN: Yeah, that’s never going to happen. How about [1/6 my going rate as a programmer].

Me: Ha. Ha ha ha. Ha ha ha ha. Ha.

I think I actually laughed at them on the phone. The number was really that low.

We tend to think that people in DC make stupid huge money. But that’s true for very few people (usually corporate lobbyists and maybe some organization directors or higher-ups). Bloggers, media content creators, journalists… all these people get paid crap (with the exception of the very top-tier, let’s call them the 1%).

So, if there is a blogger, writer, video creator, podcaster, Twitter personality, etc who you enjoy, donate something to them. Anything is helpful.

5) Disillusionment

OK… now for the real reasons. In the last election, I was approached by someone in the Romney campaign to do some visualization work, charts, videos, that kind of thing. We agreed upon a reasonable rate for my work and I got started working on some concepts. The first visual I produced for the team was a variation of this visual, showing job growth by presidential terms.

The version I made for them was cleaner, better designed, conceptually a bit firmer, but the point was the same. After a number of iterations, I felt I had a great visual that I’d be glad to see be a point of conversation.

And then the approval process began. We spent weeks trying to get an OK on the visual. They asked for references for my data which I gladly included. (The only time I deny references to data is when people on Twitter refuse to do basic research and I want to know they’re willing to do basic research before I engage them.) The approval process for the most basic inoffensive visual showing how mediocre Obama’s jobs record was required the approval of a vast number of message managers, PR managers, researchers, etc. A single veto would kill the iteration and I’d have to resubmit with changes. Sometimes I knew what those changes should be, sometimes I didn’t.

After enough time it dawned on me: These people didn’t believe me. They didn’t believe my numbers (even though they were the most basic BLS numbers out there). I felt (and this is just my intuition talking here) that they had bought, hook line and sinker, the Obama teams “I created X million jobs” line (easily shown to be little more than a flimsy propaganda line based on selective data). I believe they were more willing to swallow the line being promoted by the opposition than a friendly voice with a history of dedication to the truthful portrayal of data.

At a certain point I said “screw this” and gave up.

And I never got paid.

I liked Romney. I voted for Romney (which, incidentally, marked the first time the candidate I voted for didn’t win). There are all sorts of reasons we can point to about why Romney lost. But from my perspective, I saw an over-managed campaign untrustful of their own side and unwilling to take the smallest risks for fear of being butchered by the media. Which, of course, happened anyway.

There’s actually one more reason, but it requires it’s own post. Suffice to say I’d love to keep making data beautiful, engaging the issues, digging into charts and making videos, but my life has changed substantially and for the foreseeable future

BLS Numbers: The Case For and Against Paranoia

At first, I thought the BLS numbers for September were weird. I thought about commenting on them, but I’m very slow in writing blog posts (what with trying to understand things and run the numbers and all) so I thought the moment had passed.

But then I saw Mickey Kaus’s post on “The Case for Paranoia” and I thought maybe I should add my 2 cents.

But first, I want to put forth my position. I hear a lot about how conservatives lack basic empathy, but I’ve been pretty frustrated at how liberals lack basic empathy over the results to this recent jobs report. Empathy is the art of seeing through the eyes of another human being, and it is a beautiful art… possibly the only true art. What I want to do here is to aid empathy. Why would someone be skeptical of the BLS numbers? Would they have any good reasons? And why might that skepticism be unwarranted?

The Case for Skepticism

If you look at this data one way, it actually looks very weird, very out of place. In September 2012 according to the BLS, the unemployment rate dropped from 8.1% to 7.8%, due largely to an increase of 873,000 jobs (as measured in the “A” Tables, which are based on a survey of individuals). However the “jobs increased” number (as measured in the “B” tables, which are based on a survey of business payrolls and is the number commonly reported) was only 114K, which is a pretty weak number. After all, just to keep up with population growth the job increase needs to be 125,000, right? So how can unemployment decrease so dramatically when the jobs number didn’t even keep pace with population growth?

Let’s look at every time in the history of modern job growth (since 1948) when we’ve seen +800K jobs increase in a month.

image

Something looks kind of weird here. In the last 18 years (not counting September 2012), the only times we’ve seen +800K job growth has been during January. Why is that?

It turns out every January, the BLS (Bureau of Labor Statistics) re-aligns the data to conform with population increases. So we may see employment increases that are augmented by population adjustments (and may not actually be “real” increases). So let’s take those out.

image

We’ve had such a huge non-adjustment employment increase only 6 times in the last 70 years. And, with those other increases, did we have similarly large corresponding “payroll jobs increases”?

It turns out this last September was the ONLY TIME IN THE HISTORY OF BLS DATA that we had an “employment” increase this large where the “payroll” increase didn’t even meet population growth.

In fact, since 1950, every single +800K employment gain has been joined by a +300K payroll gain. Outside of the census hiring in May 2010, we haven’t seen such healthy monthly payroll growth for any month since 2006.

You could even go a step further. This is also the first time we’ve seen an employment growth number this large that wasn’t preceded by 3 months of solid +200K payroll growth. So we’re looking at a fairly weird number here.

And this number, this number that is unique in the history of BLS numbers and is beneficial to the incumbent administration, just happened to come out just in time to influence an election that depends heavily on jobs numbers.

So, even if you don’t agree, I hope you can see why some people are skeptical of this jobs report.

However.

The Case Against Skepticism

We have to keep in mind that there are 2 surveys that look at job growth. Think of the “B” tables as some guys calling employers and asking “how many people do you have on payroll?” Based on that number they come up with the “job growth” data. In September, that was an increase of 114,000 jobs. Very weak.

But you can’t call a company and ask “how many people do you NOT employ?” so to determine unemployment, they call individuals and ask “are you employed or unemployed”? This survey becomes the “A” tables and they take number of people looking for work, divide it by the number of people unemployed and get the unemployment rate.

Because of this, the two numbers (payroll increases in the B Tables and employment increases in the A Tables) can differ greatly. You could call all the companies in the Fortune 500 and ask how many they employ and cover millions of jobs. But if you call 500 individuals, that’s such a small sample size, it is basically meaningless. So the “A” tables (individuals) has a higher margin of error.

And we see that margin of error if we look at the data a little more holistically.

image

For those of you who (heart) some numbers, the standard deviation for the A Tables is significantly higher than the standard deviation for the B Tables (293,000 for A Tables vs. 209,000 for B Tables). This means that we’ll see a higher level of variability in the A Tables (the 873K job number) than we see in the B Tables (the 114K job number).

But looking at this data in this way, we see a couple things:

1) The September jobs report is a CLEAR outlier. It is totally reasonable to raise some eyebrows at this.

2) When we look at all the data, and not just pare it down to a few data points like we did above, we can see the September jobs report isn’t enough of an outlier to be considered unique. It could very easily be an artifact of randomness. The randomness just happens to fall  in a way some people don’t like.

And that second position is where I am. There is a lot of variability in the jobs data, especially the A Table employment data. Add into this the fact that we saw a lot of part time jobs added (600,000 of the 873,000 increase was in part time jobs) at the same time that we saw some major employers announce a shift to part-time workers in response to Obamacare and we see that maybe this report isn’t some conspiracy, maybe it is actually telling us something about the changing status of employment in the country.

UPDATE: Conn Carroll points out that part time jobs as a whole did not increase by 600,000, but instead fell by 26,000. What increased by 600,000 was the number of people working part time “out of economic necessity”, but that shouldn’t have influenced the overall job number. Only the overall number of part time workers should do that.

END UPDATE

Is it a weird jobs report? Yes. But it isn’t unique in its weirdness and there are some very important extenuating circumstances that help explain it.

I’ve been following jobs reports very carefully for about 3 years. I’ve run through the historical numbers dozens of times, looking for averages, estimates, trends and patterns. For what it is worth, I don’t see anything that would suggest any kind of conspiracy or number tampering.

I know the numbers well enough to say that this was an odd report and I wish others would give the BLS skeptics a little bit of slack. This was a weird report, no doubt about it. But an understanding of how the report is compiled and a little bit of exploration shows that this report wasn’t so weird as to warrant particular skepticism.

Why Obama Is Always Talking About “Private Sector Jobs”

My latest video is from a talk I gave back in July at the RightOnline conference. I had 5 minutes to give a talk and I had something all planned out… until President Obama gave this speech in Cleveland. In this speech he stated:

Our businesses have gone back to basics and created over 4 million jobs in the last 27 months — (applause) — more private sector jobs than were created during the entire seven years before this crisis — in a little over two years

I decided to check him on his jobs claims and I summarized my findings in my talk, which I reproduced for this video.

I make 2 big points in this video:

  • Obama selectively chose specific dates to make his fairly weak jobs numbers look better
  • There is a more comprehensive jobs number (employment) that tells a very different story.

Deception Through Selection

And here is where I give a little more detail on what numbers I used. First a little background:

President Obama gave this speech on June 14, 2012, so at that time we were using the most recent BLS jobs report which had number up to May. Counting backward from there, that means Obama was counting from March 2010 to May 2012.

March 2010 – 106,914,000 private sector payrolls

May 2012 – 111,040,000 private sector payrolls (revised up 32,000 in later reports to 111,072,000)

Difference in Obama’s “27 month number” – 4.13 million private sector payrolls

image

I was assuming that when Obama said “before the crisis” he meant before we started losing jobs. That would put the “7 year” number from February 2001 to February 2008.

February 2001 – 111,623,000 private sector payrolls

February 2008 – 115,511,000 private sector payrolls

Difference in 7 years – 3.88 million private sector payrolls

image

As you can see, the Obama graph is a nice simply upward slope including only the part of his presidency where he gained jobs. In fact, he starts counting only after the jobs number completely bottomed out. If we look at the jobs record during his entire time in office, we get this chart

image

Is there any thing wrong with not counting those initial job losses? I don’t think so. I think it is a perfectly reasonable thing to do to say “let’s look at the strength of the recovery alone” and use that metric to count. But it is incredibly disingenuous of the Obama team to completely discount job losses for themselves but then turn around and count them in the comparison data point.

In the video, I point out that using “6 years before the crisis” or “5 years before the crisis” result in vastly larger numbers (6.4 million and 7.1 million respectively), but what I’m really interested in here (and what I’d like to expand upon) is comparing private sector payroll growth that Obama is touting to the private sector payroll growth under Bush.

I looked at this a couple months ago and was a little shocked to see the following chart, but here it is. Starting at the low point of private sector jobs growth, if we chart what I will (for simplicity sake) call the Bush recovery (starting in July 2003) and the Obama recovery (starting in March 2009) using the latest data, we get:

image

As you can see… the weird thing about this current recovery is how closely it is tracking to the previous recovery in terms of private payroll increases. For Obama to pretend he is substantially better than Bush on this metric is nothing short of fantasy.

The Larger Jobs Number (Employment)

Here is where things actually get really freaking weird. The Bureau of Labor Statistics (BLS) uses two numbers to count jobs. (See more about how the BLS counts jobs here)

The first one is the establishment data (B Tables) and this is a survey counts jobs by industry. Think of it as someone calling a bunch of businesses and asking “How many people do you have on payroll?” They directly sample over 100,000 businesses and it has a margin of error of about 100K jobs.

The second one is household data (A Tables) and this is a survey of households. Think of it as someone calling a bunch of people and asking “Do you have a job?” It samples about 60,000 households and has a much larger margin of error (400K jobs).

The establishment data is usually used for month-to-month job counts in part because it tends to be a much less volatile metric (household data can swing somewhat wildly). That’s why, when you hear about “X jobs gained last month”, they use the number from the establishment survey.

However, a weird thing happened in the 00’s with the household survey. If we take the private payrolls and compare them to what I’m going to call “private employment” (the A table employment number minus government jobs), we see a massive difference in the job count.

image

That’s a 3 million job difference between private payrolls and private employment. This is way outside the margin of error. Something happened there, althoughI’m not sure what. Maybe self-employment increased, or people made ends meet w/ irregular non-payroll income or farm employment jumped. I honestly don’t know and anything I say here is pure speculation. But there it is, clear as day.

This is why Obama focuses so much on private payrolls as the metric he uses. Most fact-check organizations are not savvy enough to notice that there is this huge discrepancy in the jobs data from survey to survey. They only think to check Obama’s statements against the private payrolls data, not the overall employment.

In contrast compare the chart above to the private payrolls vs private employment change since Obama’s inauguration.

image

As you can see, the change in both jobs numbers are nearly identical. If we add in government job losses, we actually get a negative number on employment change since his inauguration. This shows that something was happening in the last recovery that isn’t happening in this one.

How President Obama Is Using Your Address

I’m a very pro-transparency type of guy in most respects. However, I’m also practical. For example, I’m very nervous about publicly available donation tracking information because it means I can’t quietly support any candidate. There is always a chance that someone will find out who I supported and yell at me / egg my house / key my car.

With great power comes great responsibility and all that jazz.

Which is why I’m writing this post. I mostly like Mitt Romney for president (it’s a long story, buy me a beer and maybe I’ll tell you). But if I gave him my address for volunteer purposes or with any kind of donation and he turned around and broadcast my address to the world, I’m be pretty angry.

And that is exactly that President Obama has apparently done. With the new Obama campaign iPhone app, I can see who exactly in my neighborhood has their address (for whatever reason) registered with the Obama campaign. And I think it is important to let these people know exactly what has happened to this information. Call it “transparency”.

Note: DO NOT harass these people. We don’t know exactly where these addresses came from, but I am 90% sure that it did not come from public donation data. Even if it did, that doesn’t matter. This is about letting people know that their information is public for the world to see, not about giving them crap for their political views (which, let me say again, we don’t know).

Step 1

Download President Obama’s campaign iPhone app.

Step 2

Open app

Step 3

Tap the “Action” tab.

Step 4

Tap “Canvass”

Step 5

Log in or sign up for a new account (this will not place your address on the map, the sign up requires only a valid email address).

Step 6

Find all the people in your area that the Obama team wants you to visit.

Step 7

Take a picture of the map with their address on it (press the power button and the square button at the same time on your iPhone).

Step 8

Drop that image into this file. The text reads:

Hi!

I’m a concerned neighbor.

I got this address from the Obama campaign’s iPhone app. If you’ve ever given money or volunteer information to President Obama, he probably has your name and address. And he has made it publically available for anyone with an iPhone.

If this bothers you, you might want to contact his campaign and ask them to not share your information with people like me (even though I’m super nice).

Step 9

Print file (10-25 copies should do it).

Step 10

Tape fliers to all the doors on that list.

My goal here is not to bash President Obama. Rest assured if the Romney campaign did something similar I would be beating the hell out of them on this issue too.

This information may seem very public and therefore very harmless to political wonk programmers, but I can assure you that the people who gave their addresses


NEVER
expected that information to be used this way.

UPDATE:

Some have complained that the Obama is only using publically available data, so my problem shouldn’t be with him, it should be with campaign finance laws. May I therefore submit the following into evidence: Here is a map of who contributed to Democratic campaigns in 2008 (blue dots indicate financial contributions during the 2008 campaign):

And here is the exact same area map pulled from the Obama app

With this, I would suggest the Obama app is not using public contribution data, but instead using data from their own supporter database.

BLS B Tables (Jobs By Industry) Treemap

I’m going to try something that is a little dependent on me always being on top of things. So I can tell you right now it’s a terrible idea.

Nevertheless.

I’ve been working for some time to make BLS data a little more accessible to the average person (read: the average wonk) and this something of a high point on that project.

In summary: Every month on the first Friday of the month, the Bureau of Labor Statistic releases two tables of jobs data. The A Tables contain employment, unemployment, the unemployment rate and labor force numbers. This is where we get the unemployment rate from. The B Tables contain detailed payroll data and a breakdown of payrolls by industry and sub-industry. This is where we get the “XYZ new jobs” number from. Due to the level of detail in the BLS B Tables, there is a lot of insight to be drawn from which industries are rising or falling (including public sector vs private sector jobs).

I’ve created a system where I can quickly snag all the BLS data from the most recent jobs report and display it in a treemap visualization, making it easy to explore.

So… here it is (interactive version).

And here’s a static version

The size of the boxes are proportional to the number of jobs in that industry and are colored according to the growth in that industry over a given time period. You can adjust the time period to color the boxes according to growth over the last month, the last 12 months, since Obama took office and over the last 10 years.

If you have a slower machine or are looking at it on a mobile device, you might be disappointed. It is a somewhat large visual and it is optimized for traditional desktop interaction. However, I’m hopeful that I can keep on top of this and post this visual monthly as the BLS numbers are released.

Right Online 2012 Presentation

Apologies for the fact that I don’t have the text of my presentation in here, but I wanted to post my presentation from m Right Online panel.

Right Online Bad Data Presentation