The Dealergate Post That Will Make No One Happy

May 29, 2009

I noticed yesterday that a good number of people are getting worked up because it looks like a large number of the Chysler dealerships that are being closed are heavy Republican donors. (Michelle Malkin does her usual roundup here)

I’m taking the time to try to do something that still seems somewhat lacking… run an actual statistical analysis of the data. I’ll post more when I get some real data, but I did want to put up a couple thoughts early on.

Thought 1: Megan McArdle says that this is likely a red herring. She points out that “Democratic and Republican dealers are unlikely to be found in the same place, and the rural counties that tend to be red are probably less profitable.  I would be less surprised to find out that the administration rescued specific donors from the hit list than to find that they deliberately closed Republican dealerships.”

If there was any behind the scenes work by the Obama administration, saving Obama dealerships seems more likely than spitefully killing Republican ones. And I think that we’ve got a pretty big “if” there to begin with.

Thought 2: All the skeptics to this story are pointing to Nate Silver’s “Car Dealerships are Republican (It’s Called a Control Group, People)“. Unfortunately for them, that post is a load of statistical garbage.

Nate is trying to establish a baseline of Republican-to-Democratic donations against which he can judge the validity of the data coming from the closed dealerships. This is a laudable goal, but I get really frustrated when people use statistical or mathematical terms and they don’t know what those terms mean. I’m starting to understand that people on both sides of the isle use “science-y” or “math-y” words because it makes it look like they’re using science and can therefore be trusted. That’s exactly what is going on here.  

Nate’s investigation does not a control group make for the following reasons:

  • There are really three categories here: Republican donor, Democratic donor, and not a donor. He doesn’t even recognize that the last category might exist.
  • He don’t make any distinction between Chrysler dealerships and other dealerships. Maybe Honda dealerships skew Republican and thereby mess up his “control group”. This is like testing a drug aimed at teenage girls and building a “control group” that includes toddlers, WWII veterans and 40-year-old soccer moms. His data is hopelessly polluted.
  • He assumes that everyone who owns a car dealership will list their occupation as car dealer (or some variant). Where I grew up, Hank Aaron owned a couple car dealerships, but I think it was unlikely he listed his occupation as “car dealer”. (If I got a business card from Hank Aaron, I would want it to say “Hank Aaron – Awesomest Person in the World… and Barry Bonds Can Die in a Ditch”)

Take your pick. I got more.

Thought 3: That fact that Nate Silver’s “analysis” is a load of crap doesn’t make the other analysis better… it just makes him something of an ass for pretending that he’s better than everyone else.

Example 1: Dan Collins says:

Statistics that are available suggest that Chrysler auto dealers donated 76% Republican and 24% Democratic.

Looks like someone else didn’t control for non-donating dealerships. (UPDATE: Dan Collins comments below that this statement was revised, although I still don’t see anyone taking into account non-donors.) 

Example 2: Doug Ross has a post called “Dealergate: Stats demonstrate that Chrysler Dealers likely shuttered on a partisan basis“. Towards the bottom, he has a “What Are The Odds” section in which he notices that one company, RLJ-McLarty-Landers, has six Chrysler dealerships that were not closed and claims that:

The approximate odds of such an occurrence can be calculated

He then proceeds to “calculate” those odds based on the assumption that the dealerships were closed at random.

His odds are meaningless. What is RLJ-McLarty-Landers happens to have remarkable market share? Or excellent customer service?

To posit an imperfect analogy, it’s like me being surprised when all the K-Marts in my area go out of business. So I do a statistical sampling of all local supermarkets and say “Ah-ha! All the Wal-Marts in the area didn’t go out of business… what are the odds of that?” And then I calculate the odds out and claim that there are nefarious plans afoot. (I love that word… afoot. Afoot, afoot, afoot.)

Thought 4: This smells like a conspiracy theory. I hate conspiracy theories. I lean toward believing that people, Republicans and Democrats, conservatives and liberals, are good people who are trying to do what they think is right.

On the other hand, if I had been editor at the Washington Post in the 70’s, I probably would have told Bob Woodward and Carl Bernstein that they were acting like crazy people.

I confess to a heavy skepticism. So I’m running the data as carefully as I can and I’ll post what I find. It might take a couple days, though.  I’m not quite ready to quit my job to chase this story full time.

If you’re looking for what seems to be the best work on this so far, it’s probably at the entertainingly named Chrysler Dealership Campaign Donation Information blog. Based off an extremely quick scan of the information, it looks like Joey Smith (the author) is trying to gather data in a meaningful way.

11 Responses to “The Dealergate Post That Will Make No One Happy”

  1. Dan Collins Says:

    We revised that, further along, and it will take more revision. The point is to make all of the pertinent evidence available, first, and to get someone to enunciate the criteria on which the decisions were made, and then to see how those criteria were applied.

    Considering that Gibbs says it was an internal Chrysler decision, and the deposed CEO suggests that it was not, the first step would be simply to determine who’s calling the shots.

  2. MJD Says:

    Maybe if you had been editor of the Washington Post in the 70’s, you would have reported a fuller story not only of Watergate, but the internal politics at the FBI. Check out
    http://www.stratfor.com/weekly/20081222_death_deep_throat_and_crisis_journalism

  3. Matt Mentecky Says:

    With regards to thought 2:

    Look, I am not a professional statistician, I do not work using statistics (I take it that you do somehow), but my first point, is that it seems to me that your criticism of what Nate Silver did revolves around the fact that he had an imperfect data set to work with. My question is, isnt that the cause a lot of the time when drawing conclusions from given data? Second, is it your position that no one can draw any conclusion any time that data is flawed? I.e. if data is flawed, it is entirely useless?

    It almost seems to me that what you complain of is always the case in statistical surveys, and it seems like chasing a “perfect” data set is mythical and that statisticians can always find a reason why a given survey is flawed, its an art.

    Example: Every single analysis based on data gathered via telephone is flawed because it obviously ignores those who do not have a telephone. Right?

    Suppose Nate Silver titled his blog “Michelle Malkin is wrong because…” and then gave his analysis. Certainly his points would be valid then wouldnt he? Because her conclusions suffer from the same problems that you accuse Nate Silver’s of, no? And yet he realies on the same dataset that she does, and returns an opposite conclusion from hers. If thats the case, then certainly Nate Silver’s post isnt garbage, Michelle Malkin’s thesis is wrong given her reliance on X and heres why…. I recognize that his post isnt written in that tone but also recognize that his post is not completely useless if viewed this way.

    If Michelle Malkin is hurling an accusation and backing it up by X data and Nate Silver is using X data and providing information that refutes her position, its not his imperative to affirmatively disprove her thesis, merely showing that its flawed, is good enough, IMO.

    I could use an analogy but I suppose if it mirrors the current case youll simply disagree with me:

    Suppose that some children were expelled from a high school, an investigative reporter found out that a majority of them were christian and concluded thats why they were kicked out. But another reporter comes along and states that the high school Christian After School Club has 8x the membership than the Atheist After School Club.

    Under your criticism, the analysis from the 2nd reporter is COMPLETELY USELESS, but I doubt that most would see it that way. No doubt that random survey with a control group would be *better* but that does not mean that the analysis is *useless*.

    • politicalmath Says:

      Matt,

      My problem with Nate Silver is that his post leads his readers to believe that he is doing real analysis… as opposed to those other guys who are talking out of their butts. He is saying that his analysis is valid, while their analysis is bull***.

      Ultimately, the fact of the matter is that he uses a term that is specifically mathematical/statistical in nature to indicate that he is using (if I can mock him lightly for a moment) “real statistics so people will think I is smart”. And he does this while bashing other people for not using the smart people statistics.

      I object to his use of the term if he isn’t going to use it properly. His “control group” isn’t a control group. Period.

      Does that make Michelle Malkin right? Nope. I did not intend to use Malkin as a sterling example of statistical brilliance, but as a way of helping my reader get up to speed on what people are saying without me having to regurgitate it myself.

      Finally, please don’t take this the wrong way, but the rest of your comment kind of comes under the “It’s not even wrong” category of things. I could try to explain how we control for bad data and how we recognize differences that are statistically significant, but that’s a topic best left to a stats class or a good book like this one or this one.

      The only thing that I can do right now is say that, from a statistical point of view, someone may have the right answer, but I don’t see many people doing the work right. And until someone does the work right, I can’t tell you which answer is correct.

      Assuming Nate is right (and I’m operating on that assumption), you’re kind of saying I shouldn’t criticize him for getting the right answer. But his answer is the equivalent of saying that 2+2=4 because elephants are purple in the winter and the roads are made of cheese. He might be right, but if he is, he’s right completely by accident.

      • Matt Mentecky Says:

        I originally became aware of you because the Volokh Conspiracy linked to your Obama/Bush budget graphics, I was intrigued because, I am admittedly a lefty and only read VC because although they are slight right, they are well written and thought out and I was looking forward to reading your stuff because I thought it would elevate debate and provide insight. Instead I have gotten links to Cartoon books about statistics and an inability for you to go beyond your initial thesis in a very annoyed and lame way. In pointing out that your economic road trip doesnt take in to case GDP, I get an analogy about a car salesman (Did you know Zimbabwe spends less net dollars than we do but that that is still nearly 200% of their GDP?)

        It is clear that you have irreconcilable disdain for someone that may even agree with a majority of what you say but questions you, this is sad and not a good way for you to build a following and I feel bad, I thought you had such promise, I was wrong, good luck with your narrow view on life in which you refuse to even explain any of your beliefs. Take care

  4. politicalmath Says:

    I’m sorry… I only recommended those books because I thought you wanted to know about statistics and they are fantastic resources for people who don’t have time to take a real class. I didn’t recommend the cartoon book because I think you’re stupid. I recommended it because it is an awesome book and a fantastic place to start for people who want to learn statistics.

    Here’s the problem, Matt. If you don’t know anything about statistics, it’s going to be hard to have a conversation about statistics. The questions you asked in your first comment are questions that indicate that you’re not really up to date on issues related to data gathering and statistical modeling.

    That’s why I’m not really bothered about your accusations. I’m sorry if you’re angry… I truly am. I’m not trying to be disdainful or flippant or anything like that. But you’re basically getting angry because you don’t understand something. This is not my fault and I’m not going to feel bad about it.

  5. 5280grl Says:

    I think Matt believes you are on to something😉 Looking forward to hearing the end of this story. Thanks for keeping it real @5280grl

  6. Becky Says:

    I think you have great promise. I find the visual presentation of the budget to be important. As someone with a great love for mathematics and the arts, I know that the human concept of large numbers is flawed. The trend in emotive verses logical reasoning favored by educators further disrupts understanding.

    I like you approach, and know that in any field the more you know, the less cock sure you are (at least it is my own experience.)

    I read an architect that said the difference between a novice and and experienced architect, is that a beginner takes a lot less time to get something wrong.

  7. legion Says:

    Select x balls from an urn filled with y balls. The balls are numbered 1-4. X is the number of dealers that were closed, and Y is the number of total dealers.

    The sample space of the experiment is the following:

    S={1,2,3,4} where 1 corresponds to no donations, 2 corresponds to Republican donations, 3 corresponds to Democratic donations, and 4 is donations to both parties.

    By measuring distribution of S in the sample set using the donation data we can calculated the Expected Value and Variance of the conditional probability experiment (selected for closure given republican donor).

    See if these results match with reality.

    This is an easy way to use discrete probability to solve this problem. There are more rigorous ways to do this but even binary problems like this end up having 4 possibilities, such as the radar problem where you calculate if a station has observed a real enemy contact based on an incoming signal or a false alret. I don’t think this is necessary for such a silly problem.

    However, because your ‘side’ is the one making the accusations I believe it is your responsibility to prove that there was foul play, due to long standing traditions in Western society (which were convieniently cast aside when making the case to invade Iraq).


  8. Fantastic site. Good Lord above and pass the ammo, this is JUST the ticket to help the boys and girls of all ages see the seamy side of Things Political when it comes to math.

    Curious, though, I just got finished with someone (Slate editor, methinks, tring to hide himself as one of us commoners) who pulled out the magic hat on math purporting to demonstrate that Cash for Flunkers, Clunkers, whatever, was a shining success story!!!

    It reduced old inventory and got those old wagon trains-a-movin’, baby!

    What say you to that, surely there is some sort of nifty visualization on that! Or maybe I missed it elsehwere.

    Ooooh Nellie.

    What is your background, BTW?

    In any event, it matters not, as you obviously know how to crunch the digits, and thusly are one of the proud and few now to be added to my blogroll.

    Good show. I look forward to poking around your site after the Holiday/Festivus/Kwanzaa/Pagan Tree rituals, etc.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: