Welcome, brain-washed masses indoctrinated by the frequentist social science numeroacracy: everything you know is wrong! Read the paper, by an "old fart who in any reasonably prudent traditional culture would have long ago been strapped into a leaking canoe and sent out with the tide" that is inspiring 20-somethings, and pissing off Boomers, across the country!

[Okay, okay, it's not that big a deal, but it was a top download from the 2010 APSA meeting and I just realized it wasn't on this site anywhere]

Seven Deadly Sins of Contemporary Quantitative Political Analysis
(Original APSA paper)

A combination of technological change, methodological drift and a certain degree of intellectual sloth and sloppiness, particularly with respect to philosophy of science, has allowed contemporary quantitative political analysis to accumulate a series of dysfunctional habits that have rendered a great deal of contemporary research more or less scientifically useless. The cure for this is not to reject quantitative methods---and the cure is most certainly not a postmodernist nihilistic rejection of all systematic method---but rather to return to some fundamentals, and take on some hard problems rather than expecting to advance knowledge solely through the ever-increasing application of fast-twitch muscle fibers to computer mice.

In this paper, these ``seven deadly sins'' are identified as

• Kitchen sink models that ignore the effects of collinearity;
• Pre-scientific explanation in the absence of prediction;
• Reanalyzing the same data sets until they scream;
• Using complex methods without understanding the underlying assumptions;
• Interpreting frequentist statistics as if they were Bayesian;
• Linear statistical monoculture at the expense of alternative structures; \item Confusing statistical controls and experimental controls.

The answer to these problems is solid, thoughtful, original work driven by an appreciation of both theory and data. Not postmodernism. The paper closes with a review of how we got to this point from the perspective of 17th through 20th century philosophy of science, and provides suggestions for changes in philosophical and pedagogical approaches that might serve to correct some of these problems.

Paper prepared for the theme panel ``A Sea Change in Political Methodology?'' at the Annual Meeting of the American Political Science Association, Washington, 2 - 5 September 2010

Link to Adobe .pdf file of the paper

Link to Adobe .pdf file of the May-2013 "compact core" 8,500 word version prepared for Journal of Peace Research (they asked...), which concludes with an updated and inspirational personal coda on my assessment of the future of quantitative work in the academic community.

Presentations

All files are in PDF format.

• University of Kentucky and UMass/Amherst November 2011
• Canadian Political Science Association May 2011
  

Supplemental Online Materials

Georg von Kalckreuth sent this picture of a sculpture of the felling of Thor’s oak taken at this location. Note that St. Boniface appears to have accomplished the deed without visible woodworking tools (saints do that sort of thing, I suppose), though he is giving the Cub Scout salute. Critically, there is no evidence that the image has ever been struck by lightening, though—perhaps to hedge their bets—the builders appear to have prudently protected the work with a metal roof. Electricity theory good; Thor theory not so good.

Hmmm,yes...the book. So given the enthusiastic reception this has received in at least some quarters, it was suggested that I expand it into a book. Under normal circumstances, a great suggestion, as the original paper was actually condensed somewhere, to say nothing of a tad incoherent in places, and I also had an assortment of unpublish[ed][able] material from earlier critiques I had written on methodology. So a book seemed like just the prescription.

Except for the nature of the author, who can best be characterized as "this is where book projects go to die..."

But life is the triumph of hope over experience (right?) and well, I'm working on it. Given the deep love I have expressed in Chapter 8 for both publishers and reviewers, and the emerging success of electronic self-publishing, that is the route I intend to follow. The book will be available in three editions—Big Mac™, six-pack, and pizza—and priced at the approximate [US, not Oslo] cost of each of those items: the books will be identical except for the item on the cover, a variation on the "select your own price" model. In any case, affordable. If I sell enough copies, I'll have it professionally edited.

After some experimentation, I've come up with a table of contents I'm reasonably happy with, and I've completed one chapter (in keeping with my conventional mode, not part of the original scheme). Others will be posted as they are completed before the whole thing is packaged in one of the electronic forms.

The work is licensed under the Creative Commons Attribution-NonCommercial 2.5 License (yes, people who recognize Creative Commons nuances, you read that correctly...)

Current Table of Contents

• 1 The Problem
• 1.1 How we got here

• 2 The Seven Deadly Sins of Quantitative Analysis
• 2.1 Greed: Kitchen-sink models and the problem of collinearity
• 2.2 Pride: Pre-scientic explanation in the absence of prediction
• 2.3 Sloth: "Insanity is doing the same thing over and over again but expecting different results."
• 2.4 Lust: Using complex methods without understanding the underlying assumptions
• 2.5 Wrath: If the data are talking to you, you are a Bayesian
• 2.6 Gluttony: Enough already with the linear models!
• 2.7 Envy: Confusing statistical controls and experimental controls

• 3 The Siren Song of Reductionism: expected utility, game theory, systems dynamics and agent-based models
• 3.1 You can't get there from here...
• 3.2 Sources of Error
• 3.3 Chaos
• 3.4 Open complex systems
• 3.5 In Defense of Induction
• 3.6 When are heuristic models useful?

• 4 Prediction, Causality and Explanation
• 4.1 The Primacy of Prediction
• 4.2 The Conundrum of Causality
• 4.3 The Rascally Irritant of Explanation
• 4.4 The Fallacies of Frequentism

• 5 Twelve Principles for Social Scientic Realism for the 21st Century
• 5.1 An objective world exists that is independent of our perceptions
• 5.2 Human behavior is not infinitely mutable
• 5.3 Accurate forecasting is the gold standard for the validity of a theory
• 5.4 Statistical knowledge must be accumulated on Bayesian rather than frequentist principles
• 5.5 Reliable models should converge across models and theories
• 5.6 Formalization is a good thing
• 5.7 Credible and broadly applicable formal universals exist, but these are insufficient to fully reduce all behavior to universals
• 5.8 Theories are generally valid only to the extent that the propositions are credible. Despite what Milton Friedman told you
• 5.9 Induction is every bit as important as deduction
• 5.10 There are not qualitative and quantitative methods, there are just methods
• 5.11 Cases cluster: Knowledge in the social sciences is structured like that in medicine, not physics
• 5.12 "We don't know" is a valid answer
• 5.13 Conclusion

• 6 Clothing the Emperor: Advice to Policy Practitioners
• 6.1 Suckers...
• 6.2 Quantitative models work
• 6.3 SMEs, wonderful SMEs
• 6.4 A month in the lab can save an hour in the library
• 6.5 Design matters
• 6.6 There is no substitute for prediction
• 6.7 An 800-lb gorilla sleeps wherever it wants, but throwing money at a method whose fundamentals are flawed won't fix it
• 6.8 Secrecy is the enemy
• 6.9 Horse cavalry in the 20th century
• 6.10 Integrating Qualitative and Quantitative Research: Cautions and Opportunities
• 6.11 A checklist for evaluating quantitative work

• 7 What is to be done?
• 7.1 Forecasts and Contingencies: From Methodology to Policy
• 7.2 Practical Predictive Models
• 7.3 Fencing in Frequentism

Appendix: "If I ran the university..."
• 8.1 North American University Reforms
• 8.2 Things the Boomers actually improved
• 8.2 Problems the Boomers created
• 8.4 The internet is not a film strip
• 8.5 The journal model is broken...let us count the ways...
• Bibliography

Chapter Eight: "If I ran the university..."

Link to Adobe .pdf file of the chapter

Introduction

This chapter is a relatively late addition to the manuscript. The origins began with my reactions to Steve Jobs's famous Stanford commencement address, possibly the only memorable commencement address in human history, which was replayed endlessly following his death in October 2011, coinciding with some of my road-trip presentations of 7DS. Jobs begins the address with a poignant story of having to drop out of an expensive private college on realizing his parents could never afford the tuition, but then how auditing one seemingly frivolous course would eventually change the world of personal computing. In a single life, the dark side and the light side of contemporary higher education.

Second, once I started thinking about these issues, I began noticing a rather large literature expressing similar views: Arum and Roksa [2011], Christensen and Eyring [2011], Ginsberg [2011], Keeling and Hersh [2011] are a small sample of what appears to be a burgeoning---and generally convergent---literature. In fact, the chapter was quite hard to finish because new material kept showing up in my day-to-day reading.

The presence of a large number of academics and management consultants all saying the same thing does not, of course, necessarily mean that change is upon us: Consider, for example, the extended discussion of the inevitability of Japanese economic hegemony appearing in the years just prior to Japan entering two decades of economic stagnation. But, as I discuss in Section 8.7, I do, in fact, believe that ``this time is different'', and in Section 8.1, I observe that historical patterns suggest that the dominant university model may be at a point where it will naturally undergo a period of radical change.

Finally, as I've noted at numerous points, this manuscript seems to be getting read by graduate students, and whether I'm right or wrong, these issues are going to confront you as professionals. And if I'm right, the future may be quite different than the past.

Outline

• North American University Reforms
• Things the Boomers actually improved
• Problems the Boomers created
• Cost
• The Student as customer
• Incoherent management and governance
• Degree-granting sports franchises
  (your institution could be the next Penn State...)
• Decline of the humanities
  (really)
• Adjuncts [maybe]
• The internet is not a film strip
• The journal model is broken...let us count the ways...
• Losing the race against time
• Proprietary Journals
• Peer review

LAST UPDATED: 26 JULY 2012