This discussion might be a little hard to follow for non-economists, but two economists (Michael Keane and John Rust) recently tried to argue that the "Freakonomics" trend has gone too far, and it's time to bring in more "structure" to applied economics research and get rid of the cute-ness and cleverness. Keane makes some good points and is very fair, but I think Rust is both unfair and unscientific:
Here's Keane making a good point:
[T]he work by Card and Krueger (1994), Neumark and Wascher (1995), and Deere, Murphy and Welch (1995) on effects of the sharp increases in the national minimum wage from $3.35 on March 31, 1990 to $4.25 on April 1, 1991, and the New Jersey specific increase to $5.05 on April 1, 1992. These quasi-experiments provide important evidence that may help distinguish among competing labor market theories. But progress can only be made if one attempts to interpret the DD estimates of these authors in light of standard theories of labor market equilibrium, such as the search and matching model of Burdett and Mortensen (1989), as well as bringing more data to bear to help resolve some of the ambiguities of the original studies.
And here's Keane being fair:
It is often treated as a feat worthy of praise to simply estimate a structural model, regardless of whether the model can be shown to provide a good fit to the data, or perform well in out-of-sample predictive exercises. I see no reason why an estimated structural model should move my priors about, say, the likely impact of a policy intervention, if it fits the in-sample data poorly and/or has not been shown to perform reasonably well in any validation exercises.
Here's Rust being unfair and unscientific:
Instead young economists these days can do freakonomics and have Steve Levitt to look up to as their role model. Winner of the 2003 John Bates Clark Award, Levitt openly confesses his ignorance of economics "I mean, I just -- I just don't know very much about the field of economics. I'm not good at math, I don't know a lot of econometrics, and I also don't know how to do theory ... (New York Times, August 3, 2003)." Levitt exemplifies recent generations of MIT graduates who have been taught that the only tools one really needs in order to be a successful applied economist is a little knowledge of regression and instrumental variables.
...
[I]t is no longer necessary to have a model, or an innovation in econometric methodology, and the question doesn't even have to be important as long as one has a clever instrument and an entertaining topic. In fact, Levitt's work demonstrates that it is no longer even necessary to do the regressions correctly in order to achieve fame and superstar status in the "dumb and dumber" regime we're in. Two of Levitt's most famous papers are false: i.e. his key findings evaporate once elementary programming errors are corrected. This includes his controversial paper with Donohue that claims that the Roe vs. Wade decision was a cause of the significant reduction in crime in the U.S. in the 1990s.
The "elementary" programming errors are indeed embarrassing, but for someone who wants to be "scientific" (rather than just "clever"), I'm surprised that Rust bluntly states that these errors render the papers "false." I think it's completely inappropriate to dismiss the core findings of Levitt's papers because of these programming errors. This is one of the main reasons why we are taught to write empirical papers with a "collage" of statistical evidence, so that if one of the pieces turns out to be wobbly you still have several other legs that your paper can stand on. In the case of Levitt's famous abortion paper, he has six different pieces of evidence to support his theory. The programming error only affected one of the pieces of evidence (as you can read about here).
It's also a little unprofessional to say that Levitt's work demonstrates "low standards of academic excellence." I would bet very good money that many of the "structural" papers that Rust (and Keane) discuss have similar "elementary" programming errors. We don't know the gory details of these errors, however, because (1) it's very very hard to reproduce the results of structural work and (2) people aren't as interested in reproducing the results.
In some sense, I think the fact that people have found Levitt's errors so easily shows a clear benefit of "transparent" empirical work. Its simplicity is a virtue in terms of ease-of-replication. And I think almost everyone agrees we should be doing more replication.