I just finished to read the latest blog of John H. Cochrane and I learned of new things, you should read it to:
https://www.grumpy-economist.com/p/causation-does-not-imply-variation
Basically, Cochrane is arguing about the causality/credibility revolution in economics. The causality/credibility revolution is marevlous thing, for sure. But we are in the realm of social science in economics and we cannot make real experiences like in physics or in chemistry. Consequently, in order to have a credible identification of the causality, economists have to use very strict hypotheses and setups that often results in studying the causal effects of an X variable on a Y variable by shutting down all the alternative causal channels. This may come with the cost of studying the effect of an X variable that explain a very small share of the variance of the Y variable.
In that sense, causation does not imply variation. In the blog you have very interesting parts that I will reproduce below (italics and emphasis are mine):
“An extra year of college or growing up in a better neighborhood might raise wages. But only a tiny fraction of why one person’s wage differs from another results from extra years of college or which neighborhood a person grew up in. Minimum wages might raise, some find, or lower, others find, employment. But only a tiny fraction of the huge variation in employment from one area to another or one person to another traces to variation in minimum wages. If you want employment, other levers are likely far more important. Demand shocks might move stock prices. Nonetheless only a tiny fraction of stock price variation comes from demand shocks.“
A very interesting part on the addition of fixed effects:
“Studies typically add “fixed effects.” In a regression, y(state, time) = state fixed effect + b x(state, time) + error. A state fixed effect means we look only at the variation in a variable within a state over time, not how the variable varies across states. A time fixed effect means we only look at the variation of a variable across states, and not how it varies over time. It is common to add both fixed effects. Yes, that’s possible. y(i,t) = a(i) + c(t) + bx(i,t) + error is not the same as y(i,t) = a(i,t) + bx(i,t) + error, which would not work. Let’s see if I can state the source of variation in words. (A great seminar question: can you please state the source of variation in x in words?) We’re looking at x in state i at time t relative to how much x is on average in state i, and relative to how much x is on average across all states at time t, and how that correlates to similar variation in y. Hmm, I didn’t do an outstanding job of translating to English. (Stating the assumption on standard errors in words gets even more fraught. Just what did you assume is independent of what? Without using the word “cluster?”)”
About the role of controls that reduces variation:
“Next, researchers add “controls.” Controls should be added judiciously: think about what else moves y, how it might be correlated with the x of interest, and then bring it in from the error term to the regression. Control for taxes, regulations, or other changes that might have happened at the same time as a change in minimum wage. Instead of y = b1 x1 + error, recognize that the error includes b2 x2 and that x1 and x2 are correlated, so run y = b1 x1 + b2 x2 + error. Drinking and cancer are correlated. But people who drink also smoke, so you want to look at the part of drinking not correlated with smoking to see if drinking on its own causes cancer. But we are now looking for that much smaller population of drinkers who don’t smoke. Technically, controls are the same thing as looking only at the variation in x1 that is not correlated with x2. We throw out variation. Fixed effects are just one type of controls.“
About carelessly adding controls:
In fact, controls tend to be added willy-nilly without thinking. Why is this control needed? What are we controlling for? That seems especially true of fixed effects and demographic controls. Extra controls and often destroying the causal implication of the regression. Tom Rothenberg, beloved econometrics teacher at Berkeley, offered two great examples. Regress left shoe sales on price and right shoe sales. The R2 goes up dramatically, the standard errors drop, the magic stars appear. But now you’re measuring the effect of price on how many people buy a left shoe without buying a right shoe. More seriously, regress wages on education, but “control for” industry. The R2 goes up, we explain much more variation of wages (sort of where this post wants to go, but not this way). But the point of education is to let you move from the burger-flipping industry to investment banking, so controlling for industry destroys the causal interpretation of the coefficient.
About the price effect of change in quantities:
“Related but slightly different, most changes in quantity have no or tiny price effects, because they are anticipated. Most people trying to buy or sell financial assets are smart enough not to surprise the market. If you show up unexpectedly with a truck load of tomatoes outside of Whole Foods at 2 am, you’re not going to get full price for them. The Treasury, for example, routinely sells hundreds of billions of dollars of debt with essentially no price impact. Why? It announces the sales well ahead of time and talks to bond traders about the sale. Quantitative easing purchases of hundreds of billions had some impact effect when announced, but no detectable price impact when the Fed actually bought securities. Initial offerings amount to an infinite percent increase in supply of shares. Investment banks exist to popularize offerings, announce them, line up investors, and limit any “sloping demand curve” price impact.”
About Macro:
Macroeconomics
Macroeconomics should take a victory lap for being first to the table here. Chris Sims’ Vector Autoregressions taught us to look for the effects of a monetary policy shock by looking at the average events not following an interest rate rise per se, but only following unexpected interest rate rises. The trouble is, markets anticipate most interest rate changes very well, so true monetary policy shocks are few and far between. If we want to subdivide, for example to monetary policy shocks that persistently raise interest rates vs. those that die out quickly, then we have fewer data points still. (In contemporary theory, persistent vs. transitory shocks have very different effects.) The result, identified monetary policy shocks explain next to none of the observed variation in prices, output, and employment, and standard errors plus the effects of small specification changes are huge.
Some part of the conclusion:
“So, causality is great, but it isn’t everything. We often do want to know, “What are the major causes of growth vs. stagnation, wealth vs. poverty, recession vs. boom, and why do stock prices wander around so much?” Causal identification can chip away at this question, but obviously there is a long way to go. And it’s not obvious we will ever get there, since so much movement in the causes is and will always be endogenous.
Still, then, one should not mistake the answer to the small causal question for the answer to the disallowed big picture question.
As I think about macroeconomics and finance, I think there is good work to be done that does not just follow the causal identification format and allows us to address the big picture question. Sometimes broad facts fit one vs. another causal story in ways that cannot be captured by these techniques.”