The American Legislative Exchange Council has for several years attempted to provide factual underpinnings for its right-wing policy agenda through an annual publication called Rich States, Poor States. This report and similar ALEC documents have come under increasing attack in recent years for their shoddy research methods and misleading conclusions. ALEC has now struck back at its critics in a report by Eric Fruits and Randall Pozdena called Tax Myths Debunked. A portion of that document is devoted to research released last November by Good Jobs First and the Iowa Policy Project in the report Selling Snake Oil to the States: The American Legislative Exchange Council’s Flawed Prescriptions for Prosperity. Some of the key findings in that report were released in the summer of 2012 in a short piece called The Doctor is Out to Lunch. It is the latter piece that is referenced in Tax Myths Debunked rather than the full research report; we refer herein, however, to the full document and use its shorthand title, Selling Snake Oil. The full report was known to ALEC well before Tax Myths was released.
The first criticism leveled in Tax Myths is directed at an analysis in Selling Snake Oil of the factors leading to economic growth and rising incomes among the states between 2007 and 2012. In that analysis we argued that state economic structure — the composition of a state’s economy — is likely to play an important role in the short run in determining how well the economy fares; states more heavily invested in 2007 in sectors poised to grow in the succeeding five years would be expected to do better than states with a concentration of jobs in sectors that would be hit hard by the recession. Thus it was important to control for economic structure in a statistical analysis that attempts to identify whether the policy prescriptions of ALEC performed as advertised, leading to growth and prosperity. ALEC, in Tax Myths, appears to have completely misunderstood what was done in our analysis; their criticism seems to be based on the assumption that our model was predicting changes in the share of employment by sector. Instead we were simply using 2007 economic structure — measured by employment shares — to predict rates of growth in overall state GDP, employment, and personal income. Their criticisms make no sense and are completely off base; 2012 state GDP cannot be a cause of 2007 economic structure, which is the circularity they argue undermines our analysis.
Second, they argue that economists have found a strong relationship between tax policy and economic health, and cite two pieces of research in support. In Selling Snake Oil, we devote several paragraphs to a discussion of the many reviews of dozens of research articles over the past 30 years that have led to the conclusion that business taxes have, at best, a small effect on business location decisions. In our piece, we looked at the consensus among a large number of economists who have examined this question; in Tax Myths, they found two that supported their position and ignored the rest.
The third criticism is directed at several scatter plots and associated correlations that were presented in Selling Snake Oil. In those charts we were illustrating how states that were ranked high or low by ALEC in the first edition of Rich States, Poor States in 2007 actually performed in the time since then. Did the states that ALEC ranked high on their Economic Outlook Ranking (EOR) actually perform better than others? Since all ALEC provided was the state rankings (not an index number showing their relative strength or weakness), we correlated those rankings with the measures of performance that ALEC emphasizes: growth in GDP, employment, and income. ALEC argues a technical point here: The formula used to calculate the correlation between two continuous variables (the Pearson coefficient) is different from the formula used to calculate the correlation between two rankings (the Spearman coefficient). We had one ranked variable (the EOR), and one continuous variable, and used the Pearson coefficient.
To respond to this criticism, we converted all of the performance variables to ranks first, and then calculated the Spearman coefficient. The conclusions were the same (Table 1). Where there was no statistically significant relation using the Pearson formula (as was the case when we looked at the EOR as a predictor of growth in GDP or jobs), there was also no significant relation using the Spearman. Where there was a statistically significant and negative relation (high ranked states have lower per capita and median family incomes) using the Pearson measure, the same result occurred with the Spearman. In only one instance did results change: Our original analysis showed a negative but not statistically significant relation between EOR and the growth in state revenues. The analysis substituting the state rank in revenue growth and using the Spearman coefficient found a negative effect as well, but this time the effect was stronger and statistically significant.
Finally, Tax Myths presents an alternative to the analyses in Selling Snake Oil, correlating the state EOR each year with the June value of the “state coincident indices” published monthly by the Federal Reserve Bank of Philadelphia for each state. The coincident indices are based on four measures of the health of the state economy: non-farm employment, average hours worked in manufacturing, the unemployment rate, and wage and salary disbursements. ALEC found a strong correlation between a state’s EOR and the value of the coincident index.
The state coincident indices are designed for tracking the course of a state’s economy over time — whether it is sliding into recession or on a path to recovery — and are pegged to a value of 100 for every state as of 1992. They are used to compare states, but only in terms of the changes in the index over time. So the value of the index as of 2008 is a measure of that state’s growth rate from 1992 to 2008, since every state started at 100. However, a high value for state X in 2008 does not mean that state X has a healthier economy in some sense than state Y with a lower value in 2008, because state Y could have started out with a much higher level of prosperity in 1992 and still have higher incomes and wages than state X in 2008, despite growing more slowly. Furthermore, the correlations performed by Fruits and Pozdena are taken as evidence that ALEC policies, as represented by EOR, cause economic health, but they have done it backwards, in effect trying to demonstrate that conformance to ALEC policies in 2008 caused states to grow more rapidly from 1992 to 2008! So why didn’t they look at the policies in place as of 2008 and see if they predicted economic growth from 2008 to 2012? The answer is, because the correlations between the EOR in 2008 and changes in the state coincident index subsequent to that are near zero. This is not the result they were looking for.
In Selling Snake Oil, we argued that a more sophisticated approach to identifying the effects of a state’s EOR would entail a statistical analysis that controlled for economic structure, as described earlier. In fact, a Philadelphia Federal Reserve Bank economist in an article about the state coincident index explains how state economic structure is an important determinant of the path of the state economy, as measured by changes in that index over time. We decided to see how the coincident index measure of economic performance fared in our regression model. So we used our 2007 economic structure variables, along with either the EOR or several key measures that are components of the EOR, to predict the rate of improvement in a state’s coincident index from 2007 to 2012. The results were much the same as our previous analysis, using growth in GDP, employment, or income as the performance measures. In other words, when state economic structure is controlled for, none of the ALEC policy variables, including the EOR, had a statistically significant effect on the rate of improvement in the state’s economy over this period.
In sum, nothing in Tax Myths actually undercuts any of the analyses or conclusions in Selling Snake Oil. In fact the authors’ misinterpretation of our use of economic structure variables and misuse of the state coincident indices serve only to further confirm the shoddiness of the research sponsored by ALEC.