A fundamental consideration in decision- and policy-making is aggregation of competing/complementary goals. For example, consider the current debate about how to measure when the “border is secure” with respect to US immigration reform. (A nice, though short, piece alluding to these issues is here.)
A recent GAO report discusses the state of border security, the variety of resources employed, and the panoply of challenges associated the the rather succinctly titled policy area known as “border security.” An even more on-point report was issued in February of this year.
Let’s consider the problem of determining when “the border is secure.” This is a complicated problem for a lot of reasons, and I will focus on only one here. Specifically, the question is equivalent to determining the “winners” from a set of potential outcomes.
In particular, there a lot of potential worlds that could follow from (say) a “border surge.” These worlds are distinguished by measurement, a cornerstone of social science and governance. For example, consider the following three measures of “border security”:
- Amount of illegal firearms brought across the border, and
- Amount of illegal cocaine brought across the border, and
- Number of (new) illegal aliens in the United States.
(Note that there are lot of ways to make this even more interesting, in terms of the strategic incentives of “the act of measurement.” For example, if you want to believe that the level of illegal firearms brought across the border is low, an arguable way to do this is to stop “looking for firearms.” But I will leave these incentive problems to the side and focus on the incentive to misreport/massage “sincerely collected” data/measurements. Furthermore, the astute reader will note that I could pull the same rabbit out of the same hat with only two measures.)
Before continuing, note that the selection of these measurements is left to the Secretary of the Department of Homeland Security (in consultation with the Attorney General and the Secretary of Defense) who is called upon in the bill to submit to Congress a `Comprehensive Southern Border Security Strategy,” which “shall specify the priorities that must be met” for the border to be deemed secure. (Sec. 5 of S.744, the immigration bill as passed by the Senate.)
In general, of course, there are multiple ways to indirectly measure—and no direct way to measure—whether the border is “secure,” (i.e., the notion of a “secure border” is one of measurement itself) and these must be aggregated/combined in some fashion to reach a conclusion.
On the one hand, it might seem like this is a simple problem: after all, for all intents and purposes, it is a binary one: the border is secure or it is not. End of story. AMIRITE? No, that’s not true. Because the issue here is that there are three potential programs to choose from.
To see this, suppose that there are three possible programs, plans A, B, and C.
Now, think about how you will/should measure if a program will result in a “secure border.”
The question at hand is how one compares the different programs. So, to make the problem meaningful, suppose that at least one of the programs will be deemed successful and at least one will be deemed unsuccessful (otherwise the measurement is meaningless).
The Gibbard-Satterthwaite (and, even more accurately, the Duggan-Schwartz) Theorem implies that such a system can/should not guarantee that one elicits truthful reports of the measurements of all dimensions (guns, drugs, illegal aliens) in all situations, even if the measurements are infinitely precise and reliable.
Why is this? Well, in a nutshell, in order to elicit truthful reports of every dimension of interest (i.e., guns, drugs, and illegal aliens), the system must be increasing in each of these measures. However, this is at odds with making trade-offs. In the context of this example, there are programs A and B such that A decreases guns but has no other effect, and B decreases drugs but has no other effect. In this case, which program do you choose? Putting a bunch of “reduction” in the black box, one must “eventually” choose between A and B, at least in some situation, because otherwise the measurement of guns-reduction and drugs-reduction become meaningless.
So, suppose that A decreases guns by “a little” but B decreases drugs by “a lot.” How do you compare a handgun to a pound of China White? Choose a ratio, and then imagine, if plan B was just a peppercorn shy of the “cutpoint” in terms of the reduction of drugs relative to the decrease in guns…but (say) A is $100Billion more expensive than B…what would you report about the effectiveness of B?
Well, you’d overreport the effectiveness of B (or underreport the effectiveness of A, possibly). AMIRITE? The measures are inherently incomparable until you choose how to make them comparable.
So…what does this mean? Well, first, that governance is hard—and perpetually so. But, more specifically “mathofpolitics,” it clearly and unquestionably indicate that theory must come before (practical or theoretical) empirics. In a nutshell: every non-trivial decision system is astonishingly susceptible to measurement issues: even when measurement is not actually a practical problem. For the skeptical among those of still reading, note that I only “played with” elicitation/reporting—I am happy to assume away for the moment the very real and fun issues of practical measurement.
With that, I leave you with this.
