Saturday, March 10, 2007

H.R.811: Fact & Friction -- Part II

Guest blogged by Mark Lindeman, Ph.D.*

In Part I of this series, Howard Stanislevic pointed out that the audits mandated by H.R.811 (as written) would be far too small to confirm the outcomes of some close elections – while being far larger than necessary to confirm the outcomes in other races.

You might wonder why it could possibly be a good idea to allow inadequate audits in some races while mandating needlessly large audits in others. (Even if you think there is no such thing as a “too large” audit, the misallocation of resources ought to trouble you.) I’m not sure, although I think Rep. Holt may be using an informal political calculation:

Last time I introduced a bill, people complained that a 2% audit sounded too small, so let’s require a minimum 3% this time – that’s fifty percent more audit! And let’s make the audits even larger in close races – but let’s not go over 10% no matter what, because election officials will get mad if they expect endless recounts.


And so, the Holt bill mandates 3% audits (i.e., 3% of precincts or equivalents) in most federal races, 5% audits in races with a winning margin under 2%, and 10% if the winning margin is under 1%.

Let me be clear: I like Rush Holt, and I don’t fault him and his people for trying to figure out how to pass a bill. And I think the 3% to 10% audits are not only “better than nothing,” but much, much better than nothing. Still, as compromises go, this one is strange. Not only do some people feel it is too soft, and some people feel it is too tough – but the numbers say that it is too soft and too tough. That’s interesting, but not really in a good way. I think we can do better.

First, let me say a word about why H.R.811 is much, much better than nothing. Some people suspect that the 2004 and/or 2006 elections witnessed vote miscount on a massive scale nationwide, on the order of several percentage points or perhaps more. There are about 180,000 precincts or equivalent units around the country, so H.R.811 would entail audits in at least 5400 precincts (3%) in every federal election. Audits that large – if truly random and immune from tampering – should detect vote miscount if it occurred in as few as one out of a thousand precincts nationwide, with 99.6% confidence.

Even a minimum audit of just one precinct per county selected at random (also required by H.R.811) would hand count the votes in about 3000 precincts, or 1.7% nationally. This audit would still detect vote miscount that occurred in as few as one out of 657 precincts with 99% confidence. If one out of every five presidential votes in each of these corrupt precincts were miscounted by the voting system so as to favor one candidate, that would only be enough to alter the popular vote margin by about 0.1%. And, similarly, a “clean” audit of this size would verify the total vote in Senate and House races to about the same tolerance. Not too shabby.

Unfortunately, verifying the approximate accuracy of the overall totals is not the same as confirming election outcomes. As we were all reminded in 2000, it actually doesn’t matter who wins the popular vote! It is very useful to have an upper bound on the overall extent of miscount, but that in itself would not confirm the outcome of the presidential election, nor any other election. Concentrated miscount could still alter the outcome in particular races – and, possibly, determine the presidency or the balance of power in Congress. Not so great.

The Norden letter: accentuating the positive

Holt’s office, sensibly, wants to portray its proposal in the best possible light. It has directed people to a letter, signed by nine prominent experts who consulted on H.R.811’s audit provisions, which seems to express support for the bill. (I will call this the “Norden letter” only because Lawrence Norden of the Brennan Center for Justice is the first signatory.) But as Howard pointed out, the Norden letter also points to some of the problems with the plan. In fact, it understates them.

The Norden letter embraces the general concept of a “tiered” audit – basically, the idea that audits should be larger in closer elections. For instance, on page 2, it argues that in an “imagined typical” congressional district, in order to have 90% confidence of detecting a miscount that would overcome a 1% margin, it would be necessary to audit 10% of the votes. The letter continues:

Mandating a 10% audit for all races would be a high burden on many States. And in the vast majority of races, a shift of 1% of the votes would not alter the outcome of the race.

In such races (says the letter), we might be “willing to live with the risk” of not detecting the 1% counting error – while in the small fraction of races in which a 1% shift would alter the outcome, we might not be. So far, so good. Certainly I think that an audit proposal should focus on delivering high confidence in election outcomes.

The trouble is, even by the Norden letter’s calculations, H.R.811 clearly does not deliver high confidence in the outcome of every race. The letter poses the question of how likely audits under H.R.811 are to detect errors that would change the outcome of particular races. Still assuming an “imagined typical” congressional district of “400 precincts of roughly equal size,” the letter presents the following probabilities of detecting miscounts just large enough to overcome various margins, for various audit sizes. (These calculations also assume that 20% of votes per precinct could be switched from one candidate to the other, without being too blatantly obvious.) The results for the audits mandated by H.R.811 appear underlined in bold.

Norden memo’s estimated probabilities of detecting outcome-altering miscount
[Edited to correct typographical, formatting and rounding errors]

# of pre-
cincts

Margin of victory

Probability in a 2% audit

Probability in a 3% audit

Probability in a 5% audit

Probability in a 10% audit

400

0.50%

10%

14%

23%

41%

400

0.75%

15%

22%

34%

57%

400

1.00%

18%

27%

40%

66%

400

1.75%

31%

43%

61%

86%

400

2.00%

34%

46%

65%

88%

400

5.00%

66%

80%

94%

99.6%


The letter rightly points out that the H.R.811 audits yield much higher probabilities than an across-the-board 2% audit would. That said, the letter carefully avoids speculating about how many people would be “willing to live with” as low as a 40% probability of detecting possible outcome-altering miscount. (If you don’t think that’s bad, keep reading.)

Note well that for margins smaller than 0.5%, the probability of detecting outcome-altering miscount will get smaller… and smaller… and smaller. Presently, some states provide for optional or mandatory recounts for races with very small winning margins, while others do not. People may disagree about how far Congress should go in mandating large audits for very close races. But let’s not kid ourselves that the issue will never arise. In 2006, the race in Connecticut’s 2nd district was decided by under 200 votes, about 0.1%. A 10% audit in such an election might confirm that the election was pretty close, but it certainly can’t confirm who won.

Even if we set aside very close races, the confidence problem is actually worse than indicated by the Norden letter. Unfortunately, the letter’s reference to “imagined typical” districts is only too accurate. In real life, congressional districts don’t have “precincts of roughly equal size.” That wouldn’t matter if we could expect that any vote miscount would be randomly scattered across precincts regardless of their size. But what if an attacker were able to target the largest precincts? Then fewer precincts would have to be miscounted in order to reverse the election outcome, and it would be even harder for a random audit to detect the fraud.

To give an idea of how much precinct size can matter, I examined vote counts in Ohio’s 18 congressional districts in 2004, and chose to use the 5th district as a baseline. Among Ohio’s congressional districts, the 5th had the greatest precinct size disparities – but the disparities are even greater in (for instance) New Hampshire’s congressional districts. So, OH-05 is a bad case, but definitely not the worst. I used the OH-05 size distribution to revise the Norden letter’s assumptions. For instance, the letter assumes that in order to overcome a 1% margin of victory, votes would have to be miscounted in 2.5% of precincts. However, in OH-05, the largest 1.1% of precincts contain 2.5% of the votes. Therefore, my size-adjusted analysis assumes that one could overcome a 1% margin of precincts by miscounting 1.1% of precincts, instead of 2.5% of precincts. (With rounding, that works out to miscount in five precincts instead of ten.) I make a similar adjustment based on the margin in each race – generally, the smaller the margin, the larger the adjustment. Adjusting for size, what happens to the probabilities of detecting outcome-altering miscount? They go down, a lot.

Size-adjusted estimated probabilities of detecting outcome-altering miscount

# of pre-
cincts

Margin of victory

Probability in a 2% audit

Probability in a 3% audit

Probability in a 5% audit

Probability in a 10% audit

400

0.50%

4%

6%

10%

19%

400

0.75%

8%

12%

19%

34%

400

1.00%

10%

14%

23%

41%

400

1.75%

17%

24%

37%

62%

400

2.00%

18%

27%

40%

66%

400

5.00%

46%

60%

79%

95.8%

If you think 40% sounded iffy, how do you feel about 23%, or 19%, or even less in a closer race?

Granted, the previous example is moving toward a worst case – but it isn’t there yet, especially since the Norden letter’s assumptions about number of precincts are also somewhat optimistic. Although it is true that congressional districts average about 400 precincts apiece, sixteen states average fewer than 300 precincts (or equivalent units) per House district. New Hampshire has a total of under 250 units, divided between two House districts, and these units vary wildly in size. In New Hampshire’s 1st district, just three units (out of 114) comprised almost 22,000 House votes in 2006 – over 11% of the total. In non-technical terms, New Hampshire is where percentage-based audits go to die. One nice feature of H.R.811 is that it would allow jurisdictions to adopt stronger audits; I hope that New Hampshire would!

It's one thing to complain that someone else's proposal isn't good enough, but the real question is: can the country feasibly do better? Howard and I are convinced that it can. Since H.R.811 audits too few ballots in some races and more than enough in others, it should be possible to attain more confidence in election results at little or no additional cost. To test our reasoning, we decided to examine all federal elections in the last three cycles (2002 through 2006) -- the presidential race, elections for all 100 Senate seats, and almost 1300 House races. We looked at the consequences of H.R.811's quirky allocation of audit resources, and we explored some alternatives. We will present the results of this analysis in Part III of this series.

* Mark Lindeman teaches Political Studies at Bard College in New York.

No comments: