Sunday, July 29, 2007

California May Get Audits Right -- So What Took 'em So Long?

Since 1975, when the solution to the electronic vote counting (election verification) problem was first presented by NIST (then National Bureau of Standards) researcher Roy G. Saltman, some (but apparently not enough) advocates have tried in vain to convince election officials, legislators and policy-makers to implement it. California Secretary of State Debra Bowen's Post-Election Audit Standards Working Group (PEASWG) has finally taken the first step.

In their report entitled: "Evaluation of Audit Sampling Models and Options for Strengthening California’s Manual Count", published July 27, 2007, the team has apparently decided that enough is enough when it comes to unverified electoral outcomes in the State of California. Rather than suggesting a continuation of the present fixed-percentage manual tally of just 1% of the State's voter-verified paper audit records, or the sometimes ineffective and often inefficient "tiered" percentage approach which is all but a requirement for federal elections in Rep. Rush Holt's House Bill, H.R.811, the team has decided to get election audits right for a change -- and the change is long overdue.

According to their report: the tiered approach is "ad hoc"; there is no statistical justification for the particular tiers; and it is inefficient in some cases and inadequate in others. In some races, it requires far more auditing than is needed to discover any errors that might affect the outcome; in others, it does not require enough auditing. Can someone please say "Amen!" now?

It's nice to see that someone is finally getting the message that Congressman's Holt's office and many activists chose to ignore during the drafting of H.R.811. That said, I'm sure some members of the working group will be quick to point out that they support H.R.811 anyway because there is no better alternative in Congress at this time, and that H.R.811 allows States to come up with better alternative audit mechanisms. In my opinion, the States would be foolish not to.

Secretary of State Bowen herself has been quoted as saying that statisticians and not politicians should be the ones to decide how to audit elections, and she has so far been true to her word. Instead of just the "usual suspects" (computer scientists and election officials), the working group she put together to develop auditing recommendations also includes a statistician and a CPA.

According to their report, "The Working Group has reached a consensus that the most effective way to conduct post-election audits is to take a risk-based approach. The sampling model that works best for this approach is the adjustable sample model, where the size of the initial random sample depends on a number of factors, including the apparent margin of victory, the number of precincts, the number of ballots cast in each precinct, and a desired confidence level (e.g., 99%) that the winner of the election has been called correctly."

In a nutshell, the group proposes an enhancement to the work of Saltman, Stanislevic and Dopp, which has been published and cited in numerous papers since Saltman's 1975 seminal work, and most recently in Verified Voting's "Percentage-based versus SAFE Vote Tabulation Auditing: A Graphic Comparison", a draft of which was provided to the group at their first public hearing and via email and can be downloaded here:

The working group's version, which has yet to be developed, would address the issues of what to do in the event that discrepancies were found in an initial audit sample, and how to deal with contests that span multiple counties. Both of these issues have been addressed to some extent in the Verified Voting report and in the New Jersey election audit bill No. S.507, as amended. The bill requires the use of a relatively small, but apparently non-random .1% vote switch in the initial sample to trigger additional auditing. The bill and the VV report both state that each county should audit its pro rata share of the total precincts to be audited for a race, rounding up to the next whole precinct in each jurisdiction.

More work needs to be done to spell out in detail how this method can be applied in the specific context of California elections, but the California PEASWG are certainly on the right track, and hopefully, we will not have to wait another three decades before the solution first proposed by Saltman will actually be implemented. The only question now is: which State will be first to get election audits right?