Since 2006 when this paper [PDF] was published by VoteTrustUSA, it's been widely acknowledged by statisticians and election integrity advocates that the outcomes of many US House races, and smaller state and local contests, can not be confirmed with high confidence using small-percentage precinct-level audits of paper ballots originally counted by computerized ballot scanners -- even if the chain of custody of all the paper ballots could be verified.
These peer-reviewed papers estimated the scope of the problem by examining almost 1,400 Federal elections over six years. The authors found that the winners of more than 17% of all Federal contests from 2002 to 2006 could not be confirmed with high confidence using a 3% audit of precincts, known in NY as Election Districts (EDs) -- even if the audits showed no erroneous vote counts.
Think about that for a moment: even if the 3% audit did not find a single miscounted vote, the winners of many elections counted by computerized voting systems could still be wrong -- and no one would ever know.
This white paper [PDF], published in 2007, covered six years of New York's Federal elections, and extended the study to the 2006 New York Senate and Assembly elections. The author found that in NY, the winners of 14 out of 87 US House races, 7 out of 62 State Senate Races, and 32 out of 150 State Assembly races could not be confirmed with high confidence using a 3% audit of Election Districts.
All of the above papers were sent to the State Board of Elections, New York's election integrity community, and various academics upon publication.
It's worth noting that some statisticians have criticized this work primarily because, they say, it's not sufficiently rigorous. They favor larger and more conservative audits [PDF] that assume even more votes could have been miscounted within the precincts. So the above papers may serve as a floor rather than a ceiling on the number of electoral outcomes that would not be confirmed with 3% precinct-level audits.
While paper ballots are easy for voters to mark, and easy to count by hand, the first paper cited above also showed how today's optical scanners, when used to their full capacity to count thousands of ballots each, are particularly unsuitable as audit units in elections. The reason is simple: for a given percentage of audited machines (such as 3%), larger machines result in fewer machines actually checked for vote-count discrepancies. This results in much lower confidence that the winners of elections are correct, than an audit of 3% of the smaller-sized Election Districts or precincts (or fractions of EDs counted by particular machines).
The calculations in all the papers cited above are therefore optimistic as far as New York is concerned, because New York will not audit its elections by precinct -- but rather by scanner. This means there will be uncertain outcomes of even more contests than previously estimated.
Consider that a problem in a single ballot scanner could miscount the votes cast on up to 4,000 ballots. Unless that scanner were picked to be audited, the problem could go undetected. With a 3% random audit, the smaller the total number of scanners, the greater the chance that a particular scanner (such as one with a 4,000-vote error) will NOT be audited. The same problem exists even if we assume, for example, that "only" 20% of the votes on a scanner could be miscounted.
By contrast, each of New York's 20,000 lever voting machines counts fewer than 1,000 ballots. Lever machines do not count votes with unobservable software, and unlike computers, lever machines can be reliably tested. Levers can't switch votes during an election either, the way scanner software can [PDF]. And of course, levers don't allow voters to cast overvotes, as scanners did in high numbers in Florida last year [PDF].
On July 25, 2009, after years of trying to influence the State Board of Elections to write effective auditing rules, advocates from around the state submitted their public comments on the 3% audits prescribed in the current draft regulations. One such comment [PDF] by New Yorkers for Verified Voting (NYVV) stated that:
"in many state Senate and Assembly races an audit of this size could easily overlook miscounts large enough to alter the outcome of a competitive contest."While NYVV did not specifically mention Federal elections, it's clear from the above peer-reviewed literature that they too are at risk.
Below, following years of much more detailed comments, discussions and suggestions, are our public comments on New York's proposed auditing regulations [PDF].
Among other problems, as of this writing, it is not clear to us whether our State Board of Elections has even been able to agree on the definition of a "complete audit" in the Election Law. A request for an interpretation sent to their legal counsel remains unanswered. However, it should be obvious from NYVV's comments, and the Law itself that "complete" means all the ballots on which a contest appeared. The Law states:
"If a complete audit shall be conducted, the results of such audit shall be used by the canvassing board in making the statement of canvass and determinations of persons elected and propositions rejected or approved. The results of a partial voter verifiable record audit shall not be used in lieu of voting machine or system tallies." (EL § 9-211 4)However, the SBoE and our county boards of elections may see things differently. They may have decided that the only way a candidate will be able to get a recount is through the courts.
Election Auditing Regulations
Regardless of mathematical considerations, if the SBoE does not interpret a "complete audit" in EL § 9-211 as a hand count of all the ballots in a contest, then candidates will have no right to a full recount under NYS Election Law and the audit will not limit the risk of incorrect outcomes of elections.
The mandatory 100% "recanvass of vote" in EL § 9-208 does not include a hand count of all paper ballots. A candidate would have to go to court to obtain a recount, probably with little or no evidence of miscounted votes from the audit, unless the audit can be improved to find such evidence. Therefore it is essential that the intent of ERMA's audit requirement, EL § 9-211 (3) and (4), be clarified and that the regulations comply with it.
Part 6210.18 Should be Amended to Include the Following:
1. Full compliance with EL § 9-211, which requires a uniform statewide standard for expanding the audit to a "complete audit" in the event discrepancies are found. The definition of "complete audit" should include all the ballots on which a particular contest appeared, and be used to determine the winner of the contest or the propositions rejected or approved (EL § -211 (4)). A complete audit should not be limited to a particular county, counties, or machine type, but should include all counties and machine types in which a contest appeared on the ballot.
2. Auditing of more, but smaller units, to provide a greater chance of finding problems that could affect outcomes of contests. Whenever possible (such as for percentage-based expansion of the audit, and for any contests not included in the initial 3% audit), audits should be based on EDs, or the portions of EDs (to be defined as "audit units") counted on a particular scanner, rather than whole scanners. The SBoE has already adopted this in their draft procedures for auditing central-count scanners, but only to avoid 100% hand counts of absentee ballots. The use of EDs, or the portions of EDs counted on particular scanners, should apply to all audits, to increase the chances of finding miscounted votes.
3. Escalation sections should apply to entire contests -- not just individual counties or machine types. Therefore, expansion of the audit should be:
- contest-wide; and
- based on EDs on each scanner (which should be defined as "audit units") instead of whole scanners.
4. Larger audits for close elections: A simple formula or table in which the initial number of selection attempts or "picks" is inversely proportional to the victory margin, has been proposed and could be used to allocate auditing resources efficiently for close races in which votes miscounted, regardless of the reason, are most likely to result in an incorrect winner. Such audits should be conducted using the smaller audit units instead of whole scanners to increase their effectiveness and reduce the overall workload for a given confidence level.
5. Close-margin full-recount triggers: The 1% automatic trigger for a full hand count that will be used in the 2009 pilot, should be included in the regulations. A smaller margin (for example 0.35%) could be used for statewide races. All such margins should be based on the contest-wide results and not the results in individual counties.
6. The regulations should require the use of overhead projectors during hand-counts so that all observers can see the ballot being counted. The SBoE should consider inclusion of a requirement for web-casts of hand-counts with close-ups of the ballots being counted.
7. The regulations should require continuous observation of all voted ballots and other election-day materials that are critical to an audit. Election inspectors must be required, and observers allowed, to accompany the ballots and other materials to a central site where meaningful, continuous observation is permitted until completion of all audits. For example, in some counties a cell in the county jail may be considered for use for secure storage of ballots and materials, with observers provided with folding chairs in the aisle outside the cell.
8. To reduce the risk of an incorrect outcome, the first of the two escalation triggers needs to be lowered from the current 0.1% change in any candidate's vote share to a 0.05% reduction in the apparent margin of victory. This could result in fewer ballots to be hand-counted. As an alternative, the error rate requirement of HAVA Section 301 (a)(5) could be used, but only errors that reduce the apparent margin of victory should be considered in the decision of whether to expand the audit. This is important especially if initial samples are not large enough to confirm the outcome of the contest, as is the case with the current draft of the regulations.
9. The regulations should allow each candidate to choose a small number of EDs to be audited in each county in which he/she appears on the ballot, as a check for implausible results.
Founder, E-Voter Education Project
Chair, Task Force on Election Integrity,
Community Church of New York Unitarian Universalist