Friday, December 14, 2007

Post-Election Auditing: A View From The "Summit"


They say it's lonely at the top but here are a couple of videos from the first-ever "Post Election Audit Summit" held near the Twin Cities back on Oct. 25-27, 2007.


So far, accounts of this event have been pretty spotty. There was some "coverage" by a blogger in California who says he wasn't there, and a bit more from folks who were there (or who got pretty close at least). This apparent lack of publicity has fueled speculation that this event was supposed to be some kind of secret.

But before you light your torches and sharpen your pitchforks, let me say that the organizers wanted this event to be as open as possible, despite the limited space that required it to be invitation-only; so they made some videos. I thought it might be a good time to break this unintentional silence before this thing starts taking on the guise of a "Skull & Bones" initiation.

I was honored to be a panelist at this event, which meant I got a whole seven minutes to explain how to audit electronic vote counts without upsetting any election officials. That turned out to be the easy part, although I did feel a little pressed for time.

As I see it, the point of this little shindig was to explain to some of the (mostly friendly) powers that be that:

  1. It's not them we don't trust -- it's the software!
  2. We can deal with this mistrust by not relying too heavily on the software.
  3. We can show them how to do this, to the advantage of all stakeholders by using statistically accurate fair and efficient post-election audits, especially if they're conducted "on the ground" rather than up at the "summit."
Unfortunately, some folks who weren't on the guest list were quite upset about this, and I and some others are also concerned about that. I'm not one of the organizers of this event -- just a grunt panelist -- but I can attest to the fact that there really was limited space (for about 100 people), so it was not possible to invite everyone who could have contributed or benefited.

And let's face it: some folks in the Election Integrity "community" have burned more bridges than Madison County! So not everyone got the call.

One of those who did is my colleague and co-author Arlene Ash from Boston University. She has a PhD in Mathematics, but also a way with words. And although she likes to edit most, if not all, of my work when it comes to post-election auditing (you can even see her do this during my LIVE performance in the second of these videos!), she usually comes up with something very constructive.

So here's Arlene's talk, in which we learn, among other things, that auditing elections is a lot like tasting soup. You can follow along with her slide show as you watch the video:



And here I am trying to cram 20 minutes of an interactive auditing demonstration into about seven! I might have been able to do it had Arlene not interjected in the middle, but I think she actually helped to clarify something that I had neglected to mention in my hectic race against the clock. So it's all good. You can follow along with some of these slides and download this spreadsheet, both of which are shown and described in the video:



To read more about this, click here to download the simple 3-step audit protocol shown in the slides and video. It was included in a looseleaf binder that was given to all "Summit" attendees.

As you can see, I managed to get through all this in about nine minutes. After the talk, one well-known election official asked if we could do an "Auditing for Dummies" version of it. Reluctantly, I had to break the news to him that, actually, this was the "dummies" version! This got some laughs as I had anticipated it would, but then I said something like, "But seriously folks, it really takes 20 or 30 minutes to explain all this properly and I only had seven. I'm sure that everyone here would understand it given another 13 minutes or so." (I was thus able to meet the requirement not to upset any election officials.)

After that came another panel discussion, and then lunch. A lot of folks had soup, so we must have done a pretty good job explaining statistical sampling. If I had a copy of the dinner menu, I'd post it here. But there is a limit to even my documentary skills, so I guess some aspects of the "Summit" may have to remain undisclosed -- just not the election-related stuff.

If you want to take 20-30 minutes to think about how to confirm electoral outcomes (independently of software of course), please watch the above videos, view the slides, read the 3-step protocol and try the spreadsheet. Then you can say that you too have been to the "Summit" -- and it was really quite down-to-earth after all.

Monday, December 10, 2007

Landmark NJ Election Audit Bill Passes Budget Committee

Trenton, New Jersey
Dec. 10, 2007

New Jersey's landmark Post-Election Audit Bill, S.507, passed the Senate Budget and Appropriations Committee today by a party line vote (Democrats for and Republicans against), despite massive opposition from the office of Democratic Attorney General Anne Milgram -- the state's chief election official appointed by Governor Corzine.

The AG sent no less than four attorneys to argue against the bill on the basis of what appeared to be some inflated cost estimates for hand-to-eye counts of voter-verifiable paper records and ballots, and some apparent last-minute misinterpretation of the bill's language regarding the establishment of an independent professional state election audit board to design and oversee the audits.

Ten election officials from 10 counties in the state also voiced their opposition to the bill, although some of these had actually been consulted along with the Attorney General's staff during the bill's drafting process in which several important compromises were reached. Unlike those productive discussions, which did not result in a weakening of the audit provisions, witnesses at today's hearing report that there was a fair amount of antagonism.

No one actually opposed the notion of statistical audits (as required by S.507) per se, but there were some unsupported arguments put forth such as the unpredictability of costs and exaggerated cost estimates of post-election audits.

Costs presented by some counties were as high as several dollars per hand-counted vote, but one county, Mercer, submitted a cost of only 14¢ per hand-counted vote. This latter cost is in line with those submitted by experts and advocates for the bill from around the nation based on data from several other states where election audits and manual recounts have been conducted recently.

The next step for this bill will be a vote on the floor of the full Senate. The bill's sponsor, Senator Nia Gill (D - Montclair), continues her rock solid support for this groundbreaking legislation. She said of today's events, "We have moved one step closer to establishing a process that ensures the integrity of our voting system. Our constitutional right to vote is more than just casting a vote; it is also the right for that vote to be counted. The combination of a voter-verified paper trail and a mandatory audit provides the protection needed when using electronic voting machines. The people deserve nothing less, and I look forward to the bill’s passage before the full senate."

Tuesday, December 4, 2007

NJ S.507 and VVPAR Bills on Voice of the Voters Radio Again 12/5

Please distribute widely:

NJ State Senator Nia Gill will be on VotV Radio again tomorrow night to discuss the NJ statistical audit bill, S.507, which was reported out of committee yesterday, as written and amended by Senator Gill.

Also up for discussion will be the voter-verified paper record bill, which needs to be fixed in the Assembly to match the Senate Version, S.2949. The Assembly version of S.2949 is A.4585.

A.4585 must now be amended to keep the Attorney General from allowing paperless electronic voting with no deadline for the introduction of paper.

The current version of the audit bill, S.507, is now online on the NJ Legislature's website. It should be passed as written as a model for the nation by the end of the month! The assembly version of this bill is A.2730 and it needs to be amended to match the Senate version. The assembly bill is a flat 2% audit. (Not very effective.)

S.507 is the first legislation to require outcomes of elections, as determined by electronic vote counts, to be confirmed independently of software through the use of statistical methods, up to and including full hand-to-eye counts of voter-verifiable paper records and ballots when necessary.

I may or may not be on the show, but I'm sure there will be another guest or two to discuss this bill with the Senator. The segment will start at 8:00 PM ET.

The page for the webcast is:
http://www.voiceofthevoters.org/
There is an archive of last week's show which was excellent. It's Part 2.

For more background, see previous posts here on
Election Integrity: Fact & Friction.

Thanks for all the support you've given to this vital piece of legislation so far! But it's not over 'til it's over.

Monday, December 3, 2007

Landmark NJ Election Audit Bill Out of Committee -- AS WRITTEN!

Just a quick update on S.507 and the VVPAT bill, S.2949:

1. The landmark election audit bill, S.507, passed through the Senate Gov. Committee today as written! The Attorney General's office did not submit their proposed amendments that would have gutted the bill. The vote was 4 to 1 in favor of S.507 as amended by Senator Gill!

2. The paper record deadline in S.2949 was moved to June 2008 but the additional language that would have allowed the Attorney General to delay the voter-verifiable paper record implementation indefinitely was removed. And a commitment was made to hold an oversight hearing on VVPAT implementation in February.

Thanks to everyone who showed up, wrote and called about these bills, and especially Senator Gill whose sponsorship has been rock solid!

The Senator will be asking for continued support as she shepherds the Audit Bill through the legislature. Stay tuned for more on this vitally important issue!

As Gandhi said: "First they ignore you, then they ridicule you, then they fight you, then you win."

Thursday, November 29, 2007

URGENT!!! New Jersey Election Audit Bill Needs OUR Help!

This is urgent folks, and should be done today but we have decided to continue over the weekend since the hearing is at 10 AM on Monday, Dec.3. Note that this a national action and NOT limited to New Jersey!

Contact Gov. Corzine:
Phone: 609-292-6000
http://www.state.nj.us/governor/govmail.html

Please pass this on to other blogs and listservs as you see fit.
It has national significance for the 2008 elections and beyond!
:

Last night, on Voice of Voters Radio, http://www.voiceofthevoters.org/
New Jersey State Senator Nia Gill, the sponsor of the NJ election audit bill, S.507, which many consider to be a model for the nation, asked for calls to be made to Governor Corzine's Office.

Why?

Because the State's Chief Election Official -- the Attorney General -- who is unelected and reports to Governor Corzine, wants to gut this vital bill which most consider to be a model for the nation!

Contact info for Gov. Corzine:
Phone: 609-292-6000
http://www.state.nj.us/governor/govmail.html

It is of the utmost urgency that we do this NOW! The bill will be voted out of committee on Monday, Dec 3.


The details:

The AG wants to CAP ALL STATISTICAL POWER REQUIREMENTS at an AD HOC AUDIT PERCENTAGE of 3%, 5% or 10%, any time these numbers would result in SMALLER, LESS EFFECTIVE AUDITS of Electronic Vote Counts than the PROPER use of statistical power (sometimes known as confidence or significance), as called for in the current version of S.507.

Not only is there no scientific or mathematical basis for doing this, but such arbitrary caps on hand counts would:

- HOBBLE THE INDEPENDENT AUDITORS required by S.507;

- make it unlikely or next to IMPOSSIBLE for audits to confirm the correct winners and losers of many elections (especially closer races where larger audits are needed the most);

- not require the use of statistical audits of any kind
if the AD HOC PERCENTAGES RESULT IN SMALLER samples.

Contact info for Gov. Corzine:
Phone: 609-292-6000
http://www.state.nj.us/governor/govmail.html

The bill currently requires statistical power levels to confirm the outcomes of elections of:
99% for ALL federal and statewide elections and
90% for other smaller elections such as the entire State Legislature
.


This will ensure that enough votes are COUNTED BY HAND to confirm the outcome of any such audited election with a high degree of certainty, completely independent of software.

Contact info for Gov. Corzine:
Phone: 609-292-6000
http://www.state.nj.us/governor/govmail.html

Your message to Governor Corzine should be:

- Tell the Legislature to PASS S.507 As Amended by Senator Gill -- NOT by the Attorney Genral.
- Tell the Attorney General to drop her amendment to cap the audits and hobble the auditors.
- Do NOT impose arbitrary caps on the size of post-election audits.
- Confirm the outcomes of ALL Federal and State elections independently of software.

BUT THIS HAS TO BE DONE TODAY or Friday Morning, Nov. 30, because the hearing to bring this bill out of committee is scheduled for Monday, Dec. 3.

Contact info for Gov. Corzine:
Phone: 609-292-6000
http://www.state.nj.us/governor/govmail.html

Let me know if you make a call and what the governor's office has to say!

Howard Stanislevic
E-Voter Education Project
NY, NY
http://e-voter.blogspot.com/

Monday, August 13, 2007

President Urges Effective Election Audits

That's right. You heard it here first folks. In a July 24th letter to Senator Dianne Feinstein (D-California), the President has come out for the competent and ethical use of statistical methods to assure the integrity of elections. (That's a real audit for those who haven't been paying close attention.)

In referring to Senator Feinstein's bill, S.1487, the letter states respectfully that effective election auditing may not necessarily be achieved by investigating a pre-specified percentage of votes or voting precincts. The President urges the Senator to replace the 2% audit requirement in her bill with “audits of sufficient statistical power to assure 99 percent discovery of a potentially outcome-reversing defect in the vote tabulation.” Imagine that -- The President!

Of course, I'm talking here about the President of the American Statistical Association (AmStat), Mary Ellen Bock, but hey, the Secretary of State has decided to require serious election audits recently too! Of course Debra Bowen is only the Secretary of the State of California. But we gotta start somewhere!

The President's letter can be found here and I would suggest sharing it with other members of Congress as well. They are on vacation now, so they should not be too hard to find back in their home states. And don't forget your State election officials! There is also a broader statement from AmStat about the many ways in which statistical methods can support the goal of highly reliable and verifiable election management. You can read that here, in the President's Corner.

As recently as a year ago, 2% audits were all the rage and with the exception of one State, North Carolina, in which the law says that the size of the sample is chosen to produce a "statistically significant result and shall be chosen after consultation with a statistician", no one in government was even considering anything other than ad hoc fixed percentages. Now, we have a bill in New Jersey, an edict in California, and a letter from the President of something!

It's almost as if some folks actually want to solve the electronic vote-counting problem after all.

Sunday, August 5, 2007

Congress is from Venus; Californians are from Mars*

* Note: Any references in this piece to planetary bodies or their names are intended to be gender-neutral.

I don't know if anyone else has noticed this, but as I read the documents from the top to bottom review and decertification of e-voting systems by California Secretary of State Debra Bowen and her team, I'm stunned by the contrast between these documents and what passes for election integrity legislation in the US Congress. Venus and Mars have more in common than these proposed laws and regulations do; but one thing's clear: not un
like Venus, Congress is full of hot air!

Consider that on Venus (where Congress resides), even the most basic voting system security precautions such as keeping election management systems (EMSs -- the computers that program ALL the voting machines and ballot scanners in a jurisdiction) off the Internet and the public switched telephone network, have not been included in legislation (such as H.R.811), while on Mars, Bowen and her team have mandated the use of no less than three isolated EMS computers to perform ballot definition programming, central tabulation and reformatting of memory cards -- with nothing but "air gaps" between them (and presumably without wireless transceivers that might be used to bridge those gaps).

Unlike the babbling on Venus, the Martian language is crystal clear too. There is no reason to have to ask a court, a Congressional committee, or a team of Martian lawyers and expert witnesses to interpret it months after an election has been held. This is inevitable on Venus though where the language in the Congressional legislation is barely readable -- even by native Venusians.

There's even an extensive DRE (touchscreen voting machine) ban on Mars, and all the Voter-Verified Paper Audit Records created by the banned Martian DREs have to be hand counted! A portion of the other Martian ballots will have to be hand-counted too, using a risk-based approach and an adjustable sample model with a desired confidence level (e.g., 99%) that the winner of each election has been called correctly. Statisticians and auditors on both planets have been advocating this for years!

But on Venus, Congress is saying that no more than 10% of the paper ballots have to be hand counted, no matter how close an election appears to be. Imagine that! The Venusian Congress is supposed to judge the elections of its members, yet it's the Martians who want to be confident that the winner of each election is called correctly!

Finally, on Venus, those who have proposed a compromise of limiting the purchase and deployment of DREs to the numbers necessary to meet Help Venus Vote Act (HVVA) Accessibility requirements for disabled voters, were actually accused of "ghettoizing" such voters! Yet on Mars, this is exactly what Bowen has decided to do to ensure election integrity.

Meanwhile, back on Earth, I've been thinking of changing the name of this blog to "Election Integrity: Science Fiction", but maybe there's still time for Congress to get their Act together on this issue. After all, some of their key members just happen to be from Mars, uh I mean California.

Let's not forget to thank Debra Bowen and her team for all their great work!

Sunday, July 29, 2007

California May Get Audits Right -- So What Took 'em So Long?

Since 1975, when the solution to the electronic vote counting (election verification) problem was first presented by NIST (then National Bureau of Standards) researcher Roy G. Saltman, some (but apparently not enough) advocates have tried in vain to convince election officials, legislators and policy-makers to implement it. California Secretary of State Debra Bowen's Post-Election Audit Standards Working Group (PEASWG) has finally taken the first step.

In their report entitled: "Evaluation of Audit Sampling Models and Options for Strengthening California’s Manual Count", published July 27, 2007, the team has apparently decided that enough is enough when it comes to unverified electoral outcomes in the State of California. Rather than suggesting a continuation of the present fixed-percentage manual tally of just 1% of the State's voter-verified paper audit records, or the sometimes ineffective and often inefficient "tiered" percentage approach which is all but a requirement for federal elections in Rep. Rush Holt's House Bill, H.R.811, the team has decided to get election audits right for a change -- and the change is long overdue.

According to their report: the tiered approach is "ad hoc"; there is no statistical justification for the particular tiers; and it is inefficient in some cases and inadequate in others. In some races, it requires far more auditing than is needed to discover any errors that might affect the outcome; in others, it does not require enough auditing. Can someone please say "Amen!" now?

It's nice to see that someone is finally getting the message that Congressman's Holt's office and many activists chose to ignore during the drafting of H.R.811. That said, I'm sure some members of the working group will be quick to point out that they support H.R.811 anyway because there is no better alternative in Congress at this time, and that H.R.811 allows States to come up with better alternative audit mechanisms. In my opinion, the States would be foolish not to.

Secretary of State Bowen herself has been quoted as saying that statisticians and not politicians should be the ones to decide how to audit elections, and she has so far been true to her word. Instead of just the "usual suspects" (computer scientists and election officials), the working group she put together to develop auditing recommendations also includes a statistician and a CPA.

According to their report, "The Working Group has reached a consensus that the most effective way to conduct post-election audits is to take a risk-based approach. The sampling model that works best for this approach is the adjustable sample model, where the size of the initial random sample depends on a number of factors, including the apparent margin of victory, the number of precincts, the number of ballots cast in each precinct, and a desired confidence level (e.g., 99%) that the winner of the election has been called correctly."

In a nutshell, the group proposes an enhancement to the work of Saltman, Stanislevic and Dopp, which has been published and cited in numerous papers since Saltman's 1975 seminal work, and most recently in Verified Voting's "Percentage-based versus SAFE Vote Tabulation Auditing: A Graphic Comparison", a draft of which was provided to the group at their first public hearing and via email and can be downloaded here: http://www.verifiedvotingfoundation.org/article.php?id=6483

The working group's version, which has yet to be developed, would address the issues of what to do in the event that discrepancies were found in an initial audit sample, and how to deal with contests that span multiple counties. Both of these issues have been addressed to some extent in the Verified Voting report and in the New Jersey election audit bill No. S.507, as amended. The bill requires the use of a relatively small, but apparently non-random .1% vote switch in the initial sample to trigger additional auditing. The bill and the VV report both state that each county should audit its pro rata share of the total precincts to be audited for a race, rounding up to the next whole precinct in each jurisdiction.

More work needs to be done to spell out in detail how this method can be applied in the specific context of California elections, but the California PEASWG are certainly on the right track, and hopefully, we will not have to wait another three decades before the solution first proposed by Saltman will actually be implemented. The only question now is: which State will be first to get election audits right?

Friday, June 29, 2007

H.R.811: Fact and Friction -- Part IV

I'd like to conclude this series by stating what I think is most important to include in federal election integrity legislation, seeing how H.R.811 stacks up against that, and finally by saying whether or not I support H.R.811 anyway. (I'll try not to spoil that for readers until near the end.)

As I and other critics of H.R.811 have been told: "If you don't like it, go write your own bill!" Gee, I thought that's what we were supposed to be doing during the H.R.811 drafting process, but I guess I was wrong.

What some of us seem to have forgotten during the drafting of this bill is that experts have told us in no uncertain terms that it's beyond the state of the art to produce error-free software -- not to mention the potential for fraud and abuse of that software which is almost infinitely scalable. So electoral outcomes need to be confirmed independently of software. I don't know how we got so far off the track of achieving this relatively straightforward goal with H.R.811, but here's how to do it (and there ought to be a law about it):

1. Publicly disclose and audit all Ballot Definition Programming before each election. Follow up with rigorous Logic & Accuracy (L&A) tests.

2. Aggregate precinct totals transparently and independently after posting and witnessing them at the precincts on election night.

3. Audit within-precinct tallies (using paper and hand-to-eye counts) with a statistically accurate, fair and efficient method. (I don't care if it's ballot sampling, precinct sampling or machine sampling as long as it's statistically accurate and audits all types of electronically counted ballots and of course, the dreaded "paper records.")

4. Follow up on any discrepancies found until correct outcomes can be confirmed with very high certainty (prior to certification of course). Ninety-nine percent has been shown to be feasible for all recent federal elections without excessive administrative burden.

5. Have plenty of paper ballots on hand in case of DRE failures (or ban the DREs altogether until someone can get them right)! The 9.2% failure rate allowed by the federal voting system standards makes DREs an unacceptable technology for running elections, especially when other methods are used in other jurisdictions within the same State.

So let's see how the new version of H.R.811 reported out of the Committee on House Administration meets these simple criteria:

1. Disclose and audit all Ballot Definition Programming before each election: Well, that may have been possible in the original version of the bill, but the new section on software (non)disclosure gives ballot definitions the same legal status as vendors' source code. This change has been attributed to lobbyists for Microsoft, but I don't recall ever seeing any of their products produce a Ballot Definition File. Even if they did, this should be no more proprietary than a Word for Windows document or Excel Spreadsheet, and should be viewable with open-source software. While there are requirements for ballot definitions to be disclosed, these files are not required to be a matter of public record, as they should be. Without seeing how our ballots and our election have been defined and configured in the vendors' election management systems (EMS), there is no way to know how the machines (be they DREs or optical scanners) will interpret voters' selections -- even on hand-marked paper ballots. So H.R.811 fails on this count.

2. Aggregate precinct totals transparently and independently: This is the goal of another bill introduced by Rep. Holt in the last (109th) Congress, H.R.6414, which has still not been introduced in this session. Its provisions are not contained in H.R.811 or any other federal legislation but they are pretty good. This is not so much a failure of H.R.811, but perhaps a political miscalculation not to reintroduce this bill.

3. Audit within-precinct tallies (using paper and hand-to-eye counts) with a statistically accurate, fair and efficient method: The best that can be said about H.R.811 with respect to this is that it would allow such methods to be used by the States -- but it certainly does not mandate them. The audits mandated by the bill are neither statistically accurate, nor fair, nor efficient. In case you're wondering about the unfair part, the audits in the bill will provide much greater assurance that statewide races are correctly decided than they would for many US House races, and that makes them unfair to candidates for and members of Rep. Holt's own legislative chamber -- the US House of Representatives. I have already written extensively about this problem and suggestions to remedy it have been mostly ignored, although, as I've said here, H.R.811 does allow States to do better. The State of New Jersey, which happens to be Mr. Holt's State, may be the first to do so if S.507, the amended bill pending in their State Senate, becomes law.

4. Follow up on any discrepancies found until correct outcomes can be confirmed with a high level of certainty (prior to certification of course): It's vague, but H.R.811 does say there should be additional audits if cause is shown. However, the States get to decide what the cause and the additional audits will consist of. There's no assurance that they will get this right. Besides, material discrepancies may never be found with the audits required by the bill, especially for close races and US House races where their statistical power to detect potentially outcome-altering miscounts is low.

5. Have plenty of paper ballots on hand in case of DRE failures: This is perhaps the most disturbing aspect of H.R.811. While there was a clear provision for this to happen for the 2008 election, it has mysteriously disappeared without any public record or amendment by the Committee on House Administration to strike it. Attempts by this author and others to find out what happened have yielded no additional information, but we have been assured that it will be restored before the bill is voted on. That's nice, but what other last minute changes can we expect to see, or not see until it's too late? Of course an outright DRE ban is "off the table."

Now, in case you're wondering, I support H.R.811 despite its flaws, so long as condition #5 above is met without any other undesirable changes. We have been told there is no room for improvement, but if there is a process to restore the missing section, why not work to improve the bill as part of this process, and to counter any other unexpected negative changes? Major improvements may not be possible but clarification of the language that, according to talking points, is supposed to ban Internet connections to voting systems (but is actually still limited only to certain "voting devices"), and clarification of the role of dial-up networking (if any) in voting systems, still needs to be included. Otherwise there is little if any improvement in overall voting system security required by the bill. It bans Internet voting while leaving the core of the voting system, the EMS, vulnerable to tampering via the Internet. If this is no longer the bill's intent, it needs to be made crystal clear.

Reasons why I support H.R.811 (anyway):

1. H.R.811 now requires all audits and recounts of federal elections to be conducted with hand counted paper ballots or records. Although the records may be corrupted by DREs incorrectly recording voter intent, the mandatory warning issued to the voters may be enough to get them to notice. Unfortunately, there is currently no requirement for DREs to be taken out of service, or for a paper ballot to be issued in the event of machine failure, so voters would still in effect be forced to vote on faulty equipment. Hopefully, this will change if the missing provision requiring paper ballots to be issued in the event of machine failure is restored. The bill does not ban DREs but it is not DRE-friendly and I believe it discourages their use.

2. H.R.811 allows States to do better. The bill allows States to get audits right and it pays for them. But activists will have to drop their slavish adherence to Holt's audits, start to do a little math, listen to the real experts (professional auditors and statisticians) instead of just the "usual suspects", and take the necessary steps at the State level to make audits work. There is finally some interest in this at the national level, thanks to the efforts of members of the American Statistical Association, Common Cause, Verified Voting Foundation and others, including this author. But it's also essential to be vigilant and not allow any State to come up with a worse audit plan than H.R.811 requires. The alternative audit language in the bill is still quite vague and gives the National Institute of Standards and Technology (NIST) a bit too much authority without sufficient statutory guidance from Congress. While I expect NIST to do a good job based on their past performance in this area, Congress should be the ones to set minimum standards for their elections without loopholes, because under the Constitution, only Congress has the right to judge the elections of its members.

3. The Senate bill, S.1487, introduced by Dianne Feinstein is a lot worse than H.R.811. I know this isn't the best reason for supporting Holt's bill, but at least, in my opinion, there's nothing in Holt's bill that will make things worse, as long as advocates are vigilant and hold their State and local officials accountable for doing things right.

It's clear that Congress still isn't taking the e-vote-counting problem seriously enough, and probably never will, but there's no reason why the States can't pick up the slack. H.R.811 may motivate some of the "slacker" States to finally do that. Without such a bill, it may not be possible.

You can read the other parts of this series here, here and here.

Wednesday, June 6, 2007

Model for the Nation: New Jersey May Do Audits Right (And Other Needed Reforms)

New Jersey Citizens’ Coalition
on HAVA Implementation


FOR IMMEDIATE RELEASE

June 7, 2007

Contact:
Renée Steinhagen, Coordinator, NJ Citizens’ Coalition on HAVA Implementation, 973-735-0523
Sandra Matsen, League of Women Voters of New Jersey,
908-236-6847

Glenn Magpantay, Asian American Legal Defense & Education Fund (AALDEF), 212-966-5932 x206

COALITION INTRODUCES GROUNDBREAKING ELECTION REFORM LEGISLATION
Bills seek to improve the administration of elections to ensure accuracy, security, inclusiveness and integrity

Newark, NJ – The New Jersey Citizens’ Coalition on the Implementation of HAVA (Help America Vote Act) is introducing six pieces of legislation that it believes are needed to ensure the integrity of New Jersey’s elections. One bill, which has a sponsor, outlines a procedure for the mandatory audit of election results (amending S.507), and is designed to serve as model legislation for election integrity post-HAVA. Its most innovative feature is that the margin of victory in a race would determine the extent of the audit. In closer races, more districts would be scrutinized, and the districts to be audited would be chosen at random. The process is transparent, nonpartisan, and completely independent of software, assuring that outcome-reversing miscounts are detected. The other five bills address inter-county provisional voting, voter assistance in Asian languages, verification procedures for voter registration information, training for poll workers, and requirements for voter registration agencies.

Over the past few years, the Coalition has been working with State and local election officials and legislators to propose and implement election reforms that build upon the requirements of federal law. By treating HAVA as a floor and not a ceiling, New Jersey is poised to become a leader in election administration. With the 2008 Presidential primaries less than a year away, the need to have reforms in place has created a sense of urgency across the state, causing the Coalition to take the lead in drafting the new legislation and forge new partnerships along the way. Its mandatory audit bill was drafted with the assistance of a Ph.D. political scientist, experienced election integrity advocates from two states that have election auditing laws, Ph.D. statisticians from the American Statistical Association, and other voting rights advocates, all of whom worked pro bono.

“We could no longer simply comment on the initiatives of others,” said Renée Steinhagen, Coordinator of the Coalition. “The time had come to change our stance and become proactive rather than reactive.”

“These bills put New Jersey voters first,” said Justin Levitt, Counsel for the Brennan Center for Justice at NYU School of Law. “They take reasonable steps – some long overdue – to protect the integrity of our elections and promote the participation of all eligible voters. We hope that the legislature will give them the speedy passage they deserve.”

“The proper administration of elections is essential to our democracy,” added Ms. Steinhagen. “All the proposed bills go a great distance toward improving the administration of elections by achieving the goals of transparency, accuracy, and voter inclusiveness.”

In addition to the mandatory audit bill, the Coalition is introducing five other pieces of legislation. “Though the mandatory audit bill is apt to garner the most attention due to NJ’s looming voter-verifiable paper record requirement on January 1, 2008, each and every one of these bills is critical for ensuring fair, nondiscriminatory and inclusive elections in our state,” said Lauren Skowronski, Executive Director, Common Cause New Jersey.

One of the bills will increase election participation among Asian American voters who have limited English proficiency. Glenn D. Magpantay, Asian American Legal Defense and Education Fund staff attorney said, “AALDEF has conducted multilingual exit polls in New Jersey for many years and has consistently found that many Asian American voters need language assistance to exercise their right to vote. Last year, during the November 2006 elections, 32% of Asian American voters we surveyed in New Jersey were limited English proficient. Ballots, voting instructions, and interpreters in Asian languages will ensure access to the vote for the Asian American community in New Jersey, which has now surpassed the 600,000 mark.”

Another bill will permit voters who move between counties to vote a provisional ballot if they do not notify election officials of their move in a timely fashion. Sandra Matsen, League of Women Voters of New Jersey, said, “With the statewide voter registration system up and running, it is time for the promise of provisional ballots to become a reality. Individuals who move between counties and fail to re-register should be able to vote in their current place of residency on a provisional ballot.”

Yet another bill imposes uniform training requirements on poll workers and another sets forth best practices for voter registration agencies. “Poll workers are the link between election administration – the laws, rules and procedures governing the election process – and the voters. We have one set of state election laws and we can no longer tolerate 21 different trainings. Statewide uniformity for training poll workers is essential for a fair and effective election system,” added Jo-Anne Chasnow, Policy Director of Project Vote’s Election Administration Program. “Furthermore, New Jersey must comply with the National Voter Registration Act of 1993 in offering voter registration opportunities in a host of state agencies. Compliance is long overdue, and one of these bills will help to ensure that this will happen.”

To date, the package of six proposed bills has been endorsed by: AARP-NJ, American Civil Liberties Union of New Jersey, Asian American Legal Defense & Education Fund, Asian American Political Coalition of NJ, Asian Pacific American Lawyers Association of NJ, Brennan Center for Justice, Coalition for Peace Action, Common Cause New Jersey, Korean American Bar Association of NJ, Korean American Voters’ Council of NY/NJ, League of Women Voters-NJ, Manavi, New Jersey ACORN, NJ Appleseed Public Interest Law Center, New Jersey Citizen Action, NJ Immigration Policy Network, Organization of Chinese Americans-NJ Chapter, Pan American Concerned Citizens Action League, Pan American Friendship Committee, Philippine American Friendship Committee, Project Vote, and South Asian American Leaders for Tomorrow.

For more information, contact Renée Steinhagen, Coordinator of the NJ Citizens’ Coalition on the Implementation of HAVA at: New Jersey Appleseed Public Interest Law Center,
973-735-0523, rsteinhagen@lawsuites.net, www.njappleseed.net.

Summary of the bills and all six bills in their entirety are available online at:
http://www.njappleseed.net/publications.php

The PDF can be downloaded here:
http://www.njappleseed.net/entity_pdfs/175.pdf


# # #

Saturday, April 7, 2007

Amend H.R.811 to Allow States to Use Audits to Confirm ALL Federal Election Outcomes

Update -- H.R.811 is out of committee with some language changes that recognize science and statistical accuracy with respect to alternate audit methods. While this language is somewhat vague, NIST is granted $100,000 to provide guidance to the States. We are seeking signatures for the Senate now. Anyone who was reluctant to try to improve H.R.811 may wish to consider improving the Feinstein bill, S.1487, by signing the letter below which will be changed to refer to that bill.

Below please find a formal letter and proposed language modification for H.R.811 to explicitly allow States to conduct manual audits of electronic vote counts to confirm the outcomes of ALL federal elections with high statistical confidence. Currently this is not provided for in the bill due to some ambiguous language. The new language, written by Larry Norden (Brennan Center for Justice) in consultation with Dr. Ron Rivest (of MIT), Dr. Mark Lindeman (Bard College Political Studies Program) and yours truly will allow all states to do a better job without weakening any existing provisions of the Holt bill.

We have only a few days left until the markup by the House Administration Committee so there is no time to waste.

For those who haven't been following this issue, there is mounting evidence and a consensus among supporters and critics of H.R.811 alike, that the audits proposed in the bill will not confirm all federal electoral outcomes with high statistical confidence.

This doesn't mean we shouldn't support the bill; it means we can and should do better; and it will not be more burdensome or expensive to do so.

If you care about this issue, please email me with your name, title and any affiliation you wish to include, and this letter will be delivered to the House Administration Committee and later to the Senate Rules committee and those working on Senator Feinstein's bill.

If you have an organization, please let me know and I will add your group to the list of signatories. Election officials are welcome to sign.

Also, please forward this to other interested parties before it's too late.


Regards,

Howard Stanislevic
E-Voter Education Project

Here is the letter:

We believe that Congress, the States, election officials, winning and losing candidates and the American People need to be able to confirm the outcomes of all federal elections independently of the software used.

The language of H.R. 811 that appears to allow alternative audit mechanisms must be clarified to ensure that States are enabled to confirm the outcomes of all Federal elections at a statistically high confidence level. Such standards would provide for maximum efficiency by minimizing redundant auditing in uncompetitive races, while allocating adequate resources to the races where they will be needed most.

On Page 12 of his written testimony of March 20, 2007, Lawrence Norden of The Brennan Center for Justice proposed language for H.R. 811 to improve the alternative audit section of the bill that would explicitly authorize States to establish more effective audits using statistical confidence standards and guidance provided by the National Institute of Standards and Technology to confirm the outcomes of all federal elections independently of software. (See: http://www.verifiedvotingfoundation.org/downloads/Testimony_1.pdf)

A similar measure has already been adopted in the State of North Carolina for low cost and a modest amount of time. Pamela Smith, President of Verified Voting, referred to North Carolina's audits in her testimony before the Committee on House Administration’s Elections Subcommittee on March 15, 2007. (See:
http://www.verifiedvotingfoundation.org/downloads/PamelaSmithTestimonyFinal_2007mar20.pdf)

We therefore propose that the following language suggested by Mr. Norden replace Section. 322 (b) of the Help America Vote Act, as proposed in H.R.811:

Current H.R.811 language (also in S.559 and S.804 with a different section number):

Section 322 --

“’(b) Use of Alternative Mechanism- Notwithstanding subsection (a), a
State may adopt and apply an alternative mechanism to determine the
number of voter-verified paper ballots which will be subject to the hand
counts required under this subtitle with respect to an election, so long
as the National Institute of Standards and Technology determines that
the alternative mechanism will be at least as effective in ensuring the
accuracy of the election results and as transparent as the procedure
under subsection (a).”

Proposed language:

Section 322 –

Use of Alternate Mechanism – Notwithstanding subsection (a), a State may adopt and apply an alternative mechanism to determine the number of voter-verified paper ballots that will be subject to the hand counts required under this subtitle with respect to an election for Federal office, so long as the National Institute of Standards and Technology determines that the alternative mechanism is as transparent as the procedure under subsection (a) and is consistent with the guidelines set forth in Section X.

Section X -- GUIDANCE ON BEST PRACTICES FOR ALTERNATIVE AUDIT MECHANISM. Not later than May 1, 2008, the National Institute for Standards and Technology shall establish guidance for States to establish alternative audit mechanisms. Such guidance shall be based upon scientifically reasonable assumptions for the purpose of creating an alternative audit mechanism that will

“(a) require the hand-count of at least 2% of all precincts (or other audited units) within each Congressional District, and ensure, with at least [90/95/99]% statistical confidence, for each federal election held in the State, that a 100% manual recount would not alter the outcome of the election; or

“(b) be at least as effective as section 322(a) in ensuring that for each federal election held in the state, a 100% manual recount would not alter the outcome of the election.

AUTHORIZATION OF APPROPRIATIONS – There are authorized to be appropriated to the National Institute of Standards and Technology $100,000 to establish the guidance required by this section.

We the undersigned election integrity organizations and advocates support the above changes as written:

Organizations

Citizens for Election Integrity Minnesota

Coalition for Voting Integrity, Pennsylvania

Commonweal Institute, Katherine Forrest, MD, President and Co-Founder, Menlo Park, CA

Florida Fair Elections Coalition

Georgians for Verified Voting

Las Vegas (New Mexico) Peace & Justice Center

New Jersey Appleseed Public Interest Law Center, Renee Steinhagen, Exec. Dir.

North Carolina Coalition for Verified Voting, Joyce McCloy, Founder

United Voters of New Mexico

VotersUnite.Org, John Gideon and Ellen Theisen, Co-Directors


Individuals

Thomas L. Tedeschi, Esq., Election Attorney, NY, NY

John L. McCarthy, Ph.D., Computer Scientist, Berkeley, California

Mark Lindeman, Ph.D., Bard College Political Studies Program, NY

Arlene Ash, Ph.D., Statistician, Fellow of the American Statistical Association, Boston, MA

Mary Batcher, Ph.D., Statistician, Fellow of the American Statistical Association, Silver Spring, MD

David Marker, Ph.D., Statistician, Fellow of the American Statistical Association, Columbia, MD

John S. Gardenier, Research and statistical ethicist, Vienna, VA

Vittorio Addona, PhD, Statistician, Macalester College, MN

Howard Stanislevic, Research Consultant, E-Voter Education Project, NY, NY

Jerry Lobdil, Physicist, Election Integrity Researcher, Fort Worth, Tarrant County, TX

Lisa Young, I.T. Risk Management Consultant, Tampa, FL

Paul Stokes, Electrical Engineer, Corrales, NM

Pamela de Maigret, Documentary Film Maker and Writer, Los Angeles, CA

Jerry Adams, Ph.D., Oregon

Jerry Depew, Laurens, IA, http://iowavoters.org

Donna Mummery, Election Inspector, Monroe County, NY

Henry Greenspan, Ph.D., Justice in Michigan, Ann Arbor, MI

Judy Bertelsen, M.D., Ph.D., Berkeley, CA

Teresa Hommel, wheresthepaper.org, NY, NY

Wednesday, March 28, 2007

H.R.811: Fact & Friction – Part III

By Mark Lindeman, Ph.D. and Howard Stanislevic, Research Consultant

In Part I and Part II of this series, we pointed out that the post-election audits provided for in H.R.811 are too small to achieve a high level of statistical confidence that close races for US House seats will be correctly decided. While this observation is actually in agreement with a letter being circulated by Rep. Holt’s office signed by nine prominent experts, we also pointed out that this letter actually overstates the confidence level of Holt’s audits by not taking variations in precinct size into account. The same letter also stated:

“In truth, it may be that attempting to prevent an ‘unacceptable’ level of error on electronic voting machines through audits is too administratively burdensome.”


Put that way, frankly, audits seem pretty useless: if attempting to prevent unacceptable error is just too much of a burden, then what is the point? But on a close reading, the intended point here is subtler and more useful: while it would be ideal to confirm that every vote count is accurate within a fraction of a percentage point, it is much easier to confirm that at least every outcome is correct. Unfortunately, the letter stops short of recommending an audit protocol that actually confirms outcomes. We think that election audits should not only indicate the general accuracy of the counts, but provide a solid basis for believing that the winners really won. Getting the winner wrong is the “unacceptable” error, in most people’s minds – and we believe that it can be prevented without undue administrative burden.

In a nutshell, we believe – and will show below – that the basic problem with the H.R.811 audit is simply the misallocation of resources. By making smarter choices, the country can achieve high confidence in the outcomes of almost all federal races with about the same amount of count-auditing effort. (We say “almost” because in extremely close races, there can always be grounds for controversy about what votes should be counted. Verifying the substantial accuracy of the count isn’t the only task – but it is a very important one.)

Looking too hard in all the wrong places

First, we decided to examine all federal elections in the last three cycles (2002 through 2006) – the presidential race, elections for all 100 Senate seats, and almost 1300 contested House races (about 575,000,000 total votes cast) – to explore the consequences of H.R.811’s quirky allocation of audit resources. For each race, we estimated the size in hand-counted votes of an H.R.811 audit, and the probability that this audit would detect hypothetical outcome-altering vote miscount, using the precinct size adjustment and the other assumptions we made in Part II of this series. We chose to measure the audit size in votes, instead of the number of precincts, because precincts come in many different shapes and sizes. Even using Holt’s methodology, some precincts will require only one race on their ballots to be audited, while others may require audits of all three federal contests. The cost of doing the audit is therefore not necessarily proportional to the number of precincts audited, but is much more closely related to the number of accurately hand-counted votes.

We then figured the size of conducting what we call a 95% SecureAudit in each race: an audit large enough to achieve 95% confidence of detecting outcome-altering miscount under the same assumptions as we have used in evaluating the H.R.811 audits. The closer the race, the larger the necessary audit size. (Some folks call this idea a “probability-based audit,” as in, “based on yielding at least an ‘X’% probability of detecting outcome-altering miscount.” We don’t care much what the audits are called, as long as the idea is clear.) We did the same calculations for a 99% SecureAudit.

(The fine print: these estimates are crude because we do not use actual precinct-level data, but they do consider important variables such as the total number of precincts in each state and the average number of precincts per Congressional District based on the statewide totals. We don’t think that the estimates are biased for or against any particular audit approach. To detect widespread systemic corruption, for all these audits we assume that at least one precinct per county is randomly audited, as required under H.R.811.)

Over these elections, a 95% SecureAudit would support much greater confidence in election outcomes than H.R.811 audits, at somewhat lower cost. Across all three election cycles, H.R.811 audits would mandate manually auditing about 20.3 million votes (about 3.5% of the total vote in these elections).

Here is a table showing the number of races with outcomes that would NOT be confirmed with the H.R.811 audit for various confidence levels:


Unconfirmed Outcomes w/H.R.811 Audit
Confidence Level# of Races
99%238
95%162
90%135
50%49

In 135 of the races, the estimated probability of detecting outcome-altering miscount would be less than 90%. In 49 races, the probability would be less than 50%. These low-confidence races include not only the very closest races, but some in which the winning margin is over 4%. They even include at least one race (NH-CD1) in which the winning margin is over 7%, but the number of precincts – and, therefore, the audit size – is unusually small. (We’re not saying that these races are likely to be decisively miscounted. We are saying that if one of them were decisively miscounted, an H.R.811 audit very well might miss the problem.)

By comparison, for an estimated total SecureAudit size of about 18.7 million ballots, over 7% less than in H.R.811 audits, we could have attained 95% confidence of detecting outcome-altering miscount for all of these races, as shown in the following table:







SUMMARY OF AUDITED VOTES
RacesHR811 95% Conf.
SecureAudit
PRES '044,426,2933,681,772
SENATE '021,835,4162,063,447
HOUSE '022,521,6262,556,846
SENATE '042,964,9541,691,639
HOUSE '043,661,6572,925,976
SENATE '062,098,3751,725,986
HOUSE '062,774,2094,103,394
TOTAL20,282,53018,749,060


Thus, high confidence levels can be achieved not by mandating “burdensome” audits across the board, but by shifting resources used in the H.R.811 audit from races and precincts where they aren’t needed to confirm the outcome, to those where they are.

Furthermore, we could attain 99% confidence for every audit in every race by auditing a total of about 22.7 million ballots, about 12% more than under H.R.811:







SUMMARY OF AUDITED VOTES
RacesHR81199% Conf.
SecureAudit
PRES '044,426,2934,449,975
SENATE '021,835,4162,457,639
HOUSE '022,521,6263,142,558
SENATE '042,964,9541,933,563
HOUSE '043,661,6573,507,100
SENATE '062,098,3752,146,380
HOUSE '062,774,2095,100,798
TOTAL
20,282,53022,738,013


If counting those additional votes to get to 99% confidence seems a bit excessive, consider that the above numbers do not include any of various optimizations that could be used to achieve higher confidence levels if the States were permitted to do so. We will say more about this issue below.

Why are 95% or 99% SecureAudits about the same total size as the hit-or-miss H.R.811 audits? Because, as we’ve said, the H.R.811 audit throws audit resources into races where they aren’t especially needed. Consider the 2006 California Senate race, which Dianne Feinstein won by about 24 points over Republican challenger Dick Mountjoy. Assuming for a moment that Mountjoy actually won this election, there would have to be 20% miscount favoring Feinstein in almost half of California’s 21,000+ precincts. In principle, a truly random sample of just 10 precincts would let us detect such massive miscount with about 99.8% confidence. Our SecureAudits use 58 precincts because California has 58 counties, so they achieve better than 99.9999999999999% detection probabilities. California law mandates a 1% audit (about 210 precincts) – let’s just say that would be a lot of 9s! But H.R.811 says, in effect, that even a 2% audit is too lax – we “need” a 3% audit – presumably to bolster public confidence. This mandate would hamstring California and other states by misallocating resources that should be used to confirm other races in need of stringent audits.

For instance, H.R.811 would settle for a 3% audit in small, competitive races such as in California’s 4th district. (We have no reason to think that this race was actually miscounted; our question is what sort of audit would be needed to justify confidence that it wasn’t.) The race in CA-CD4 wasn’t even all that close, by H.R.811 standards: the winning margin was 3.2 points, qualifying only for a 3% audit. However, a 3% audit in a California congressional race amounts to about 12 precincts – an audit size that would confer only about 45% confidence of detecting outcome-altering miscount. In Connecticut’s 4th district, which had a similar margin, a 3% audit would have counted only five precincts – yielding about 21% confidence. So, in the interest of public confidence, should the federal government pay to audit hundreds more precincts in an uncompetitive statewide race, or should it put some of that money into the small and close races? Which is a better use of money? Which is a better use of election officials’ time? And which is more administratively burdensome?

Notice that in 2006, the SecureAudits end up being larger overall than the H.R.811 audits. That is a good thing: it means that more races were competitive in 2006, and SecureAudits would have done a much better job of confirming the results. We don’t advocate auditing on the cheap; we advocate auditing intelligently in order to achieve high confidence across the country.

Choices, we need choices…

Another way to increase the confidence level (without increasing cost) is to randomly audit smaller units such as individual DREs or optical scanners instead of entire precincts. This approach can be especially helpful in states with fewer, larger precincts such as New Hampshire. For example, if there were two scanners per precinct and 200 precincts, randomly auditing 16 scanners instead of 8 precincts in a race with a 10% margin would achieve 99% confidence as opposed to only the 90% confidence achieved by selecting whole precincts. In each case, about 4% of the votes would be counted but the larger number of smaller audit units results in a higher confidence level. (Of course no particular ballot type should be excluded from an audit, and this is another area where H.R.811 could stand some improvement. It’s very hard to understand how the bill deals with absentee ballots!)

A purely random audit has some limitations. It doesn’t “care” whether a precinct is large or small, whether the apparent winner did surprisingly well or poorly. Doing away with random audits would be a terrible mistake: properly done, they provide an excellent check on the overall accuracy of the system. However, to detect possible concentrated miscount in particular races, it may be useful to add a “challenge” element to the audit. Losing campaigns could draw up lists of precincts whose results they deem most suspicious; immediately after the random sample is selected, campaigns could choose additional precincts to be audited. If some attacker did attempt to switch 20% of the votes in 5% of the precincts, probably any savvy analyst would choose at least one of those precincts. So the attacker may choose to steal fewer votes in each of more precincts, but that would increase the attack’s visibility to the random part of the audit. In simulations, we have found that adding even as few as 5 or 10 “challenge” precincts often does as much to improve the detection as quadrupling the random audit size! For instance, in Connecticut-CD4 in 2006, we estimate that auditing about 10% of precincts – evenly divided between random and challenge precincts – could yield the same 99% confidence as randomly auditing about 48% of precincts. Take that result with plenty of salt, but still, it’s interesting. It also appears that auditing larger precincts more heavily than smaller precincts could be substantially more efficient than treating them all equally.

There are other suggestions about how to improve upon the H.R.811 audits, and we won’t try to do justice to all of them here. Suffice it to say, we see no reason to settle for H.R.811’s 3%/5%/10% audits. Unfortunately, not only does H.R.811 not implement such suggestions, but it isn’t entirely clear whether H.R.811 would even allow individual states to implement them. The bill does contain language that allows the National Institute of Standards and Technology (NIST) to approve alternative audit methods that are “at least as effective.” Obviously we think that a SecureAudit is much more effective than the H.R.811 audit overall – but we can imagine the lawsuits over whether it is ever acceptable to audit less than 3% of precincts in any federal race, no matter how large or how uncompetitive. Let us be clear that we support vote-count audits even in uncompetitive races, and we don’t have a fixed position on what the minimum audits should be. But we certainly don’t think that large minimums in some races should be allowed to discourage efforts to confirm the outcomes of other races. Above all, we don’t think that Congress should endorse the premise that audits that often yield less than 50% confidence – sometimes less than 20% confidence! – of detecting outcome-altering miscount are good enough. Instead of crossing our fingers that activists around the country will be allowed to fix the audits state by state, wouldn’t it be smart to do better at the federal level right now?

Click here to learn how to tell the House Administration Committee to allow the states to conduct high-assurance, probability-based audits like the SecureAudits proposed above.

Tuesday, March 20, 2007

So 2008 is history -- ALREADY? I don't think so, and here's why:

Some of the biggest proponents of voter-verified paper ballots, records and trails are sounding pretty gloomy lately about the chances of implementing them in time for the 2008 elections. Fortunately, there are other ways to improve election integrity.

On Dec. 7, 2006, just before the end of the 109th Congress, H.R.6414, Rush Holt's Vote Tabulation Audit Act of 2006 was introduced with no cosponsors and little fanfare. But unlike some of the provisions of this year's H.R.811, the "other" Holt bill seemed to make a lot of sense.

It was drafted last year using a much more collaborative process than H.R.811, and as a result, some of the major mistakes in the bill (such as allowing printouts of cast vote records from DRE memory to be used to correct central tabulator totals -- yikes!) were actually corrected before the bill was introduced. And unlike H.R.811, the provisions of the other Holt bill can be implemented in time for the 2008 elections, or even in time for a practice run in 2007!

The bill as written would rule out all errors in the vote count except within-precinct errors, and although these are the most dangerous and difficult to detect, they are also more difficult to generate than just switching vote tallies around in a Microsoft Access or other commercial off the shelf database program on a central tabulator PC. They say even a chimpanzee can hack an election that way, and even Hand Counted Paper Ballot tallies can be altered at the central tabulator, more properly referred to as the Election Management System (EMS).

So why not put the other Holt bill on the "fast track" for 2007 now? And while you're at it, get those EMSs off the Internet so they can't be hacked so easily in the first place, Mr. Holt!

There might still be a few bugs in this bill, but at least the major ones were fixed last year and it will get elections officials in every precinct in the nation used to the idea of auditing something. In fact, this bill requires a 100% audit of precinct totals.

Also, some of the better parts of H.R.811 could be included in the 2007 version of the other Holt bill. For example, how about disclosure of all ballot definition programming? And how about an audit of all ballot definition programming before and after every election? It would only take two trained auditors per county to do that -- one from each of the major parties, and of course those from other parties on the ballot should be welcome too!

For those who may not be informed, ballot definition programming error is an unequivocally documented source of vote miscount and is almost never examined or audited by anyone other than the few election insiders who are allowed access to it. It's an open invitation to deliberate malfeasance. But unlike e-voting source code, it's not even a "trade secret" because it's the definition and configuration of the election itself -- not the so-called "intellectual property" of the voting machine companies. Besides, a properly written law could force its disclosure anyway. So what are we waiting for?

I think a Holt Ballot Definition and Tabulation Audit Act of 2007 would be a huge step in the right direction that could be implemented now. And it would not require any voting equipment to be replaced next year -- not even lever machines!

"Don't let the perfect become the enemy of the good!" That's what proponents of H.R.811 keep saying. But haste makes waste, a stitch in time saves nine, and an apple a day...well, you get the drift. Besides, the reforms in this "other" Holt bill are needed regardless of whether or not H.R.811 passes, so we might as well get with the program!

For more information on how this Holt bill could improve election integrity almost immediately, please see: Section VIII: Transparent Aggregation of Voting Results Using the Internet by Juan Jover in the DNC's report, "Democracy At Risk: The 2004 Election in Ohio", as well as the links to VotersUnite's excellent documentation of ballot programming errors above. And of course, the famous Bev Harris/Howard Dean central tabulator hack (as seen on TV).

And then call Holt's office!

Saturday, March 10, 2007

H.R.811: Fact & Friction -- Part II

Guest blogged by Mark Lindeman, Ph.D.*

In Part I of this series, Howard Stanislevic pointed out that the audits mandated by H.R.811 (as written) would be far too small to confirm the outcomes of some close elections – while being far larger than necessary to confirm the outcomes in other races.

You might wonder why it could possibly be a good idea to allow inadequate audits in some races while mandating needlessly large audits in others. (Even if you think there is no such thing as a “too large” audit, the misallocation of resources ought to trouble you.) I’m not sure, although I think Rep. Holt may be using an informal political calculation:

Last time I introduced a bill, people complained that a 2% audit sounded too small, so let’s require a minimum 3% this time – that’s fifty percent more audit! And let’s make the audits even larger in close races – but let’s not go over 10% no matter what, because election officials will get mad if they expect endless recounts.


And so, the Holt bill mandates 3% audits (i.e., 3% of precincts or equivalents) in most federal races, 5% audits in races with a winning margin under 2%, and 10% if the winning margin is under 1%.

Let me be clear: I like Rush Holt, and I don’t fault him and his people for trying to figure out how to pass a bill. And I think the 3% to 10% audits are not only “better than nothing,” but much, much better than nothing. Still, as compromises go, this one is strange. Not only do some people feel it is too soft, and some people feel it is too tough – but the numbers say that it is too soft and too tough. That’s interesting, but not really in a good way. I think we can do better.

First, let me say a word about why H.R.811 is much, much better than nothing. Some people suspect that the 2004 and/or 2006 elections witnessed vote miscount on a massive scale nationwide, on the order of several percentage points or perhaps more. There are about 180,000 precincts or equivalent units around the country, so H.R.811 would entail audits in at least 5400 precincts (3%) in every federal election. Audits that large – if truly random and immune from tampering – should detect vote miscount if it occurred in as few as one out of a thousand precincts nationwide, with 99.6% confidence.

Even a minimum audit of just one precinct per county selected at random (also required by H.R.811) would hand count the votes in about 3000 precincts, or 1.7% nationally. This audit would still detect vote miscount that occurred in as few as one out of 657 precincts with 99% confidence. If one out of every five presidential votes in each of these corrupt precincts were miscounted by the voting system so as to favor one candidate, that would only be enough to alter the popular vote margin by about 0.1%. And, similarly, a “clean” audit of this size would verify the total vote in Senate and House races to about the same tolerance. Not too shabby.

Unfortunately, verifying the approximate accuracy of the overall totals is not the same as confirming election outcomes. As we were all reminded in 2000, it actually doesn’t matter who wins the popular vote! It is very useful to have an upper bound on the overall extent of miscount, but that in itself would not confirm the outcome of the presidential election, nor any other election. Concentrated miscount could still alter the outcome in particular races – and, possibly, determine the presidency or the balance of power in Congress. Not so great.

The Norden letter: accentuating the positive

Holt’s office, sensibly, wants to portray its proposal in the best possible light. It has directed people to a letter, signed by nine prominent experts who consulted on H.R.811’s audit provisions, which seems to express support for the bill. (I will call this the “Norden letter” only because Lawrence Norden of the Brennan Center for Justice is the first signatory.) But as Howard pointed out, the Norden letter also points to some of the problems with the plan. In fact, it understates them.

The Norden letter embraces the general concept of a “tiered” audit – basically, the idea that audits should be larger in closer elections. For instance, on page 2, it argues that in an “imagined typical” congressional district, in order to have 90% confidence of detecting a miscount that would overcome a 1% margin, it would be necessary to audit 10% of the votes. The letter continues:

Mandating a 10% audit for all races would be a high burden on many States. And in the vast majority of races, a shift of 1% of the votes would not alter the outcome of the race.

In such races (says the letter), we might be “willing to live with the risk” of not detecting the 1% counting error – while in the small fraction of races in which a 1% shift would alter the outcome, we might not be. So far, so good. Certainly I think that an audit proposal should focus on delivering high confidence in election outcomes.

The trouble is, even by the Norden letter’s calculations, H.R.811 clearly does not deliver high confidence in the outcome of every race. The letter poses the question of how likely audits under H.R.811 are to detect errors that would change the outcome of particular races. Still assuming an “imagined typical” congressional district of “400 precincts of roughly equal size,” the letter presents the following probabilities of detecting miscounts just large enough to overcome various margins, for various audit sizes. (These calculations also assume that 20% of votes per precinct could be switched from one candidate to the other, without being too blatantly obvious.) The results for the audits mandated by H.R.811 appear underlined in bold.

Norden memo’s estimated probabilities of detecting outcome-altering miscount
[Edited to correct typographical, formatting and rounding errors]

# of pre-
cincts

Margin of victory

Probability in a 2% audit

Probability in a 3% audit

Probability in a 5% audit

Probability in a 10% audit

400

0.50%

10%

14%

23%

41%

400

0.75%

15%

22%

34%

57%

400

1.00%

18%

27%

40%

66%

400

1.75%

31%

43%

61%

86%

400

2.00%

34%

46%

65%

88%

400

5.00%

66%

80%

94%

99.6%


The letter rightly points out that the H.R.811 audits yield much higher probabilities than an across-the-board 2% audit would. That said, the letter carefully avoids speculating about how many people would be “willing to live with” as low as a 40% probability of detecting possible outcome-altering miscount. (If you don’t think that’s bad, keep reading.)

Note well that for margins smaller than 0.5%, the probability of detecting outcome-altering miscount will get smaller… and smaller… and smaller. Presently, some states provide for optional or mandatory recounts for races with very small winning margins, while others do not. People may disagree about how far Congress should go in mandating large audits for very close races. But let’s not kid ourselves that the issue will never arise. In 2006, the race in Connecticut’s 2nd district was decided by under 200 votes, about 0.1%. A 10% audit in such an election might confirm that the election was pretty close, but it certainly can’t confirm who won.

Even if we set aside very close races, the confidence problem is actually worse than indicated by the Norden letter. Unfortunately, the letter’s reference to “imagined typical” districts is only too accurate. In real life, congressional districts don’t have “precincts of roughly equal size.” That wouldn’t matter if we could expect that any vote miscount would be randomly scattered across precincts regardless of their size. But what if an attacker were able to target the largest precincts? Then fewer precincts would have to be miscounted in order to reverse the election outcome, and it would be even harder for a random audit to detect the fraud.

To give an idea of how much precinct size can matter, I examined vote counts in Ohio’s 18 congressional districts in 2004, and chose to use the 5th district as a baseline. Among Ohio’s congressional districts, the 5th had the greatest precinct size disparities – but the disparities are even greater in (for instance) New Hampshire’s congressional districts. So, OH-05 is a bad case, but definitely not the worst. I used the OH-05 size distribution to revise the Norden letter’s assumptions. For instance, the letter assumes that in order to overcome a 1% margin of victory, votes would have to be miscounted in 2.5% of precincts. However, in OH-05, the largest 1.1% of precincts contain 2.5% of the votes. Therefore, my size-adjusted analysis assumes that one could overcome a 1% margin of precincts by miscounting 1.1% of precincts, instead of 2.5% of precincts. (With rounding, that works out to miscount in five precincts instead of ten.) I make a similar adjustment based on the margin in each race – generally, the smaller the margin, the larger the adjustment. Adjusting for size, what happens to the probabilities of detecting outcome-altering miscount? They go down, a lot.

Size-adjusted estimated probabilities of detecting outcome-altering miscount

# of pre-
cincts

Margin of victory

Probability in a 2% audit

Probability in a 3% audit

Probability in a 5% audit

Probability in a 10% audit

400

0.50%

4%

6%

10%

19%

400

0.75%

8%

12%

19%

34%

400

1.00%

10%

14%

23%

41%

400

1.75%

17%

24%

37%

62%

400

2.00%

18%

27%

40%

66%

400

5.00%

46%

60%

79%

95.8%

If you think 40% sounded iffy, how do you feel about 23%, or 19%, or even less in a closer race?

Granted, the previous example is moving toward a worst case – but it isn’t there yet, especially since the Norden letter’s assumptions about number of precincts are also somewhat optimistic. Although it is true that congressional districts average about 400 precincts apiece, sixteen states average fewer than 300 precincts (or equivalent units) per House district. New Hampshire has a total of under 250 units, divided between two House districts, and these units vary wildly in size. In New Hampshire’s 1st district, just three units (out of 114) comprised almost 22,000 House votes in 2006 – over 11% of the total. In non-technical terms, New Hampshire is where percentage-based audits go to die. One nice feature of H.R.811 is that it would allow jurisdictions to adopt stronger audits; I hope that New Hampshire would!

It's one thing to complain that someone else's proposal isn't good enough, but the real question is: can the country feasibly do better? Howard and I are convinced that it can. Since H.R.811 audits too few ballots in some races and more than enough in others, it should be possible to attain more confidence in election results at little or no additional cost. To test our reasoning, we decided to examine all federal elections in the last three cycles (2002 through 2006) -- the presidential race, elections for all 100 Senate seats, and almost 1300 House races. We looked at the consequences of H.R.811's quirky allocation of audit resources, and we explored some alternatives. We will present the results of this analysis in Part III of this series.

* Mark Lindeman teaches Political Studies at Bard College in New York.