Pa. starts testing new election auditing procedures

The risk-limiting audit refers to the risk that the audit would fail to detect a problem.

  • Emily Previti

(Mercer) – Pennsylvania’s elections overhaul isn’t limited to deploying new voting machines and making sweeping changes to absentee voting and registration deadlines.

Officials also are working on new post-election auditing procedures that employ statistical modeling. Test runs occurred earlier this week in Mercer County and are scheduled for Thursday in Philadelphia.

Post-election audits already happen in Pennsylvania. State law requires counties to audit 2 percent of ballots cast – or 2,000, whichever is less – in each race. 

Other auditing criteria – such as sample ballot selection – are largely left up to county election officials.

That’s expected to change in 2022. The state agreed to implement a more robust post-election audit system — called risk-limiting audits — as part of the settlement of a lawsuit brought by 2016 Green Party presidential candidate Jill Stein. 

“The process that’s in place now is practically meaningless,” Stein’s spokesman Dave Schwab wrote in an email Tuesday. “In contrast, risk-limiting audits are designed to use the paper records to ensure that the machine count didn’t produce the wrong winner.”

The paper records Schwab referred to are part of the same legal settlement that includes RLAs. The settlement requires counties to have voting machines in place for the 2020 presidential primary that produce a voter-verifiable paper audit trail (in other words, a paper record of each ballot that each voter can review before casting, and which can be audited after an election).

For the test in Mercer County, about 30 election experts – academics, consultants and county, state and federal election officials – spent Monday testing risk-limiting audit procedures in the county courthouse.

The risk-limiting audit – where “risk” refers to the risk that the audit would fail to detect a problem – began with generating a number known as a random seed. 

The seed is, basically, used to ensure ballots are truly sampled at random. It’s relatively unpredictable compared to other variables — such as the total number of ballots cast and the winning margins in various contests — that are plugged into the algorithm that dictates which ballots are sampled. 

 

Each of the random seed’s 20 digits was determined by a different person from the crowd rolling a 10-sided die (with their order selected by more dice rolling).

The algorithm spit out a list of numbers.

Each number corresponded to the target sample ballot’s position in the stack of those cast in a given precinct.

Auditors located the target ballots by counting, using a calibrated scale or a method known as k-cut that involves repeatedly cutting the stack to get to the desired sample ballot.

After a few hours, the sample ballots had been pulled. Then, results from those ballots were tallied and run through another algorithm. The formula estimated the risk that this audit would fail to detect a problem was less than one-thousandth of one percent.

The test run used a sample of 80 ballots to audit an election where nearly 24,000 were cast – far fewer than what the law requires now.

But the method has the backing of groups such as the American Statistical Association.

Liz Howard, senior counsel for the Democracy Program at the Brennan Center for Justice at the NYU School of Law, compared running RLAs to flavoring broth.

“When you are making broth, and you want to determine whether or not there’s enough salt in it, you stir up the broth,” Howard said. “You don’t need to take an entire cup of your broth. You don’t need to take half of your batch to determine if there is enough salt in it. You just need to make sure that it’s sufficiently stirred up and the teaspoon works, whether or not your broth batch is just a cup or a huge gallon or 20 gallons.”

The “sufficient stirring” and functional spoon, she noted, are analogous to the ballot sampling determined by the algorithm.

There are multiple ways to conduct an RLA

Officials expect to test other methods during future simulations.

“This is all part of the learning process so that we can kind of build those protocols, put together the procedures,” said Jonathan Marks, deputy secretary for Elections and Commissions at the Pennsylvania Department of State. “So that when the time comes, we can say, ‘This is exactly how it’s going to occur in Pennsylvania.’”

When the RLA guidelines are completed, Pennsylvania counties may no longer be required to audit every single race. Instead, state elections officials will tell counties which contests to audit and how to do it.

But details aren’t yet fully known (county officials, for example, have even provided mixed opinions about whether the state’s election law will need to change).

The advisory panel making recommendations on new auditing procedures will take into account from the pilot audits that began this week. 

Consultants stressed the importance of transparency — publishing code and algorithmic inputs online, for example, or live-streaming audits — no matter what other procedural steps look like, in the end.

Editor’s note: This post has been updated to add a caption and correct the number of votes cast in Mercer’s election. 

Up Next
Politics & Policy

After years of pressure from advocates, state House advances gift ban.