I was reading Super Crunchers a little while ago. I got to this passage which is one of those things that are deceptively low-key but then make you go WTF? A big WTF?!! Such a big one in fact that I am quoting it here. At the end of a section about data mashing the author adds the following cautionary tale (pp. 138-139):
Clearly, technically, DBT made a blatant mistake as the author concludes. But how come? Why did the government of Florida give directions that led directly and predictably to the "mistakes"?First of all, why allow any false positives at all? It's perfectly possible to get to almost zero false positives if you tolerate more false negatives, i.e. err on the safe side. In fact, in legal terms, that's the rule: "innocent until proven guilty" -- not 90 percent, but beyond a reasonable doubt. How could they accidentally forget this principle when it came to denying basic rights like voting? Second, in addition to the bias mentioned in the passage, it seems obvious to me that African Americans have a higher frequency of occurence of the same names. So not only did they err on the unsafe side, but the way in which the error expanded happened to be doubly targeted at a particular demographic group -- how come? Oh and who ran the state government of Florida at the time, and who benefited from those errors? Those are rhetorical questions by the way. But it's still surprising. Anyway it's history now...
Speaking of history, the title of this post comes from the name of a disco in Nairobi way back in the day. The first time I ever heard about the concept of a nightclub was when we drove by Florida 2000 one day, and I asked what's that place? I was too young to even think about going in but it was a fascinating thing -- it actually looked like a flying saucer.
And speaking of flying saucers, I wonder what it must be like for kids to not have the "Year 2000" in the future... Which reminds of a song by Fela and Roy Ayers.
Yet the art of indirect matching can also be prone to error. Database Technologies (DBT), a company that was ultimately purchased by ChoicePoint, got in a lot of trouble for indirectly identifying felons before the 2000 Florida elections. The state of Florida hired DBT to create a list of potential people to remove from the list of registered voters. DBT matched the database of registered voters to lists of convicted felons not just from Florida but from every state in the union. The most direct and conservative means to match would have been to use the voter's name and date of birth as necessary identifiers. But DBT, possibly under direction from Florida's Division of Elections, cast a much broader net [...] Its matching algorithm required only a 90 percent match between the name of the registered voter and the name of the convict. In practice this meant that there were lots of false positives [....] For example the Rev. Willie D. Whiting, Jr., a registered voter was initially told that he could not vote because someone named Willie J. Whiting, born two days later, had a felony conviction. The Division of Elections also required DBT to perform "nickname matches" for first names and to match on first and last names regardless of their order -- so that the name Deborah Ann would also match the name Ann Deborah, for example.Note that this book is all about number crunching, not politics, and overall very optimistic, gung-ho even. But this brief passage, specifically the things that I have highlighted in bold above, gave me pause... It's been bothering me for a couple of weeks.
The combination of these low matching requirements together with the broad universe of all state felonies produced a staggeringly large list of 57,746 registered Floridians who were identified as convicted felons. The concern was not just with the likely large number of false positives, but also with the likelihood that a disproportionate number of the so-called purged registrations would be for African-American voters. This is especially true because the algorithm was not relaxed when it came to race. Only registered voters who exactly matched the race of the convict were subject to exclusion from the voting rolls.
[...] What makes the DBT story so troubling is that the convict/voter data seemed so poorly matched relative to the standards of modern-day merging and mashing.
Clearly, technically, DBT made a blatant mistake as the author concludes. But how come? Why did the government of Florida give directions that led directly and predictably to the "mistakes"?First of all, why allow any false positives at all? It's perfectly possible to get to almost zero false positives if you tolerate more false negatives, i.e. err on the safe side. In fact, in legal terms, that's the rule: "innocent until proven guilty" -- not 90 percent, but beyond a reasonable doubt. How could they accidentally forget this principle when it came to denying basic rights like voting? Second, in addition to the bias mentioned in the passage, it seems obvious to me that African Americans have a higher frequency of occurence of the same names. So not only did they err on the unsafe side, but the way in which the error expanded happened to be doubly targeted at a particular demographic group -- how come? Oh and who ran the state government of Florida at the time, and who benefited from those errors? Those are rhetorical questions by the way. But it's still surprising. Anyway it's history now...
Speaking of history, the title of this post comes from the name of a disco in Nairobi way back in the day. The first time I ever heard about the concept of a nightclub was when we drove by Florida 2000 one day, and I asked what's that place? I was too young to even think about going in but it was a fascinating thing -- it actually looked like a flying saucer.And speaking of flying saucers, I wonder what it must be like for kids to not have the "Year 2000" in the future... Which reminds of a song by Fela and Roy Ayers.
great song
ReplyDelete