Poorly anonymized logs reveal NYC cab drivers’ detailed whereabouts

Botched attempt to scrub data reveals driver details for 173 million taxi trips.

City officials released the data in response to a public records request and specifically obscured the drivers' hack license numbers and medallion numbers. Rather than including those numbers in plaintext, the 20 gigabyte file contained one-way cryptographic hashes using the MD5 algorithm. Instead of a record showing medallion number 9Y99 or hack number 5296319

It turns out there's a significant flaw in the approach. Because both the medallion and hack numbers are structured in predictable patterns, it was trivial to run all possible iterations through the same MD5 algorithm and then compare the output to the data contained in the 20GB file. Software developer Vijay Pandurangan did just that, and in less than two hours he had completely de-anonymized all 173 million entries.

Taxi license numbers are always six-digit numbers or seven-digit numbers that begin with a five. That makes for a maximum of two million possible numbers, a sum that takes a matter of seconds to exhaust using programming rules built into cracking apps such as Hashcat. Medallion numbers similarly conform to specific patterns that make for a total of only 22 million possible combinations.

Link

Oops. Some body didn't think. They could have at least added a salt to increase the number of possible combinations.
Why not use a sequential number or random series for each.

And so????

a reply to: roadgravel

The interesting/worrying part is that some taxi drivers never pick up passengers but are just employed to do drug runs. That was the result of one police investigation. Then New York pedestrians would frequently complain that they tried to flag down an empty cab, and the driver wouldn't stop.

Oops! The average script kiddie who has done any password cracking could have explained how that wasn't going to work.

a reply to: stormcell

I imagine then the police have gone through this data for years.

a reply to: theantediluvian

Who do they employ as software people? That had fail written all over it. I wouldn't be surprised if some manager said this was how it was to be done.