Review: Your Face Belongs to Us, by Kashmir Hill

Big thoughts, and reading between the lines

Kashmir Hill is one of the best reporters going on tech issues, and this book is no exception. She’s very good, and very balanced, though at one point she uses the word “colonizer” instead of the standard English “colonist.”

I’ve known bits and pieces of this story for some time, and the book puts them together in a way that makes sense. I’ve been talking to Charles a bit in the last few days about some of the details and places I have questions. He was also the key source for this book, so if anyone objects to my conversations with him about all this, take it up with the New York Times.

By way of introduction, this is the book-length story of Clearview AI, the controversial facial recognition company that took the market by storm, mostly by way of law enforcement, because the big guys at Facebook and Google were wary of legal and societal backlash. It appears to be pretty indisputable that their algorithm is the best, relying on the largest amount of training data and possessing the largest database of photos. Chapters about its development, from the 2016 Republican National Convention to the center of a global privacy dispute, are interspersed with others about the development of machine learning and facial recognition.

I don’t have much to say about the latter, except two individuals briefly discussed in them. One is Marvin Minsky, whose mother was a Zionist activist. It’s obvious to everyone at this point that he was quite wrong about neural networks, which are now being used in all the major AI projects, which raises the question of whether his pooh-poohing of the idea intentionally crippled the field’s development in some way, a question worth asking given his ties to the late Jeffrey Epstein.

The second is Matthew Turk. It is rumored that his departure from UC Santa Barbara was related to his involvement with the Chinese. His early career at Martin Marietta, then the MIT Media Lab is covered in detail. He pops up later as an expert witness for Facebook in the Illinois Supreme Court, in a case Facebook settled, to say their facial recognition is OK. Shortly thereafter he travels to China. I’m intrigued by this anecdote from 1991, about his eigenfaces paper that became seminal:

After he finished, a man in the audience with a big, bushy white beard stood up with a question. Turk recognized him immediately. His mathematics textbook was used in engineering graduate courses.

If he had a white beard in 1991 he’s probably dead today, but I’m curious who this is. Hill suggests it’s this paper that caused researchers to try a new approach: “The spark that computers needed to go from seeing a collection of facial features to seeing an individual was a lot of faces, a bit more freedom in determining how to analyze them, and more computing power.”

Now, for where Clearview fits in to our moment in technological history. The company, and the litigation surrounding it, exposes some of the inadequacies of traditional civil libertarian organizations and left-leaning privacy advocates in this strange new world we’re living in. After the Illinois Supreme Court moves to block searches, the company’s response is to filter out photos with metadata showing that location there. That neither deletes the photos, nor necessarily reflects which photos were actually taken there. At one point the ACLU argues, “Clearview cannot use nonconsensually captured faceprints,” which, though well intentioned, seems unlikely to carry the day. The right verb is probably “generated,” rather than “captured.”

Many of the feared dystopias that bother them so much should probably be thought of as already here.

Here are five areas where I think this is the case:

• We should probably assume that the Cambridge Analytica-level data gathering on individuals is already being done by hostile states.

• The surveillance dystopia that concerns privacy advocates will be more or less achieved absent the government through the application of machine learning to open-source information. The anecdote in the book about the Miami Police Department watching everyone’s Snapchats during a rave festival is a good instance of what’s likely to just be further refined.

• The Washington Post’s reporting from yesterday on Chinese genetics collection is a demonstration that they’re already moving there.

• Fear of a government social credit score strikes me as somewhat misplaced for a couple of reasons: 1) something like that is already here in the form of credit bureaus, which already dock a person’s score for a variety of personal things, and 2) those credit bureaus have been hacked, which when combined with the first item on this list raises the prospect of overlapping social credit systems.

• The hope of crypto advocates to escape financial surveillance is probably also misplaced, as technology to track crypto transactions appears to be further along than anyone really understood a couple of years ago, not to mention the other issues with crypto.

For all these reasons, the cyber-anarchist sensibility of the oughts and 2010s strikes me as a little outdated, at best. The realization is dawning on the world that we’ve had a little too much openness, thank you, and we’re not quite sure what to do with it all.

Toward the end of the book, Hill rather optimistically suggests, after talking about where this is all headed, that “We still have time to decide whether we want to build it,” meaning the digital panopticon. I think it’s probably more realistic to think it’s too late. The question is whether we want our government, hostile governments, or corporations to control it, and how. Where Hill is right is that this is all an outgrowth of the ideology of openness and free access underpinning Silicon Valley, which has begun to shade in a much darker direction. At this point, given how Chinese the investment banking structure is, or was, on the West Coast, it’s worth at least considering the possibility that much of Silicon Valley was simply a means to collect, on the behalf of foreign powers, information on Americans.

There are also interesting instances in here of technology initially introduced as a civil libertarian measure, can become the opposite, like linking police bodycams to a central signals processing unit, also discussed in the book in Miami.

While a lot of this stuff has troubling implications, I’m not entirely persuaded by much of the regulatory framework, and certainly not the rhetoric, of privacy advocates in Europe and Australia. It strikes me as the sanctimony of those who have the luxury of free-riding on the American security state. There’s also the purely pragmatic question of whether it will do to assert a de jure right to privacy, when privacy does not exist de facto.

I struggle to understand, for instance, the principle by which civil libertarians or privacy advocates in Australia would want Julian Assange freed, and also have a problem with Clearview. I guess the belief is that facial recognition is simply inimical to freedom, as one U.S. advocate for banning it in the book believes. The trouble is that probably won’t stop others from using it here, or on us from elsewhere. What Clearview requires essentially nothing more than large numbers of photos to replicate. Hill writes about one Swiss data journalist who did so, and was told not to.

There isn’t really any practicable means to ban this technology, and maybe not even a sound legal reasoning to do so. It’s essentially just an open source algorithm plus photos. So when privacy advocates or civil libertarians have attempted to take countermeasures, they end up with these funny double standards. An activist in Portland created an algorithm to identify police officers, showing “the same people opposed to facial recognition by the police approved of its use on them, to hold them accountable.” There’s also a strange anecdote I had never heard, in which German pro-tech people vandalized the houses of people who had their homes blurred out on Google Maps, leaving notes in their mailboxes that said “Google’s Cool,” which is deeply disturbing.

Moreover, if the concern is that ubiquitous surveillance cameras with facial recognition make it easy to track people anywhere (though mostly in cities), I would note that the FBI appears to already have the means to do that, through the iPhone, with the NSO Group Landmark tool, which they had a secret contract for. It’s not clear to me why Clearview is a better technology for, for instance, COVID contact-tracing, than the geolocation tools already used by various state agencies.

The thing that I would like to impress upon the progressive elites I grew up with is that there’s a symbiotic relationship between the surveillance state and rioting. As I would hope they have begun to appreciate, when you force the local police to retreat, the demand for law and order remains, which gets filled through surveillance technology and the FBI. I would argue that’s worse in some ways, especially now that there’s a major corruption scandal at the top of the Bureau. There’s a very real sense in which the BLM style of politics cashes out as robocop fascism. That worries me quite a lot. Avoiding dystopia may depend on people’s willingness to extend a little sympathy to their local beat cop. I’m serious.

I also think a lot of lefty pieties, the tendency to go yikes, you can’t say that, probably need to be rethought for the utterly practical reason of preserving civil society, but not quite in the way that a typical conservative would think. When you’re looking at genetic technology, or facial recognition—raw genetic determinism, and the harsh judgment of an algorithm—certain older ways of thinking, right-coded though they are, are actually your more liberal friend. Parlor-trick physiognomy isn’t as harsh as facial recognition. Focused on dead white males though it may be, a dynastic history is considerably more liberal and humanistic than a genetic history.

Most of the reporting on Clearview has succumbed to the temptation that it’s something different from the rest of Silicon Valley, usually by focusing on the politics of its founders. That suits the tech giants just fine, but it’s not true. The company didn’t develop any new technology—Hoan Ton-That coded and pulled it together, but all the actual research was there, open source. And the concept, steal the photos, let them sue us, and by the time the world catches up it’ll be too late, is basically how most VCs think. The phrase Hill uses at one point is “ethical arbitrage,” which is nicely put.

It also seems to be the case that other options are worse. There are two instances in the book, one in London and the other in Detroit, of the combination of Hikvision cameras and NEC (Japanese) algorithms making false positive IDs of black people. Those algorithms are almost certainly worse than Clearview’s. But with that combo, I would also be somewhat worried about the intentional placement of false positives, the Chinese government, of course, having an interest in increasing racial tensions in the West.

That’s most of the big-picture issues. Now let’s go through the actual story of the company with a fine-toothed comb. Charles was the one who put Richard Schwartz and Hoan Ton-That together. The latter the genius coder, the former, the Rudy Giuliani associate who had the powerful connections. The initial meeting took place at a Manhattan Institute event, and the first big chunk of cash was kicked in by Peter Thiel. The author of the legal memo designed to reassure local police departments they could use it without breaking the law was Paul Clement, George W. Bush’s solicitor general. Schwartz involved him.

This is a very good example of how, to do Trumpy things, you had to use pre-Trump networks, like this Manhattan one, which is very Russian and Jewish. I suspect, given Charles’s run-ins with the neocons, during for instance the Menendez story (who has now been charged), he was aware of some of the problems here. His co-founders were, in Ton-That’s case, not, and in Schwartz’s case, part of it. Ton-That rented from WeWork, better understood as an Israeli real estate play than a tech company, and was reportedly obsessed with the relationship between Adam Neumann and Jared Kushner. Ton-That was very interested in getting the big foreign cash from places like Sequoia and SoftBank.

So, of course, given the biases of the other two founders and seeking to ingratiate themselves with the Jewish financiers who underwrite much of conservative politics, they fucked Chuck and cut him out of the company. Richard Schwartz, neocon shyster that he is, has a history of this sort of thing. Both Ton-That and Johnson had what is euphemistically called a “Gawker history,” a company which received a large investment from a Russian oligarch in 2016, so I’m sure that was an excuse frequently raised. It’s probably not a coincidence that Gawker loses its big suit against Thiel two months after Vekselberg makes his investment.

In the prologue of the book, there is an incident at the beginning of Hill’s reporting, in which a Texas investigator runs her own photo as a test search, whereupon he is called by an employee of Clearview, Marko. That’s Marko Jukic, now of Bismarck Analysis, known as Mark Yuray in the neoreactionary blogging world. So right away this should disabuse you from thinking a neocon like Schwartz has any sort of principled ideological reason not to associate himself with Charles Johnson.

The first chapter reveals that Ton-That moved from Australia to the United States at the bidding of Naval Ravikant, a well-known Silicon Valley personage, who has a number of China issues, including Chinese investment in his Indian Twitter clone (Chindia?), and is deeply involved in crypto. That’s worth a follow-up.

Let’s move on to Chapter 3, which has some details about Schwartz’s past. He was a big deal in civic privatization. Hill notes he was the editorial page editor of the New York Daily News during 9/11, and then at some point big Bob Menendez donor Mort Zuckerman got tired of him. There is an allegation in the late oughts, that Schwartz tried to cut his business partner Ken Frydman out of a deal, so a second time he appears to have done something like this—there’s a pattern.

Chapter 5: One of the first applications of what became Clearview—it was still called Smartcheckr at this point—was at the DeploraBall in 2017. There’s a lot of fake history about this event, which is at some point worth documenting further. The version in Jack Murphy’s book seems rather hilariously self-serving. I am told Jeff Giesea encouraged them at this point to get involved with former FBI Director Louis Freeh, who has a number of issues previously discussed on this blog. Also in this chapter is the pitch to the Hungarian government. The Hungarian parliament passed an authorization to license Pegasus from NSO group in October 2017, so they were probably already working with the Israelis by this point.

Chapter 7: This is where Terence Z Lieu comes in. The book says he received his BA in China at Nanjing University, and met Hoan Ton-That in New York City. Liu becomes a sort-of-employed, sort-of-not freelance figure in the company, described as its “algorithm savant.” It is believed by someone I respect that Clearview is simply a clone of SenseTime, the Chinese facial recognition app, and if that’s true, this might be the nexus there. But that would render almost all the other detail about the company’s development false. For now we’ll continue to describe it. At any rate, Liu seems to have a second connection to Hal Lambert, who becomes an early Clearview investor, having worked on the MAGA ETF. Lambert shows up later in Chapter 11. In this chapter, Johnson is cut out of the company. Also, at the end of the chapter, there is a brief description of Ton-That’s descent into the murky world of data brokers, having “about a dozen contractors from all over the world” to “hunt faces on the internet for him.” I am told some of those introductions were made by Joseph Malchow, who has been covered on this blog here, for his role in the Princeton network, and a number of them seem to have been from China. There’s also a line about Ton-That being naturalized in late 2017, for which there may be an interesting story, maybe not.

Chapter 9: This is a very strange chapter, which gives new light to some of the weird alt-right intrigues of the Trump years. Smartcheckr was briefly transformed into a political consultancy. Schwartz connects with Holly Lynch, who wants to run against Jerry Nadler in New York 10. She says it’s important to have women in Congress now that Trump is in the White House. Lynch agrees to pay him a $5000 a month retainer, and in return Schwartz promises, among other things, “extreme opposition research,” aided by Smartcheckr. We are introduced at a meeting with Lynch to Douglass Mackey, also known as Ricky Vaughn. A few things are worth noting about this: Johnson was not actively taking part in the company when Mackey begins his involvement, and Mackey here is assisting the campaign of a female Democratic candidate. When one of her friends tips Lynch off to how weird this is, she folds her campaign. I’m left wondering if that was sort of the point, whether this was a tar-baby thing (I’m sorry if you’re offended by the metaphor, it’s just too useful here), to leave the seat in the hands of a strong pro-Israel vote like Nadler. Next the Smartcheckr team goes with Paul Nehlen. None of this works, of course, so they get out of politics, Johnson finds out they fucked him, he yells at Ton-That, and they agree to give him 10 percent of the company on the condition of his silence. Schwartz is looking, very strongly to me at this point, like some kind of Israeli ratfuck artist.

Chapters 10 and 14 have a lot to do with the companies Facebook worked with on this technology, and their role in generating all this face data, as well as some competitors. I won’t bore you with it, but it’s a big part of the story.

In Chapter 11 the company gets its first real investment, from David Scalzo. At this point, in 2018, they appear not to have been considering selling to law enforcement. Who knows if that’s actually true. At this point trial versions of the app start going out, which get into the hands of John Catsimatidis. Apparently, in something out of Walter Winchell, he used it to spy on a man on a date with his daughter. I find these two some of the funniest characters in the hilarious clown show that is the New York Republican Party. Ashton Kutcher gets his hands on it at some point, which raises a number of questions given his ties to Scientology, and the fact that it was initially pitched to his nonprofit Thorn, which is having a huge scandal right now.

In Chapter 13 they meet the NYPD and do a trial. People who are more familiar with the intrigues of that police force could probably provide a better map than I, so I’ll leave that alone. It’s only worth noting, I think, that they’re continuing the scraping, presumably from a few Chinese contractors, as they’re starting to get law enforcement contracts, building out a database orders of magnitude larger than the FBI’s. Their first real contract was in Indiana. I note that Banjo, too, had a trial run going in Indiana. I wonder, given how Israeli some of these companies appear to be, how much of this has to do with Pence. In 2019 they get a contract with DHS.

In Chapter 15, Hill gets the call in late 2019 from Freddy Martinez, now at POGO, revealing what Clearview has been up to, and the legal and publicity shit storm starts to brew. There’s not much I can add here.

At some point there’s a hack here, and internal Clearview records are given to Buzzfeed, a heavily Israeli website, and Charles Johnson claims he contacted the FBI. The last few chapters are a fine discussion of many of the ethical issues here, which Hill seems to know quite well that privacy advocates are likely to lose.

In a coda that will make nobody happy, Clearview was used on J6 to identify MAGA people. It started at the 2016 Republican National Convention, and in 2020 was used to catch MAGA people at the Capitol. There is a real discussion to be had about the uses of this, and what the government isn’t telling us about what they know about these people and groups.

Anyway, a final few thoughts about some of the intrigues here. I am told Ton-That and Hal Lambert both didn’t want to talk to Johnathan Buma. Apparently Eric Garland, in a post today, was also a source for Buma. That is very interesting.

Lastly, why was Bronze Age Pervert and his Likud network going so hard at Charles? We know a few of them are tied into the world of pro-Israel high finance, and a number of players in Giuliani’s milieu here have links to China, Russia, and organized crime. This whole idea that they can be a subterranean, almost Foucaultean right-of-the-margins seems at odds with the development of all this technology here, and it seems likely what they’re hiding is just that they’re a bunch of typical Zionist Republicans, playing at Nietzschean piratical dreams. Paul Singer is himself, of course, a pirate. The Naval custom when it comes to pirates is to hang them by the neck until they’re dead.