C2PA - The peril of bad metaphors

I read a couple of blog posts about C2PA, both an organization steered by big names like Microsoft and Adobe and the nascent technology they promote which is a standard for putting digital signatures on media files. C2PA and prominent supporters market the signatures as both providing provenance of the files, i.e. who created and/or published them, and information as to how they were created. They liken the signatures to a nutrition label for digital files

This list of ingredients will show verified information as key context so people can be sure of what they’re looking at… including whether or not generative AI was used, as well as any edits that were made along the way

I fell into a pretty deep hole and so I feel like I have to write about it to justify the time investment. This is pretty closely aligned to a lot of the topics I’ve worked on in my time as an engineering leader building security infrastructure but I’m far from an expert in C2PA or the degree to which they’re already working on the gaps I’m going to discuss. I’m not here to slag anyone for not already being done with their big project, merely to offer an outsider’s thoughts and to summarize for others who know even less than me.

The bottom line is that I’m positive on the idea of media signatures as one component of a multi-layered fight against disinformation actors (which I’m going to call liars because it’s short and pithy). However the current implementation falls short of the marketing promise in ways that seem like they would make these “nutrition labels” ineffective or worse if they were to become widely adopted right now. Further, the fundamental gaps seem like the kind that you need to solve before wide adoption because the wide adoption would make them really hard to address later.

Nutrition labels are regulated

One thing that is necessary for nutrition labels to be successful in changing consumer behavior is that you can assume that what they say is true. C2PA doesn’t provide that assurance. All that C2PA guarantees you is that if a file has a signature, that the individual who holds the signing key is responsible for the claims in the signature. I.e. you can believe what’s in the nutrition label as long as the brand is trustworthy.

Security researcher Dr Neal Krawetz has written extensively about flaws in the C2PA protocol and will provide you with a much deeper dive than I’m giving you here. He documents two good examples of C2PA signatures that lie to you about how they were created. In the first he uses a C2PA enabled Microsoft service to create a valid signature claiming that the photo was created by a French ambassador on an image that was AI generated. No assertion in the signature says anything about the AI generation. In the second the BBC signed a video with assertions describing their validation of the video as genuine. However, the assertion was in free-form text and didn’t clearly indicate how the validity was assessed or what edits had been made by the BBC. An analysis after the fact showed that additional audio had been added which is not clear in the signature. There’s no evidence the BBC did anything wrong; the point made here is that the signature tells you a lot less than it might seem.

The issue is in what mechanisms exist to make the assertions in a signature likely to be true. The C2PA documentation allows implementors, for example, to create assertions that are backed by hardware based mechanisms. For example in 2023 Leica released a camera that signs the images it takes. Because they have designed the hardware such that the owner of the camera is unable to make the camera sign an image it didn’t take or to steal the signing key and sign images themselves, a consumer can be confident that an image signed by one of these cameras is in fact an image that was taken by the camera.

In the case of the BBC we see that there is no such mechanism to tell us that the BBC actually did anything to validate the video as genuine or specifically what edits their technical team made. Similarly Microsoft Designer doesn’t do anything to validate that the user is actually an ambassador to France. The C2PA spec and guidance allow for assertions to be strongly validated in the manner that Leica cameras do but don’t require them to be and so as you can see the quality of assertions varies.

Unregulated nutrition labels don’t scale

That’s a problem because in the current state C2PA signatures are mainly useful if the reader is sophisticated and well informed, reads the signature carefully, knows the signer, understands how they generate signatures, and can thereby determine how much to trust the signature. Maybe that’s good for evidence used in court (Krawetz says not) or a newspaper working to protect its supply chain. But it won’t scale to the average information consumer. That level of analysis is too much to expect. The two examples from Krawetz demonstrate that it’s not even enough to ask them to only trust info they get from trusted sources because firstly the average consumer could already be doing that but they don’t and secondly are the BBC and Microsoft not to be considered trustworthy sources? Disinformation spreads because the average consumer doesn’t apply a lot of discrimination to what they’re consuming. I am extremely skeptical that we can address that by giving them more information that they have to be discriminating about.

Proponents are encouraging us to think of these signatures as nutrition labels. That makes the technology seem more mature and ready for wide adoption than it is. It also tells consumers that they can read them like they read nutrition labels. You don’t have to ask yourself whether the nutrition facts on a nutrition label are true or not, or whether the absence of a trans fat line might just mean they didn’t report the huge amount of trans fats that are in the product. You can safely assume that the label is formatted the same way as all other nutrition labels and that the numbers on the label are true and make a decision about whether to eat what’s in the box based on what the label says. A consumer who sees a C2PA signature in that frame of mind might be more likely to believe the authenticity of a disinformation image than they would have been without the label thereby making the situation worse than if it had no signature at all.

What I’d do differently

I of course have the luxury of not actually having to deal with the hard tradeoffs that the C2PA folk do. The last thing I want to convey is that all of this is easy but that having been said:

C2PA folks might look at all this and say that I’m making too much of the nutrition label. That after all is just marketing. When you look at the actual documentation it’s extremely clear that C2PA is just about making the assertions available and ensuring the integrity of them. It’s not C2PA’s job to ensure the assertions are true and drive impactful changes in consumer behavior.

But I want to note a few conclusions. First off, the marketing influences how we talk and think about the product which influences how the product is presented to consumers which matters because it’s dangerous if that drives the wrong behaviors.

Secondly, a nutrition label for digital media would be an actual impactful thing in my opinion. I suspect that’s why the metaphor is being used; that’s what we want to be building. But while the C2PA folks say in the technical docs that the quality of the assertions isn’t their job it needs to be someone’s job if we’re going to actually build that scalable solution. The position taken by the technical documentation is defensible as long as you’re clear across the board that you don’t consider it your job to actually deliver a full end-to-end solution that will scale. But by using that nutrition label metaphor they make it sound like they do think it’s their job. If no one takes that job then we don’t get a real nutrition label with all the downsides I’ve already described.

Further I don’t think you do that job without changes to the spec. Some of the things that come to mind are that you’d want to put limits around the assertions that can be made and that you’d want to require certain validations for assertions rather than making them optional. Outside of the spec I think you’d want to have some organization be responsible for ensuring compliance to the requirements and have some way to revoke signing certificates if an issue with a signer is identified. Wide adoption of C2PA in its current form makes these issues a lot harder to solve later because of all the implementations that would have to be changed to accommodate a revised specification.

Thirdly I’m not totally convinced that the nutrition label is actually practicably buildable or that we’d like the limitations we’d have to accept to get it built. Consider image processing software. We’d want any C2PA enabled software to validate signatures on images given to it to modify. Then to sign any saved files with assertions as to what type of edits were made, including the use of any genAI. And it needs to do that in such a way that the owner of the hardware running the software can’t maliciously meddle with the signing process to create fraudulent signatures. That all requires really complex solutions like trusted execution environments which drastically increases the complexity of getting this stuff deployed, and may shut out open source software and software produced by small developers. This totally worthwhile article I found by Martin Thomson makes the same point and a bunch of other good ones you should read. In this way, C2PA has the potential to entrench powerful interests because others can’t afford to produce signatures or refuse to take a dependency on one who can. To their credit, C2PA lists this in their list of potential harms. In any case, if it’s not practically buildable or not buildable without unacceptable societal harm then I think it’s rather unhelpful to build something much lesser and imply that it’s the same thing.

But that’s not to say that the lesser thing wouldn’t be useful. I actually think C2PA signatures as described in the technical docs have the potential to be valuable due to the additional value they provide to analysis processes that could assess the likeliness that a given image is fraudulent. I may write another post on that topic. It’s just that if the lesser thing is all we can practically build then we shouldn’t be pushing to put the signatures in front of users because as I said that has the potential to make things worse.

So altogether I’m positive about the effort but yes the marketing bugs me. The way forward for C2PA IMO is either to change the marketing or do the necessary things to make it true before pushing for wider adoption.