Justin St. Cyr

Check it out, @Adobe is going to release a C2PA browser extension. It reads unique IDs from images that were embedded as a watermark, and uploads data to third-party servers. This will facilitate a whole new class of online surveillance.

Adobe's C2PA Is a Nightmare for Privacy and Online Freedom The C2PA Content Credentials system developed by @Adobe facilitates surveillance of online content on a massive scale that spans all web platforms. It comes down to two justifications: Copyright and misinformation. C2PA is intended to be used as a form of DRM for controlling the use of copyrighted digital media, including its use in training AI models. They also claim it will help eliminate misinformation (such as deepfake impersonations) by authenticating the provenance of media, including AI generated content. Both of these objectives rely on revealing the identity of the content creator. Aside from the obvious privacy concerns of binding your ID to your content, there is a major flaw in C2PA's design that opens the door for massive surveillance and censorship. Here's the problem: Anyone can strip C2PA metadata from a file and then attach new C2PA metadata that claims it was created by another person and process. For example, someone could strip C2PA from a CNN video and claim they created it themselves using generative AI. It all comes down to identity and trust. If you don't trust the entity that signed the C2PA metadata, then there is no reason to trust anything that it claims. How can you verify if the C2PA metadata is genuine and not swapped with something deceptive? The "solution" to this problem is what facilitates surveillance on a massive scale. Adobe's solution is for someone (not yet identified, to my knowledge) to run third-party servers called C2PA manifest repositories. To check if the C2PA data is genuine, users need to upload data from their local computing device to one of these third-party servers, either a key hidden inside of the media using watermarking, or the entire media file itself. The server will then retrieve whatever C2PA metadata it has associated with that file in its database, and send it to the user's device. If the C2PA data was missing it will be restored, and if the C2PA data was fake then it will be replaced with the "real" data. This has a number of grave implications related to both privacy and censorship. Whoever operates these C2PA servers will be able to monitor who is viewing what, at what time, how often, who it spread to next etc. They would have the power to aggregate this media viewing data across all web platforms and even local computing devices, far surpassing the surveillance capabilities of individual social media platforms. Surveillance data of this magnitude would be incredibly valuable, both to corporations and governments. Another major issue is that whoever runs these C2PA servers would have the power to censor information and spread (actual) disinformation. They would essentially get to dictate what is "real." For example, a malicious C2PA server could refuse to return C2PA metadata for real images and videos that they want to suppress, causing users to mistrust them as potentially AI generated. Even worse, a malicious server could return fraudulent C2PA metadata, making it appear as if real images and videos are actually AI generated when they are not, and vice versa. Who is going to run these servers, and who is going to audit them to look for malicious behavior? Will governments be playing a role in any of this? Will governments have access to the surveillance data? These are questions that no one is asking. The censorship capability is enhanced by the fact that anonymously signed C2PA metadata is worthless: If you don't trust the entity that signed the C2PA manifest, there's no reason to trust anything it says. There will be massive pressure on users to bind their identity to their content in order for it to be taken seriously, and it would be easy for operators of malicious C2PA servers to simply blacklist content based on the identity of the creator. Requiring a creator to reveal their identity to build "trust" in their content will take power away from regular citizens, who may want to remain anonymous in order to protect themselves, and shift it to news corporations. A huge objective of C2PA seems to be to bolster the credibility of major news media corporations; they identify themselves in the C2PA metadata, therefore it's trustworthy and not AI generated. In reality, C2PA metadata is absolutely useless for proving how digital media is created. Anyone can create a C2PA manifest that says an image was created with a digital camera or was AI generated, regardless of the truth. It all comes back to trust and identity: Do you trust the source identified in the C2PA manifest? NO IDENTITY (ANONYMOUS) = NO TRUST Reveal your identity and bind it to your online content, then let C2PA servers run by unknown entities track who views your media and how it spreads online, otherwise no one should trust it. This is Adobe's stated objective and system design.