Detecting "deepfakes," or when an existing image or video of a person is manipulated and replaced with someone else's likeness, presents a massive cybersecurity challenge: What could happen when deepfakes are created with malicious intent?
Artificial intelligence experts from Michigan State University and Facebook partnered on a new reverse-engineering research method to detect and attribute deepfakes, which gives researchers and practitioners tools to better investigate incidents of coordinated disinformation using deepfakes as well as open new directions for future research.
Technological advancements make it nearly impossible to tell whether an image of a person that appears on social media platforms is actually a real human. The MSU-Facebook detection method is the first to go beyond standard-model classification methods.
"Our method will facilitate deepfake detection and tracing in real-world settings where the deepfake image itself is often the only information detectors have to work with," said Xiaoming Liu, MSU Foundation Professor of computer science. "It's important to go beyond current methods of image attribution because a deepfake could be created using a generative model that the current detector has not seen during its training."
The team's novel framework uses fingerprint estimation to predict network architecture and loss functions of an unknown generative model given a single generated image.
The new method, explained in the paper, "Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images," was developed by Liu and MSU College of Engineering doctoral candidate Vishal Asnani, and Facebook AI researchers Xi Yin and Tal Hassner.
Solving the problem of proliferating deepfakes requires going beyond current methods — which focus on distinguishing a real image versus deepfake image that was generated by a model seen during training — to understand how to extend image attribution beyond the limited set of models present in training.
Reverse engineering, while not a new concept in machine learning, is a different way of approaching the problem of deepfakes. Prior work on reverse engineering relies on preexisting knowledge, which limits its effectiveness in real-world cases.
"Our reverse engineering method relies on uncovering the unique patterns behind the AI model used to generate a single deepfake image," said Facebook's Hassner.
"With model parsing, we can estimate properties of the generative models used to create each deepfake, and even associate multiple deepfakes to the model that possibly produced them. This provides information about each deepfake, even ones where no prior information existed."
To test their new approach, the MSU researchers put together a fake image dataset with 100,000 synthetic images generated from 100 publicly available generative models. The research team mimicked real-world scenarios by performing cross-validation to train and evaluate the models on different splits of datasets.
A more in-depth explanation of their model can be seen on Facebook's blog.
The results showed that the MSU-Facebook approach performs substantially better than the random baselines of previous detection models.
"The main idea is to estimate the fingerprint for each image and use it for model parsing," Liu said. "Our framework can not only perform model parsing, but also extend to deepfake detection and image attribution."
In addition to Facebook AI, this study by MSU is based upon work partially supported by the Defense Advanced Research Projects Agency under Agreement No. HR00112090131.
Story by Caroline Brooks, courtesy of MSUToday.
This MSU-Facebook research is generating national attention:
ANNOUNCEMENT TWEETS:
- Facebook AI: Our researchers have partnered with @michiganstateu to develop a method of detecting and attributing deepfakes that relies on reverse engineering from a single AI-generated image to the generative model used to produce it. Learn more: http://ow.ly/15u450FbcAU
- Mike Schroepfer: This is great work: an AI system that doesn't just detect deepfakes, it figures out the method used to create them. This helps us identify when multiple manipulated images or videos come from a single source — great for spotting coordinated behavior
- Mike Schroepfer: This is a key responsibility that comes with building cutting-edge AI: creating systems that can counter misuse of the technology, and releasing them for everyone to use. The dataset, code, and trained models here are all being open sourced
NOTABLE TWEETS:
- NECN: AI researchers at Facebook and Michigan State University say they have developed software that can reveal where so-called deepfakes have come from.
- Kyle Wiggers, VentureBeat: Facebook's AI reverse-engineers models used to generate deepfakes
- Mikael Thalen, Daily Dot: Facebook develops new method to reverse-engineer deepfakes and track their source
- Sabena Siddiqui, AI Monitor: Facebook can now reverse-engineer #deepfakes and track their source
OUTLET TWEETS:
- The Verge: Facebook develops new method to reverse-engineer deepfakes and track their source
- CNBC: Facebook scientists say they can now tell where deepfakes have come from
- CNET: Facebook steps up efforts to study deepfakes
- VentureBeat: Facebook's AI reverse-engineers models used to generate deepfakes
ARTICLES:
- Facebook Scientists Say They Can Now Tell Where Deepfakes Have Come From, CNBC, Sam Shead
- Reposted by NECN
- Facebook steps up efforts to study deepfakes, CNET, Queenie Wong
- Reposted by National Cybersecurity News
- Facebook's latest AI doesn't just detect deep fakes, it knows where they came from, Engadget, Andrew Tarantola
- Facebook says it's made a big leap forward in detecting deepfakes, Fortune, Jeremy Kahn (full text below)
- Facebook and Michigan State University Work on Deepfake Detection, The Mac Observer, Andrew Orr
- MSU, Facebook develop research model to fight deepfakes, MSU Today, Caroline Brooks
- Facebook AI aims to identify deepfake images and trace their creators, New Scientist, Chris Stokel-Walker
- Facebook reverse-engineers deepfakes to detect the AI models used to create them, SiliconAngle, Mike Wheatley
- Facebook's AI reverse-engineers models used to generate deepfakes, VentureBeat, Kyle Wiggers
- Facebook develops new method to reverse-engineer deepfakes and track their source, The Verge, James Vincent
- Facebook, Michigan State Develop Deepfake Detection Technique, Wall Street Journal, John McCormick
- MSU team develops deepfake detection model with Facebook, Detroit News, Alex Harring
- MSU working with Facebook to stop deepfakes, WILX