*/
How to navigate open source evidence in an era of deepfakes. By Professor Yvonne McDermott Rees and Professor Alexa Koenig
In 2021, a story of a mother in Pennsylvania, arrested for harassment after allegedly creating ‘deepfake’ footage of her daughter’s cheerleading rivals, quickly went viral. The police officer in charge of the investigation declared, ‘This tech is now available to anyone with a smartphone – your neighbour, somebody who holds a grudge… All one needs to do is download an app and you’re off to the races.’ Yet, it later turned out that the incriminating videos were, in fact, real.
This story is the perfect example of ‘the liar’s dividend’, which legal scholars have long been warning about. The idea is that, as the ability to create convincing fake footage increases, so too does the scope for denying the veracity of real footage – exactly the script adopted by the cheerleading rivals, who didn’t want to be linked to the behaviour captured in the videos. As our previous research has demonstrated, given the very real value of digital information in proving atrocity crimes and mass human rights violations, this is concerning – especially given the growing popularity of digital open source investigations, which rely on information that any member of the public can access online through observation, purchase or request, such as videos or photos posted to social media. There is a risk that lawyers will self-censor, excluding important and relevant imagery, and that judges may unwittingly rely on manipulated evidence, or discredit real evidence as ‘potentially fake’.
This is why we co-authored, as part of a group of experts in open source investigations, Evaluating digital open source imagery (‘the Guide’) – a guide that seeks to demystify digital open source information for judges and other fact-finders. The Guide also provides insights for lawyers on the unique facets of this kind of evidence, including how to ensure that it is what it purports to be.
Image manipulation is nothing new. Raquel Vazquez-Llorente of WITNESS has written about the fraud trial, in 1869, of William Mumler. Mumler’s ‘spirit photography’, which promised to capture customers’ deceased loved ones in photographs, was made possible through a basic double exposure technique, using the same glass plate to merge an old and new image. Nor too is image-based evidence new, with CCTV, smartphone evidence and dashcam footage playing an important role in trials today.
However, as we note in the Guide, certain facets of digital open source imagery are distinctive. First, where images are posted to social media, the ‘metadata’ (or ‘data about the data’) is routinely stripped. Metadata provides important information on the time, date, and device used in capturing the footage, which may be indicia of authenticity or reliability. Content is often posted by individuals who did not generate that image or video. Second, and relatedly, the poster’s identity may be anonymous, pseudonymous or unknown, or the information may have been posted to an account controlled by multiple individuals, obscuring the specific source. Third, ‘takedowns’ are common, making it difficult to determine which account posted the material initially.
Fourth, digital open source imagery may be inauthentic in multiple ways. The most common that we’ve seen is so-called ‘misattribution’. This is where content from the past is posted with a caption suggesting it is from a current event. In the war in Gaza, for example, alleged images have included misattributed footage from Syria, Yemen, Egypt, and Sweden. Less commonly, content may be ‘staged’, using actors or manipulating scenes to tell a particular story. In 2014, a video of ‘Syrian Hero Boy’, a little boy rescuing a girl from supposed crossfire in Syria, was revealed to have been filmed in Malta by a professional film crew. By the time the truth emerged, the video had been watched and shared millions of times, and reported as true by leading media agencies.
Other forms of manipulated imagery include deepfakes – synthetic media generated using artificial intelligence. While deepfakes from the conflicts in Ukraine and Gaza have been relatively easy to spot due to mismatched audio and visuals, or odd visuals, such as an incorrect number of fingers or toes, the underlying technology is becoming increasingly sophisticated; these ‘tells’ will soon disappear. That is why judges and lawyers need to rigorously interrogate the source, content, and technical aspects of potential evidence, as outlined below. We also need to be on the lookout for so-called ‘shallowfakes’, or content edited using techniques such as cutting or otherwise altering the footage, changing the speed, or applying filters. A video of former United States House Speaker Nancy Pelosi, slowed down to make it appear as if she were drunk, is a good example of this form of manipulation.
The Berkeley Protocol on Digital Open Source Investigations establishes that verification – the process of establishing the accuracy or validity of online information– can be broken down into three considerations, viewed holistically: source analysis, technical analysis, and content analysis. As noted above, the original captor of a video or image may be unknown, but an account’s posting history might be relevant. In the ‘Syrian Hero Boy’ example, had media outlets examined the account where this video first appeared, they would have seen that this was the first video of its kind from this particular source, unusual in the context in Syria where videographers were often active ‘citizen journalists’, posting frequently from the conflict.
Technical analysis can help determine what a piece of content is, as well as when, where and how it was captured. Online tools such as InVid or WeVerify can extract metadata from images and videos. However, metadata can be manipulated or incorrect, purposefully or not. In our Guide, we give an example of a single video with two different times and dates of creation depicted in the metadata. Researchers concluded that one indicated the time and date that the file was saved to a computer, while the other was the actual time and date of creation. For AI-generated content, invisible watermarks, imperceptible to the human eye or ear, may be embedded in the content and detected by software. A forensic image or video analyst may also be able to detect AI-generated content using specialised methods of analysis.
To further assess where a piece of content was captured, multiple methods can be used, including ‘geolocation’, the process of matching geographic features depicted in an image to satellite imagery. While a tree, a lamppost, or a fence may not be particularly distinctive features, powerful examples show that the relative positioning of features like these can lead investigators to match an image to a location.
Reverse image searching – running a visual piece of content across one or more search engines – can help clarify when a piece of imagery evidence was created, for example by indicating whether it has appeared before online. Aside from advanced chronolocation techniques like shadow analysis, the when of a piece of content can often be determined by more basic clues like the weather or signage that changes over time.
Lastly, in determining the why of a piece of content, fact-finders should ask whether the source demonstrates a political affiliation or bias. This does not necessarily mean that the content is unreliable, but it may indicate that footage might have been edited or framed to tell a particular story that is less than objective. The New York Times has convincingly shown how the perspective of police body camera footage can lead viewers to believe a confrontation was threatening, where the same scene shot from a different angle tells a very different story.
In training groups as diverse as judges, lawyers, investigators, police forces, and journalists worldwide, the question we most commonly get asked is, ‘Isn’t there software that I can just use to check whether a digital item is real or fake?’. We caution against relying on such tools, for two reasons. The first is that these tools, while sometimes improving in accuracy, are far from fool proof. Their accuracy depends on the data used to train them, and the specific type of content they were trained to detect. Typically, such tools underperform with grainy or low-resolution footage, and with non-English language content – all common in conflict-related contexts.
Second, reliance on such tools may lead to what Riana Pfefferkorn has dubbed ‘the CSI effect’, where juries give little weight to a video unless the proponent uses a shiny technical programme to prove its veracity. Given that such tools may not be as accurate as an investigator-witness explaining how they assessed a video or photo’s source, metadata, and content, or the witness who captured it vouching for its authenticity, lawyers need to be careful not to throw the baby out with the epistemic bathwater. We hope that our Guide will help lawyers and judges to more confidently assess the authenticity of digital open source evidence, strengthening its use in justice processes.
Yvonne McDermott Rees is Professor of Law at Swansea University and an Academic Bencher of the Honourable Society of the Inner Temple.
Alexa Koenig is Research Professor of Law and Co-Faculty Director of the Human Rights Center at the University of California, Berkeley.
In 2021, a story of a mother in Pennsylvania, arrested for harassment after allegedly creating ‘deepfake’ footage of her daughter’s cheerleading rivals, quickly went viral. The police officer in charge of the investigation declared, ‘This tech is now available to anyone with a smartphone – your neighbour, somebody who holds a grudge… All one needs to do is download an app and you’re off to the races.’ Yet, it later turned out that the incriminating videos were, in fact, real.
This story is the perfect example of ‘the liar’s dividend’, which legal scholars have long been warning about. The idea is that, as the ability to create convincing fake footage increases, so too does the scope for denying the veracity of real footage – exactly the script adopted by the cheerleading rivals, who didn’t want to be linked to the behaviour captured in the videos. As our previous research has demonstrated, given the very real value of digital information in proving atrocity crimes and mass human rights violations, this is concerning – especially given the growing popularity of digital open source investigations, which rely on information that any member of the public can access online through observation, purchase or request, such as videos or photos posted to social media. There is a risk that lawyers will self-censor, excluding important and relevant imagery, and that judges may unwittingly rely on manipulated evidence, or discredit real evidence as ‘potentially fake’.
This is why we co-authored, as part of a group of experts in open source investigations, Evaluating digital open source imagery (‘the Guide’) – a guide that seeks to demystify digital open source information for judges and other fact-finders. The Guide also provides insights for lawyers on the unique facets of this kind of evidence, including how to ensure that it is what it purports to be.
Image manipulation is nothing new. Raquel Vazquez-Llorente of WITNESS has written about the fraud trial, in 1869, of William Mumler. Mumler’s ‘spirit photography’, which promised to capture customers’ deceased loved ones in photographs, was made possible through a basic double exposure technique, using the same glass plate to merge an old and new image. Nor too is image-based evidence new, with CCTV, smartphone evidence and dashcam footage playing an important role in trials today.
However, as we note in the Guide, certain facets of digital open source imagery are distinctive. First, where images are posted to social media, the ‘metadata’ (or ‘data about the data’) is routinely stripped. Metadata provides important information on the time, date, and device used in capturing the footage, which may be indicia of authenticity or reliability. Content is often posted by individuals who did not generate that image or video. Second, and relatedly, the poster’s identity may be anonymous, pseudonymous or unknown, or the information may have been posted to an account controlled by multiple individuals, obscuring the specific source. Third, ‘takedowns’ are common, making it difficult to determine which account posted the material initially.
Fourth, digital open source imagery may be inauthentic in multiple ways. The most common that we’ve seen is so-called ‘misattribution’. This is where content from the past is posted with a caption suggesting it is from a current event. In the war in Gaza, for example, alleged images have included misattributed footage from Syria, Yemen, Egypt, and Sweden. Less commonly, content may be ‘staged’, using actors or manipulating scenes to tell a particular story. In 2014, a video of ‘Syrian Hero Boy’, a little boy rescuing a girl from supposed crossfire in Syria, was revealed to have been filmed in Malta by a professional film crew. By the time the truth emerged, the video had been watched and shared millions of times, and reported as true by leading media agencies.
Other forms of manipulated imagery include deepfakes – synthetic media generated using artificial intelligence. While deepfakes from the conflicts in Ukraine and Gaza have been relatively easy to spot due to mismatched audio and visuals, or odd visuals, such as an incorrect number of fingers or toes, the underlying technology is becoming increasingly sophisticated; these ‘tells’ will soon disappear. That is why judges and lawyers need to rigorously interrogate the source, content, and technical aspects of potential evidence, as outlined below. We also need to be on the lookout for so-called ‘shallowfakes’, or content edited using techniques such as cutting or otherwise altering the footage, changing the speed, or applying filters. A video of former United States House Speaker Nancy Pelosi, slowed down to make it appear as if she were drunk, is a good example of this form of manipulation.
The Berkeley Protocol on Digital Open Source Investigations establishes that verification – the process of establishing the accuracy or validity of online information– can be broken down into three considerations, viewed holistically: source analysis, technical analysis, and content analysis. As noted above, the original captor of a video or image may be unknown, but an account’s posting history might be relevant. In the ‘Syrian Hero Boy’ example, had media outlets examined the account where this video first appeared, they would have seen that this was the first video of its kind from this particular source, unusual in the context in Syria where videographers were often active ‘citizen journalists’, posting frequently from the conflict.
Technical analysis can help determine what a piece of content is, as well as when, where and how it was captured. Online tools such as InVid or WeVerify can extract metadata from images and videos. However, metadata can be manipulated or incorrect, purposefully or not. In our Guide, we give an example of a single video with two different times and dates of creation depicted in the metadata. Researchers concluded that one indicated the time and date that the file was saved to a computer, while the other was the actual time and date of creation. For AI-generated content, invisible watermarks, imperceptible to the human eye or ear, may be embedded in the content and detected by software. A forensic image or video analyst may also be able to detect AI-generated content using specialised methods of analysis.
To further assess where a piece of content was captured, multiple methods can be used, including ‘geolocation’, the process of matching geographic features depicted in an image to satellite imagery. While a tree, a lamppost, or a fence may not be particularly distinctive features, powerful examples show that the relative positioning of features like these can lead investigators to match an image to a location.
Reverse image searching – running a visual piece of content across one or more search engines – can help clarify when a piece of imagery evidence was created, for example by indicating whether it has appeared before online. Aside from advanced chronolocation techniques like shadow analysis, the when of a piece of content can often be determined by more basic clues like the weather or signage that changes over time.
Lastly, in determining the why of a piece of content, fact-finders should ask whether the source demonstrates a political affiliation or bias. This does not necessarily mean that the content is unreliable, but it may indicate that footage might have been edited or framed to tell a particular story that is less than objective. The New York Times has convincingly shown how the perspective of police body camera footage can lead viewers to believe a confrontation was threatening, where the same scene shot from a different angle tells a very different story.
In training groups as diverse as judges, lawyers, investigators, police forces, and journalists worldwide, the question we most commonly get asked is, ‘Isn’t there software that I can just use to check whether a digital item is real or fake?’. We caution against relying on such tools, for two reasons. The first is that these tools, while sometimes improving in accuracy, are far from fool proof. Their accuracy depends on the data used to train them, and the specific type of content they were trained to detect. Typically, such tools underperform with grainy or low-resolution footage, and with non-English language content – all common in conflict-related contexts.
Second, reliance on such tools may lead to what Riana Pfefferkorn has dubbed ‘the CSI effect’, where juries give little weight to a video unless the proponent uses a shiny technical programme to prove its veracity. Given that such tools may not be as accurate as an investigator-witness explaining how they assessed a video or photo’s source, metadata, and content, or the witness who captured it vouching for its authenticity, lawyers need to be careful not to throw the baby out with the epistemic bathwater. We hope that our Guide will help lawyers and judges to more confidently assess the authenticity of digital open source evidence, strengthening its use in justice processes.
How to navigate open source evidence in an era of deepfakes. By Professor Yvonne McDermott Rees and Professor Alexa Koenig
The beginning of the legal year offers the opportunity for a renewed commitment to justice and the rule of law both at home and abroad
By Louise Crush of Westgate Wealth Management sets out the key steps to your dream property
A centre of excellence for youth justice, the Youth Justice Legal Centre provides specialist training, an advice line and a membership programme
By Kem Kemal of Henry Dannell
By Ashley Friday of AlphaBiolabs
Providing bespoke mortgage and protection solutions for barristers
Joanna Hardy-Susskind speaks to those walking away from the criminal Bar
From a traumatic formative education to exceptional criminal silk – Laurie-Anne Power KC talks about her path to the Bar, pursuit of equality and speaking out against discrimination (not just during Black History Month)
Irresponsible use of AI can lead to serious and embarrassing consequences. Sam Thomas briefs barristers on the five key risks and how to avoid them
Yasmin Ilhan explains the Law Commission’s proposals for a quicker, easier and more effective contempt of court regime
James Onalaja concludes his two-part opinion series
