Generative Artificial Intelligence (GenAI) has significantly increased the sophistication and ease of image tampering techniques, posing challenges for digital forensics in identifying manipulated images. A lack of dataset standardisation hinders the ability to effectively benchmark and compare GenAI inpainting localisation techniques, reducing their reliability in digital forensic applications. This paper aims to address this gap by exploring the need for standardised criteria for datasets in digital forensics for benchmarking detection techniques through preliminary experiments.
To address the limited diversity in existing datasets, a small-scale dataset was developed, consisting of 240 tampered images, 20 masks and 20 authentic images. This dataset includes four subject image classes (animals, objects, persons, scenery) and three inpainting tools (GLIDE, GalaxyAI, Photoshop). The dataset was evaluated against 13 localisation algorithms from the Image Forensics MATLAB Toolbox to determine key components that should be considered in the standardisation of testing environments.
The results show that the images in the animals and persons categories achieved the highest F1-Scores and accuracy over the other classes. Among tools, GLIDE inpainted images were consistently shown to be the most challenging to detect, underscoring the importance of further investigating these images. These findings provide foundational insights for identifying a set of criteria to establish robust testing environments, enabling the development of reliable and accurate GenAI inpainting localisation techniques.