A lower bound on effective performance testing for digital forensic tools

The increasing complexity and number of digital forensic tasks required in criminal investigations demand the development of an effective and efficient testing methodology, enabling tools of similar functionalities to be compared based on their performance. Assuming that the tool tester is familiar with the underlying testing platform and has the ability to use the tools correctly, we provide a numerical solution for the lower bound on the number of testing cases needed to determine comparative capabilities of any set of digital forensic tools. We also present a case study on the performance testing of password cracking tools, which allows us to confirm that the lower bound on the number of testing runs needed is closely related to the row size of certain orthogonal arrays. We show how to reduce the number of test runs by using knowledge of the underlying system