NIST CFTT validation and what it establishes
- BODY
- NIST, U.S. Dept. of Commerce
- UNIT OF TESTING
- Tool version × function
- READ TIME
- 9 min
NIST’s Computer Forensic Tool Testing project tests a named version of a forensic tool against a published specification for one function — imaging, write blocking, deleted-file recovery — and reports where it conformed. A CFTT report is the strongest available evidence on Daubert’s error-rate factor. It says nothing about the examiner, the artifacts, or the inference drawn from them.
What is NIST CFTT?
The Computer Forensic Tool Testing project at the National Institute of Standards and Technology publishes test methodologies and results for digital forensic tools, function by function, against specifications developed in public.
What makes it valuable in litigation is its independence. Almost every other statement about a forensic tool’s accuracy originates with the party that sells it or the party relying on it. A CFTT report originates with a federal standards body that has no position on the matter, and it is published in full — specification, test assertions, procedure, and the anomalies found.
The project’s working method is worth understanding because it explains the shape of the output. For each category of tool, CFTT develops a specification setting out what a conforming tool must do, derives a set of test assertions from it, writes a test plan and test cases, and then reports how a named version behaved against each assertion. The deliverable is therefore not a grade and not a percentage. It is a conformance record with named exceptions.
Two related NIST resources come up in the same conversations. The Computer Forensic Reference Data Sets project publishes images whose contents are documented, so a tool can be run against material with a known correct answer. The National Software Reference Library publishes hash sets of known software, which is how an examiner excludes operating-system and application files from a review set rather than reading them.
What does CFTT actually test?
Functions, not products. This is the single most useful thing to know about it, and the thing most often got wrong in a reliability declaration.
A commercial forensic platform performs dozens of unrelated operations. CFTT testing addresses categories such as disk imaging, hardware and software write blocking, deleted-file recovery, forensic media preparation, string searching, and mobile-device acquisition — each with its own specification. A tool that conforms on imaging has been tested on imaging. It has not thereby been tested on its registry parser, its carving engine, or its timeline builder.
The version is part of the finding
Parsing behaviour changes between releases. A tool that reported one record count for a database in one build can report a different count in the next, because the developers changed how the write-ahead log and freelist pages are handled. A conformance record therefore attaches to a version, and a declaration citing a report for a version the examination did not run has cited the wrong document.
What does a CFTT report establish for Rule 702 purposes?
It converts one half of Daubert’s third factor from assertion into citation. That factor asks about the known or potential rate of error and about the existence and maintenance of standards controlling the technique’s operation, and a published conformance test against a public specification speaks directly to both halves — there is a standard, it is maintained by a named body, and this version was measured against it.
It converts one half of Daubert’s third factor from assertion into citation.
It also does something subtler that is worth naming in a declaration. Testability, Daubert’s first factor, asks whether the technique can be tested and has been. For most of what a forensic examiner does, the answer is that it can be and was — by someone else, in public, with the specification and the procedure available for anyone to repeat. That is a stronger answer on testability than most disciplines can give.
What does a CFTT report not establish?
Four things, and every one of them has been the ground a digital forensic opinion was actually narrowed on.
| THE GAP | WHY THE REPORT CANNOT CLOSE IT | WHAT CLOSES IT |
|---|---|---|
| The examiner | Testing measures the software. A conforming imager in untrained hands still produces an unreliable acquisition, and Rule 702 reaches the application as well as the method. | Documented training on that tool, the volume of examinations run with it, and contemporaneous notes — the foundation accepted in State v. Pratt. |
| The function you actually relied on | Coverage is uneven. Imaging and write blocking are well covered; artifact parsers, timeline builders and cloud collectors are much less so, and those are where contested findings come from. | Dual-tool verification or a manual read of the structure, per the error-rate guide. |
| What the artifact means | No conformance test speaks to interpretation. A parser can be certified to read a registry key correctly and the conclusion drawn from that key can still be wrong. | The artifact index discipline: what the record contains, and what it cannot establish. |
| This examination | A test report is about a build, not about a matter. It cannot say whether the write blocker was in line, whether hashes were recomputed at verification, or whether the working copy was handled. | The acquisition log, the custody record, and both hash values at both points — produced, not asserted. |
The pattern in the reported rulings is that tool reliability is rarely the losing ground. In Nucor Corp. v. Bell the court treated a bit-for-bit forensic copy as a reliable method, noting that many authorities recognise the imaging tool as reliable, and the challenge to the duplicate failed as unsupported speculation — while the same expert’s opinions on two named wiping programs were struck. The imaging was never the problem.
What if the tool an examination relied on has no CFTT report?
This is the common case rather than the exception, and it is survivable. The reported decisions show courts accepting a bundle of other indicia, and the bundle has a recognisable shape.
- Name the tool, the version, and who else uses it for this purpose. In In re Digital Music Antitrust Litigation the court treated the use of a commonly accepted digital forensic utility, named in the opinion, as part of the reliability showing. Naming the specific utility is cheaper than defending a bespoke script.
- Assemble the five-part proffer. Acceptance in the field, availability such that anyone can test it, the fact that it has been tested, published comparisons, and a stated potential rate of error. That is the structure the court accepted in Williford v. State, and its elements travel to any current product.
- Run the tool against known data yourself. A reference image with documented contents, or a device the examiner populated deliberately, produces a record that this installation gave the right answer on material where the right answer was known. That is validation in the sense that matters, and it is available to any laboratory.
- Reproduce the load-bearing finding a second way. In United States v. Chiaradio the agent had never seen the tool’s source code — it was deliberately secret — and was admitted because he could describe re-creating its output by hand. Independent corroboration is the substitute for access, and it is worth doing before the motion rather than after.
- Build a human review step and say so. In State v. Roberts the hash-matching toolkit survived partly because officers independently reviewed the files it identified rather than acting on the output directly. A documented review converts an argument about the tool into an argument about a check that was performed.
CFTT, SWGDE, ISO/IEC 17025 and in-house testing
Four different things get offered as “validation” and they answer four different questions. Confusing them produces a declaration that sounds thorough and covers one point twice.
| SOURCE | WHAT IT IS ABOUT | WHAT IT IS GOOD FOR IN A 702 FIGHT |
|---|---|---|
| NIST CFTT | A named tool version performing a named function, measured against a public specification. | The error-rate factor, and the strongest answer available on testability. Cite the report for the version actually run. |
| SWGDE publications | What the discipline expects of acquisition, examination, reporting and quality practice. Guidance rather than binding rules. | The 'standards controlling the technique's operation' half of factor three, and the benchmark a challenge will measure the examination against whether or not the examiner consulted it. |
| ISO/IEC 17025 | A laboratory's management system — documented procedures, competence records, equipment control, method validation — for a defined scope of testing. | Institutional process. It attaches to an organisation rather than a person, is not required in order to testify, and says nothing about work outside its scope, so the question to ask is what the scope covers. |
| In-house validation | This installation, on this workstation, producing the correct answer on material whose contents were known in advance. | The gap between a published test of a build and the machine the examination was actually run on. It is also the only one of the four a small practice controls entirely. |
What belongs in the report, and what belongs in an exhibit
- In the report: the tool and version at each step, the verification performed, and any known limitation of the tool that bears on a stated finding.
- In an exhibit or a reliability declaration: the test reports, the specification they were run against, the written procedure the examination followed, and the accreditation scope where one applies.
- Nowhere: a general statement that the tools used are “industry standard and validated”. It is the sentence that reads as a reliability showing and supplies none, and it decays — the field-standard claim accepted in Sanders v. State was true of that product in 2006 and is a claim that has to be re-established for the version in the report.
Frequently asked questions
Does a NIST CFTT report make forensic evidence admissible?
No. It supplies evidence for one part of one Daubert factor — that a named version of a tool performed as specified on a defined function. Admissibility also requires that the examiner applied the tool competently, that the artifacts relied on record what the opinion says they record, and under Rule 702(d) that the opinion reflects a reliable application of the method to the facts of this case.
Is a forensic tool inadmissible if CFTT has not tested it?
No, and most tools in daily use have no current report. Courts have found reliability from other indicia: general acceptance in the field, published comparisons, commercial availability so that anyone can test the tool, the examiner's documented training and volume of use, and independent verification of the specific result. In United States v. Chiaradio a tool that had not been independently tested was admitted on exactly that basis.
Does CFTT test the whole forensic suite?
No. CFTT works at the level of a function rather than a product — disk imaging, write blocking, deleted-file recovery, string searching, mobile acquisition and similar categories each have their own specification and test assertions. A report covers a named version performing a named function, which is why the report cited in a declaration has to match the version and the function the examination actually used.
What is CFReDS?
The Computer Forensic Reference Data Sets project, also at NIST, which publishes reference images with documented contents so an examiner can run a tool against material whose correct answer is known in advance. It is the resource that makes in-house function-level validation practical for a laboratory that is not going to build its own reference images.
What is federated testing?
A CFTT initiative that distributes the test methodology as a self-contained suite so a laboratory can run the tests itself and produce a report in the same format. It matters in litigation because it means a laboratory can hold a version-specific test report for the build it actually runs, rather than citing someone else's test of a different version.
Should the tool and version appear in the expert report?
Yes. Naming the tool and version is what makes any testing record citable at all, and it is also what lets an opposing examiner reproduce the work — the practical form of Daubert's testability factor. A report that says the device was imaged and analysed, without naming what did the imaging, has forfeited both.
Law & Forensics records the tool and version at every step and the verification performed on each load-bearing finding, so a reliability declaration can cite documents rather than adjectives. If tool validation is in issue — start a conflicts check or reach us directly below.
ENGAGE AN EXPERT→Or write to info@lawandforensics.com or call 855-529-2466.
Related reading
- What is a forensic tool's error rate?
The three different things litigators call an error rate, and the five ways courts have found reliability without a published figure.
- Open-source forensic tools under Rule 702
Why “commercial” is not a reliability finding, and why the real factor-four problem is a bespoke script rather than an open codebase.
- The Daubert factors applied to forensic method
All four factors translated for a digital forensic examination, and the point where 702(d) takes over from them.
- Glossary of digital forensic terms
CFTT, SWGDE, ISO/IEC 17025, write blocker, E01 — each defined by its evidentiary consequence rather than its mechanics.
- Computer forensics
How acquisition, verification and analysis are actually run, and which artifact answers which question.
Attorney advertising / expert services. General information about evidence law and forensic practice, not legal advice, and not a substitute for checking the rules and case law of your own forum.