The Daubert factors applied to forensic method

The four factors, restated

Daubert listed four observations a court may use to test the reliability of expert testimony: testability, peer review and publication, known or potential error rate together with the standards controlling the technique’s operation, and general acceptance in the relevant community.

The list is not a test and was never presented as one. What makes it awkward for digital forensics is that the four factors were written with a laboratory in mind — a technique applied repeatedly to comparable specimens, where a false-positive rate can be measured. A forensic examination is not that. It is a sequence of decisions about which records to read on one machine that will never exist in that state again.

The result is that each factor has to be translated before it means anything here, and the translation is where most Rule 702 briefing on digital evidence goes wrong in both directions. Counsel attacking an examination asks for a published error rate that no honest examiner can produce. Counsel defending one answers the factors about the tool and never about the inference, which is the half the court is actually being asked to admit.

The remainder of this page takes the factors one at a time, says what each one is asking of a forensic examination, and identifies what courts have accepted as an answer. The general framework — the gatekeeping standard, the December 2023 amendment, and how opinions are attacked and defended — is set out in the guide to Daubert challenges to digital evidence.

Factor one: what does testability mean when the evidence is a disk image?

Testability in a laboratory means the technique can be run against known material and its output compared with the known answer. In a forensic examination it means something narrower and more useful: another competent examiner, given the same image and the same written record of what was done, can repeat the work and get the same result.

That makes testability a property of the documentation rather than of the discipline. Hashing is testable by anyone; an examination described in a report as “the device was imaged and analysed” is not testable by anybody, including the examiner who performed it. The three things that decide the question are the acquisition record, the tool and version at each step, and contemporaneous notes.

What is actually reproducible, and what is not

Some of a forensic examination reproduces exactly, and some of it does not. A hash computed over a static image reproduces to the digit. A carved file recovered from unallocated space may not reproduce at all between two tools, because carving reconstructs content from fragments and two engines will make different choices about which fragments belong together. A parse of the same SQLite database can return different record counts depending on whether the tool reads the write-ahead log and freelist pages or only the committed tables.

None of that is malfunction, and an examiner who claims everything reproduces identically has said something that is easy to disprove. The defensible position is to state which findings are exactly reproducible, which are tool-dependent, and what was done about the second category.

Factor two: is artifact interpretation peer reviewed?

Partly, and the distinction matters. There is a substantial published literature on what individual Windows, macOS and Linux artifacts record — where shellbags are written, what fields a Prefetch file holds, how the two timestamp sets in the master file table are populated by different code paths. Much of it is vendor and independent research rather than journal publication, and the tooling that implements it is documented in public.

What is not peer reviewed is the step after that. No literature says what a particular shellbag on a particular machine means about a particular person’s conduct, and no publication can, because that is the part of the opinion that belongs to the examiner. Treating the existence of artifact research as peer review of the conclusion collapses the same two things the rest of this page keeps apart.

Peer review inside the engagement

There is a second and more persuasive kind of review available: a documented internal check of this examination. The D.C. Circuit’s opinion in United States v. Morgan records that the expert had entered a wrong code during his draft analysis and that peer review caught it — and the testimony was admitted. A review step that finds an error and documents the correction is evidence of a controlled process, not an admission of unreliability. An examination with no review step has nothing to offer on this factor at all.

A review step that finds an error and documents the correction is evidence of a controlled process, not an admission of unreliability.

Factor three: what is the error rate of a forensic examination?

This is the factor that produces the worst answers on both sides, and it has its own page — what a forensic tool’s error rate actually means — because the honest answer is usually about validation rather than a number. In short: courts have accepted testimony that a tool has a low potential rate of error, testimony that a tool has no error rate with respect to the specific thing it identifies, and testimony that a hash comparison is accurate to a stated confidence. What they have not required is a published false-positive rate for a forensic examination as a whole.

The second half of the factor is quietly the more useful one, and it is routinely dropped. Daubert paired error rate with the existence and maintenance of standards controlling the technique’s operation. That half is answerable with documents: SWGDE best-practice publications, NIST CFTT test reports for the named tool and version, a written laboratory procedure, and ISO/IEC 17025 accreditation where the laboratory holds it and the work falls inside its scope.

Read that way, factor three asks two questions of an examiner and not one: how wrong can this go, and what governs how you ran it. The second question has a paper answer. Producing it is a better use of a reliability declaration than arguing about a statistic nobody publishes.

Factor four: general acceptance of the tool, or of the inference?

General acceptance is the easiest factor to satisfy and the easiest to satisfy for the wrong proposition. Mainstream imaging and extraction products are accepted; that has been established in reported decisions for two decades. In Williford v. State the reliability showing for EnCase was that it is generally accepted in the computer forensic community, is commercially available and therefore testable by anyone, has been tested, has been the subject of published comparisons, and has a low potential rate of error. In Sanders v. State the court accepted that the same product was a field standard about which treatises had been published.

Both of those proffers are about a tool. Neither says anything about whether the field accepts the inference the examiner drew from the tool’s output, which is usually the contested part. And the field-standard claim has a shelf life: a product that was the standard in 2006 may not be in the year of the report, so the showing should cite current literature and the version actually run.

The bespoke-script problem

Where this factor genuinely bites is a one-off script. A parser written for this matter, run once, never tested against known data, and never used by anyone else has no general acceptance to claim, no published comparison to cite, and no error characteristics anybody has measured. That is a real factor-four problem, and it is distinct from the question of whether the tool is open source — which is addressed separately in open-source forensic tools under Rule 702.

Which factors does a court actually apply to digital forensics?

Whichever ones fit. Kumho Tire gave trial courts broad latitude to decide how to test reliability, and the reported digital-forensics rulings show courts using that latitude freely. The First Circuit described the factors as the basis for a flexible inquiry into overall reliability rather than a definitive checklist, and affirmed admission of testimony about a peer-to-peer tracing program that had not been independently tested and whose source code was deliberately kept secret.

Two things follow. First, a challenge built entirely on factor three is weak, because courts have repeatedly found reliability without it. Second, a proffer built entirely on the tool is exposed, because nothing in that table reaches the inference.

Where the factors stop and Rule 702(d) begins

The four factors are about principles and methods — Rule 702(c). Since December 1, 2023 the rule has separately required that the opinion reflect a reliable application of those principles and methods to the facts of the case, and the proponent must show each requirement is more likely than not satisfied. That is Rule 702(d), and it is where digital forensic opinions are actually lost.

The mechanics are visible in the reported rulings. In United States v. Evans the agent was qualified and his general testimony about how cellular networks operate was reliable; his theory for estimating tower coverage range was excluded, because the link between the undisputed call records and his conclusions was deficient. The data was fine. The method for getting from the data to the opinion was not. The court then split the demonstratives along the same line, admitting the maps without estimated coverage rings and excluding those with them.

The contrast is United States v. Owens, where the Seventh Circuit found the opinion connected to the data by more than the expert’s say-so: matching info hashes, a hash match for every one of the 226 pieces of the file, the installed client version, and a most-recently-used folder entry. Four independent artifacts pointing the same way is the practical answer to the analytical-gap objection, and it is a 702(d) argument rather than a factor argument.

How to tell which half you are arguing about

• A factor argument is about the class of work. Is disk imaging a reliable technique? Is hash verification accepted? Does this parser have a testing record? These are answerable with documents and rarely decide a digital-evidence motion, because the answers are yes.
• A 702(d) argument is about this examination. Was the method the report describes actually followed? Does the cited artifact record the thing the conclusion asserts? Was the retention window of the log established or assumed? These decide motions.
• The verb change marks the boundary. Where a report moves from “the registry records” or “the journal contains” to “the user copied”, the factors have stopped applying and 702(d) has started.

One practical consequence for anyone writing a report. The factors can be satisfied in advance, once, in language that will serve every matter: the tool, the version, the testing record, the procedure. The application requirement cannot. It is satisfied only by the notes taken while the work was being done, which is why the defensive work on a Rule 702 motion happens during the examination and not during the briefing.

Frequently asked questions

Related reading

• Daubert challenges to digital evidence

  The framework this page sits inside — amended Rule 702, Daubert versus Frye, and the six ways forensic opinions actually get excluded.

• What a forensic tool's error rate actually means

  Factor three at length: why the honest answer is about validation rather than a number, and what courts have accepted instead.

• The Daubert Docket

  Fifty-five rulings on the admissibility of digital forensic testimony, each traceable to the opinion it was read from.

• Daubert exposure check

  A structured pass over an examination or a report, factor by factor, to find the ground a motion would be filed on.

• The artifact index

  What each artifact records and what it cannot establish — the reference the reliable-application question is answered from.

• Expert witness testimony

  Rule 26 reports written so the opinion, its basis, and its limits are all on the page — plus rebuttal and neutral engagements.

Attorney advertising / expert services. General information about evidence law and forensic practice, not legal advice, and not a substitute for checking the rules and case law of your own forum.