ISACA CISA: Evidence Quality in IT Audits
An audit conclusion is only as strong as the evidence supporting it. Large quantities of screenshots, exported reports, policy documents, and interview notes can create the appearance of rigor while still failing to prove that a control operated as described across the relevant period and population.
Within Security Governance & Assurance, evidence quality depends on relevance, reliability, completeness, timing, source, and the relationship between the artifact and the audit objective. The auditor should know what claim each piece of evidence supports and what uncertainty remains after reviewing it.
Good evidence design begins before fieldwork. It identifies what would demonstrate the control, which sources are trustworthy, how populations will be reconciled, and when reperformance or independent extraction is necessary.
Link every artifact to an audit assertion
Evidence should answer a specific question: did the control exist, did it operate, did it cover the population, did it produce the intended result, and was the responsible party authorized? Collecting artifacts without a defined assertion produces files that are difficult to interpret later.
Workpapers should make that link explicit so another qualified reviewer can follow the reasoning from evidence to conclusion without reconstructing the engagement from memory.
Prefer evidence generated by the process being tested
System logs, transaction histories, configuration exports, immutable pipeline records, and access histories can provide stronger evidence than manually prepared summaries because they are closer to the activity being evaluated. Reliability still depends on whether the source itself is controlled and complete.
Operational evidence illustrates the broader point: telemetry becomes assurance evidence only when collection, retention, time, and integrity are understood.
Treat screenshots as narrow evidence
Screenshots are useful for configuration state, but they usually prove only what was visible at one moment. They may omit filters, hidden fields, history, and evidence that the same setting existed throughout the audit period.
Pair screenshots with system exports, histories, configuration records, or other data when the audit assertion covers continuous operation or a population rather than one point-in-time example.
When screenshots are appropriate, capture enough context to make them interpretable: the system, record identifier, date, relevant filter, and the field or control being demonstrated. Cropped images with no surrounding context can be impossible to validate later and may conceal a filter that excluded exceptions.
Use native exports or structured data for population testing whenever possible. Screenshots should support understanding, not replace data that can be independently analyzed.
Validate completeness of populations
Sampling is meaningful only when the population is complete. Reconcile audit extracts to an independent source such as transaction totals, deployment logs, inventory counts, or system-generated control totals before selecting samples.
If the population is generated manually by the control owner, the auditor should understand how records could be omitted. A perfect sample from an incomplete population can support the wrong conclusion.
Automated extracts should be versioned or preserved so the auditor can show exactly what was tested even if the live system changes later. Save query criteria, timestamps, row counts, and control totals. When evidence is too large to store directly, preserve reproducible extraction logic and a cryptographic or platform-supported reference to the source dataset.
If data is transformed before sampling, validate the transformation. Filters, joins, deduplication, and timezone conversion can silently remove or misclassify records. The auditor should be able to explain why the processed population still represents the control population accurately.
Assess evidence independence
Evidence prepared by the person responsible for the control may still be useful, but it carries different reliability than independently generated data. The auditor should adjust corroboration according to the risk that evidence could be incomplete, biased, or altered.
Security governance depends on transparent accountability. Assurance is stronger when evidence can be corroborated across owners, systems, or independent control functions.
Independence also applies to tools. A dashboard maintained by the control owner can be useful, but the auditor should understand whether its filters, calculations, and source data can be changed by the same person whose performance is being measured. For high-risk controls, corroborate the dashboard with underlying system data or an independently configured query.
IT audit practice relies on professional judgment about evidence sufficiency. More artifacts do not compensate for a weak source; sometimes one reliable system record proves more than a folder of manually prepared documents.
Preserve provenance and timing
Record where evidence came from, who extracted it, when it was obtained, which parameters or filters were used, and what period it covers. Those details determine whether the artifact can be reproduced and whether it actually matches the engagement scope.
For dynamic systems, timestamp and version context matter. A configuration retrieved after remediation may not demonstrate the state that existed when the issue occurred.
Chain-of-custody considerations become important when evidence may support a fraud, legal, regulatory, or forensic matter. Limit unnecessary handling, preserve original files, record transfers, and use hashes or platform controls where appropriate so the integrity of critical artifacts can be demonstrated.
For ordinary audits, the same principle still improves quality: preserve the original source and perform analysis on a copy when transformations are required. This makes it possible to reproduce calculations or correct an analysis without recollecting the evidence.
Use reperformance for critical controls
When a control is important and can be independently executed, reperformance can provide strong evidence. The auditor might recalculate a sample, re-run a report, test a permission path, or independently reproduce a configuration query using controlled inputs.
Reperformance is not required for every control, but it is valuable when management-generated evidence is complex, judgment-heavy, or easy to manipulate.
Reperformance should use independently selected inputs when possible. If management provides both the test sample and the expected result, the auditor may only be confirming management’s demonstration. Selecting inputs from a validated population creates stronger evidence that the control works beyond a curated example.
Where reperformance is impractical, combine observation, inspection, and corroborating system evidence. The goal is sufficient confidence in the control, not adherence to one test technique regardless of cost or feasibility.
Use analytics to test more than a handful of records
Data analytics can identify outliers, duplicates, missing approvals, unusual timings, stale accounts, policy exceptions, and other patterns across the full population. Samples can then be targeted toward the risk rather than selected blindly.
Risk-based scoping is stronger when analytics reveal where control behavior differs from the norm and where deeper testing is likely to matter.
Analytics can also challenge management assertions before sampling. Compare declared change windows with actual timestamps, approved user populations with active accounts, backup schedules with completed jobs, or policy requirements with configuration states. Population-level exceptions can reveal control design problems that a traditional sample might miss.
Then use targeted samples to understand why the exceptions occurred. Analytics identifies where behavior differs; detailed testing establishes whether the difference represents error, legitimate exception, or a deeper process weakness.
Handle contradictory evidence directly
Audits often encounter artifacts that disagree: a policy says one thing, configuration shows another, and an interview describes a third process. Contradiction should not be smoothed over by selecting the most convenient source.
Investigate the difference, determine which evidence represents actual operation, and document the uncertainty. Contradictory evidence can reveal governance drift, undocumented exceptions, or ineffective change control.
Write findings from evidence outward
The finding should state the condition supported by evidence, the expected criterion, the risk created by the gap, and the practical action needed. Avoid conclusions that are broader than the evidence can sustain.
CISA emphasizes evidence collection, audit analytics, reporting, and quality improvement because audit credibility depends on conclusions that another professional can reproduce from the workpapers.
Preserve contradictory or negative evidence in the workpapers even when management remediates quickly. The engagement should show the condition that existed during the audit period, the corrective action taken, and any validation performed afterward. Replacing the original evidence with a clean post-fix screenshot destroys the chronology that supports the finding.
Cloud and change audits are good examples: cloud evidence and change evidence often come from fast-changing systems. Provenance and time are essential to distinguish historical control operation from the state observed after remediation.
Quality review should challenge whether the evidence supports the strength of the wording. Terms such as “all,” “never,” or “systemic” require broader support than one failed sample. Precise language protects audit credibility and makes it easier for management to design proportionate remediation.
Preserve enough evidence for later follow-up. When remediation is tested months later, the original condition, population, and criteria should still be understandable so the reviewer can compare the new state with the issue that was actually reported.
Evidence retention should align with the audit function’s review and regulatory needs. If source records expire before quality review, external inspection, or follow-up can occur, preserve the necessary artifacts through an approved evidence repository. Retention should be proportionate and secure so audit traceability does not become a new uncontrolled store of sensitive information.
When evidence contains sensitive personal, security, or commercial information, access to the workpapers should follow least privilege. Audit teams need enough detail to support the conclusion without copying entire production datasets unnecessarily. Minimizing collected data reduces both confidentiality risk and the burden of protecting audit repositories over their retention period.