What You Can Prove: Turning a Restore Test Into Evidence

Last in a six part series on restore testing. Same composite client throughout: a business administration firm, around 500 staff, two offices.

About eight months after the first test, one of their clients sent over a supplier assurance questionnaire. A regulated financial entity, doing the third party diligence that DORA has made everybody do, asking the usual forty questions about resilience.

Normally that’s three or four rounds of back and forth over five or six weeks. Somebody in operations writing prose about what they believe happens, somebody on the other side asking for substantiation, everybody mildly irritated.

They sent the pack from their second test, with a two page cover note mapping it to the questions. It came back agreed in one round.

That’s the part of this subject nobody talks about. The evidence isn’t just for the bad day. Most of the time it’s for a Tuesday, and it pays for itself in meetings that don’t happen.

three audiences, one document

Back in part one I said a restore test gets asked to do three jobs. Prove to yourself the thing works, prove to your insurer it happened, prove to your regulator you can do it repeatedly.

The mistake I see most often is treating those as three exercises. Somebody runs a technical test for IT, somebody else fills in the insurance proposal from memory in March, and a third person writes a continuity document for the auditor that bears no relationship to either.

They’re one exercise and one artefact with different front doors. The test produces the evidence, and the cover note changes depending on who’s asking. What doesn’t change is the underlying record, and if the underlying record is thin then all three front doors are decorated boxes with nothing behind them.

what each one actually wants

The underwriter wants your attestation to match reality. That’s it. The proposal form asks whether you test recovery and you tick a box, and eighteen months later at claim time somebody compares the box to what existed. The pack’s job is to make that comparison boring.

Worth adding a habit here that costs nothing and has saved people real money. When you answer an ambiguous question on a proposal form, and most of them are ambiguous, write down how you interpreted it and keep that note with the pack. “Do you test your backups annually” could mean any rung on the scoping ladder from part one. If there’s ever an argument about whether you answered honestly, a contemporaneous note showing you thought about the question is worth a great deal more than your recollection three years later.

The regulator wants a capability, not an event. Under NIS2, business continuity including backup management and disaster recovery is one of the named risk management measures, and the expectation is an ongoing programme rather than something you did once. DORA is more explicit again about testing on a regular basis and about the resilience of the third parties you depend on.

The practical consequence of both is the same. One pack proves an event. Four packs across two years, showing findings that got closed and numbers that moved, prove a capability. The series is the evidence, not any individual document in it.

I’d add the obvious caveat that I’m an engineer rather than a lawyer, and where something has contractual or regulatory consequence you want someone qualified reading it. What I can tell you is what holds up in the room.

Your own organisation wants to know what to fix. Which is the one that actually improves anything, and the one that gets dropped first when the pack becomes a compliance chore.

what goes in it

Layered, so that the person who needs one page gets one page and the person who needs the console output can get to it.

A one page attestation at the front. What was tested, when, by whom, under what scope, what the outcome was, signed by somebody whose name means something. If a reader only ever sees this page, it should still be true and still be specific.

A scope statement using the rung language from part one, because “we tested recovery” is where all the ambiguity lives. Say whether you restored a file, a workload, a service, or a subset of the estate, and name the service.

The conditions. Time of day, who was present, what infrastructure was already warm, what was in scope and what wasn’t, and what help was given that wouldn’t be available at three in the morning. I banged on about this last week and it matters more here than anywhere. A number without its conditions is not evidence, it’s a claim.

The timeline. Phase timestamps, the way part five described, and the derived durations underneath them.

The findings. What failed, what surprised you, what took longer than expected.

Remediation, with an owner and a date against each item, and then closure evidence when it’s done.

And an appendix with the raw material. Job logs, console output, screenshots with clocks visible, whatever your platform produced. Nobody reads it. Its existence is the point.

a pack with no findings is not a good pack

This is the bit I’d most want to land.

Every instinct in an organisation pushes toward a clean report. Nobody wants to write down that the DSRM password was missing, and there’s usually somebody senior who’d rather the document said everything went well.

Resist it, because a clean pack is worth almost nothing to any of the three audiences. To the underwriter it reads as light touch. To a supervisor it reads as a test scoped not to fail. And to you it means you learned nothing and spent a day doing it.

If a restore test produces no findings, the test was too small. I said that in part one and it’s just as true at the point where you’re writing it up.

The corollary is that open findings are fine, as long as they have an owner and a date. What damages you is a finding with neither. An item sitting there for fourteen months with nobody’s name against it is the single worst thing you can put in front of anyone who assesses these for a living, because it tells them the process exists and nothing comes out the other end.

I’ve written bad ones myself. Early on I produced packs that were all outcome and no findings, because I thought that was what the client wanted to receive. They read like marketing, and the one time somebody with a proper audit background looked at one, the first question was why the findings section was empty. Fair question. I didn’t have a good answer.

language discipline

Small thing, disproportionate consequences. Write what you did, not what it means.

“Restored the case management platform and its database into an isolated environment and validated authentication and record retrieval for three named user accounts” is a sentence you can stand over.

“Successfully tested disaster recovery” is not. It’s broader than what happened, it invites a reader to assume estate scope, and if it ever gets held up against the actual exercise the gap is yours to explain.

The same applies to anything you send a client. Precision reads as competence anyway, so you lose nothing.

don’t build it afterwards

Reconstructing a pack weeks later produces a document that looks reconstructed. The timestamps are too round, the findings have quietly evaporated, and the file properties say it was created a fortnight after the date on the cover.

The fix is entirely mundane. Somebody takes notes as it happens, the way part five described with the phase timings. Screenshots get taken at the time rather than recreated. Photograph the whiteboard before somebody cleans it. Then the write up is an hour of tidying rather than an act of memory.

this month

If you’ve run a test in the last year and never wrote it up, write it up now from whatever you have, and be honest in the document about what you’re reconstructing and what you recorded at the time. An honest partial record beats nothing, and it beats a polished one that a forensic reader will pull apart.

If you haven’t run one, the pack template is not where to start. Go back to part four and build the small isolated environment, then run something modest and write down what happened. The document falls out of the exercise. It doesn’t work the other way round.

And find out who in your organisation currently answers the resilience questions on insurance proposals and client questionnaires. There’s a decent chance it’s somebody who has never seen a restore test and is answering from a document written in 2021. Get them the pack.

the end of the series

Six weeks of this, and the through line has been the same thing from different angles.

The data coming back was never the hard part and it was always the part that got measured. Identity, dependencies, somewhere safe to do it, and an honest clock are where the time actually goes, and none of them show up on a job report. The exercise that finds them is the same exercise that produces your evidence, which is why doing it properly costs you once and pays three times.

This client’s number went from a stated four hours against a real six hours forty, to a stated eight hours against a demonstrated ninety minutes of technical recovery inside a five hour picture they could explain line by line. Nothing about their backup platform changed. They bought no hardware.

Thanks for sticking with it. Next week I’m going back to something with PowerShell in it, which I’m told some of you have been waiting on since February.