Scoring Tabletop Exercises: Evaluation Guides and Rubrics

Evaluation turns a tabletop from an interesting conversation into evidence you can act on. Done badly, it turns into a report card that makes people defensive and teaches them to perform rather than to be candid. The rule that makes evaluation work is simple: evaluate capabilities, not people. This guide shows how to structure evaluation around objectives, which rating scales to use, what to measure, and how to keep the process fair.

It builds on the planning guide and the MSEL; your objectives and injects define what evaluators look for.

From objectives to capability targets

The HSEEP methodology frames evaluation around capabilities: what the organisation must be able to do, and the target level of performance. For a cyber tabletop, convert each objective into one or more capability targets with observable critical tasks.

Critical tasks must be observable in a discussion. "The team understands containment" cannot be observed; "the team names who can approve disconnecting the virtualisation network" can.

Exercise Evaluation Guides

An Exercise Evaluation Guide (EEG) is a one-to-two-page form per capability that tells evaluators what to watch for and gives them a place to record evidence. HSEEP provides EEG templates for many capabilities; for cyber exercises you will usually adapt them. A practical EEG contains:

• The capability and its target.
• The critical tasks, each with space for observations and the exercise time they occurred.
• Related injects from the MSEL.
• A rating field.
• Space for strengths, areas for improvement and root-cause notes.

## EEG — Containment decision-making

Capability target: Within 30 min of a containment request, the incident
commander obtains an approved go/no-go decision with a documented
rationale.

Related injects: INJ-03, INJ-04

| # | Critical task                                   | Observed? | Time    | Evidence / notes |
|---|--------------------------------------------------|-----------|---------|------------------|
| 1 | Decision owner for production-wide outage named  |           |         |                  |
| 2 | Business impact assessed with service owner      |           |         |                  |
| 3 | Evidence-preservation impact considered          |           |         |                  |
| 4 | Decision, approver and time recorded             |           |         |                  |
| 5 | Business and comms informed before execution     |           |         |                  |

Rating (P / S / M / U): ___

Strengths:
Areas for improvement:
Suspected root cause (plan / training / resources / authority / tools):

Assign one or two evaluators per capability. Evaluators should sit where they can hear everything, and they should not speak during play.

Rating scales

HSEEP P/S/M/U

HSEEP uses a four-level qualitative scale for each capability target:

P/S/M/U works well because it is descriptive and ties directly to the objective. Always pair the rating with evidence.

A 1–5 maturity rubric

Some organisations prefer a maturity view that can be tracked over time. Keep the levels descriptive so two evaluators would reach the same score.

Use one scale per exercise, not both, to avoid confusing readers of the report.

Decision and timing metrics

Discussion-based exercises are not precise stopwatches, but exercise-time metrics are still valuable, especially when re-testing. The scribe's timestamped notes make them easy to extract.

Report these as observations, not targets. A team that declares severity in five minutes of exercise time by skipping triage has not done better than one that took twenty minutes and did it properly.

Evaluating without grading people

People speak candidly only if they trust that ratings will not be used against them. Protect that trust:

• Rate capabilities, never individuals. The report says "containment decision authority was unclear", not "the CIO could not decide".
• No names in findings. Refer to roles where necessary.
• Look for root causes in the system. Was the gap caused by the plan, training, resources, authority or tools? Almost never by a person being "bad".
• Share draft ratings with the planning team before the report, to catch misunderstandings.
• Keep raw notes restricted. Distribute the report, not the scribe's verbatim log.

Capturing observations well

Good evaluation depends on good observations. Train evaluators to record:

• What happened, factually, with exercise time.
• What was expected, referencing the plan or the MSEL's expected actions.
• The gap, if any, between the two.
• Strengths as carefully as weaknesses. Strengths tell you what to preserve and are important for morale.

Useful observation shorthand: [+] strength, [GAP] area for improvement, [Q] open question, [ASSUME] untested assumption, [DEFER] decision pushed elsewhere.

Consolidating ratings after the exercise

Within a day or two, evaluators meet with the facilitator and scribe to:

1. Walk through each capability's EEG and agree the rating.
2. Merge duplicate observations.
3. Identify three to seven key findings, each with a root cause.
4. Draft recommendations that are specific, owned and testable.

These feed directly into the after-action report and its improvement plan.

Measuring improvement over time

Tabletop programmes show their value across exercises, not within one. Track:

• Capability ratings for the same capability across exercises.
• Improvement-plan items closed before the next exercise.
• Repeat findings: the same gap appearing twice is a governance issue, not a training issue.
• Coverage: which scenarios, teams and capabilities have been exercised in the last year.

A simple coverage matrix, with scenarios as rows and capabilities as columns, quickly shows blind spots, for example no exercise has tested out-of-band communications or supplier-driven incidents such as a supply-chain compromise.

Related

• Designing tabletop injects and the MSEL
• Writing the after-action report
• Glossary: HSEEP, after-action report