Security

Threat modelling a public-sector system

How we model abuse of a credential-issuing system before writing code — trust boundaries, the abuse cases that actually occur, and what each one costs to stop.

Written forArchitects and security reviewers scoping a government or institutional system

Reading time11 min read

Last reviewed2026-08-14

Most threat models produced for procurement are inventories of controls: encryption at rest, TLS in transit, role-based access, a firewall. None of that is a threat model. A threat model names who would attack the system, what they would gain, and which specific mechanism stops them — and it is worth doing before the schema is written, because half of what it produces is a change to the data model rather than a control bolted on top.

What follows is the method we use on credential-issuing systems: passes, permits, registrations, anything where a piece of paper or a QR code grants a person access to a place or a right. It is drawn from building a digital pass system that runs inside a state legislature during live sessions.

Start with trust boundaries, not with assets

The standard advice is to enumerate assets. In practice that produces a list of tables. The more useful first pass is to draw every point where data or authority crosses from one party to another, because that is where the interesting failures live.

Boundary	What crosses it	What an attacker gains by crossing it wrongly
Applicant → issuing department	An identity claim and a purpose	A pass under a false identity or pretext
Department A → Department B	An approval decision	A pass approved by a department that never saw it
Issuing system → the credential	A signed artefact the holder carries	A forged or cloned credential
Credential → gate device	A scan, often offline	Entry with a revoked, expired or replayed pass
Gate device → central log	An entry event, sometimes minutes late	An entry that never appears in the record
Operator → the system	Administrative authority	Issuance without an application, or silent revocation

Six boundaries on one pass system. Note that only two of them are network boundaries — the rest are organisational, and no firewall addresses them.

The abuse cases that actually happen

Institutional systems are rarely broken by exotic attacks. They are broken by ordinary people taking the shortest path through a process under time pressure, and by insiders with legitimate access using it outside its purpose. The abuse cases worth modelling are therefore mundane, and the mitigations are mostly about making the shortcut harder than the correct path.

Pass sharing at the gateOne credential is issued, then handed to a second person after the holder is inside. Mitigation is not cryptographic: bind the credential to a photo the guard sees at scan time, and make the scan display it large enough to check in under two seconds. A control the guard cannot perform in the time available is not a control.

Replay of a captured QRA photograph of a valid pass is presented at a second gate. Rotating codes fail here because gates are frequently offline. What works is a short-validity signed token plus server-side single-use enforcement once the device syncs — the replay is not prevented at the gate, it is detected and attributed within minutes, which is what an audit actually needs.

Pressure issuanceA senior official asks an operator to issue a pass immediately, outside the approval chain. This is the most common real failure and it is a data-model problem: if the schema permits a pass with no linked application, the shortcut exists. Make the application row mandatory and the pressure moves to creating a retrospective application, which is visible, attributable and reviewable.

Quiet revocationAn operator revokes or edits a record to hide an earlier action. Prevented by making the credential table append-only, with state changes as new rows carrying actor, timestamp and reason. Nothing is deleted; the current state is derived.

The lost deviceA gate handset walks out of the building holding a local cache of valid credentials. Cache only what the gate needs to make a decision — identifier, validity window, photo hash — never the full record, and give the cache a hard expiry shorter than the session it serves.

Rating without theatre

Numeric risk scores invite argument about whether something is a 6 or a 7 and settle nothing. We rate on two axes a room of non-engineers can agree on: whether the abuse is visible when it happens, and whether it is attributable afterwards. Those two produce the priority order directly.

	Attributable	Not attributable
Visible	Accept and monitor	Fix the identity chain
Invisible	Add detection, then monitor	Fix first — this is where the incident you cannot explain comes from

The bottom-right quadrant is the one that ends careers, because the first time anyone learns about it is when a journalist asks who was inside the building. Everything there gets designed out before the build starts.

Producing the evidence as you go

A threat model written for an audit and a threat model used by the team diverge within a month. Keep one, and make it produce the audit artefact automatically: each abuse case carries its mitigation, and each mitigation names the artefact that proves it operates — a test, a log query, a screenshot of the gate flow, a configuration file under version control.

yaml

- id: ABUSE-03
  title: Pass issued without an approved application
  boundary: operator -> issuing system
  visible: false
  attributable: true
  mitigation:
    - schema: passes.application_id NOT NULL REFERENCES applications(id)
    - process: retrospective applications flagged in the daily issuance report
  evidence:
    - test: tests/issuance/test_requires_application.py
    - query: reports/daily_issuance_anomalies.sql
  review: every release

One entry from the register we keep alongside the code, reviewed each release.