Book a consultation

Automation Change Management

PLC Ransomware Recovery: The Runbook for Controllers, HMIs, SCADA and Engineering PCs

PLC ransomware recovery step by step: the order to restore controllers, HMIs, SCADA and engineering PCs, what must exist first, and how to validate logic.

Vladimir Romanov


PLC ransomware recovery is the work of getting a plant’s control systems back to a known, trusted state after an attack: verifying or restoring controller programs, rebuilding HMIs and SCADA servers, and bringing engineering workstations back so the plant can be changed again. In most incidents the PLCs keep running. What stops production is the loss of the Windows systems around them, and what prolongs the outage is not knowing whether the programs in the controllers can still be trusted.

This page sets out the recovery sequence, what has to exist before the incident for that sequence to work, why offline and immutable copies decide the outcome, and how to validate restored logic. It is written for controls and maintenance leaders working alongside an incident response team, not as a substitute for one. It sits under the automation change management hub.

What ransomware does to a plant

Dim, quiet steel plant hall with idle processing lines, stacked coils and overhead lighting
A ransomware outage often looks like this: the controllers are intact, but with no screens and no engineering tools nobody can safely run the line.
AssetTypical state after an attackRestore fromHow to validate
PLCs and safety controllersUsually still running; may have been altered in a targeted attackLast known-good program held offlineUpload and compare with the known-good version; functional and safety test
HMI terminals (Windows-based)Often encrypted or untrustedClean image plus the HMI projectCheck screens, alarms and write paths against the controller
SCADA and historian serversOften encrypted; the primary target alongside ITImage from before the incident window, or a rebuild from installersLicences, tag database, connections, historian gaps
Engineering workstationsOften encrypted; may hold the only project copiesClean image built from known-good mediaSoftware releases, licences, no persistence of the attacker
Network devicesUsually intact; configuration may be alteredSaved configurationCompare running configuration with the saved copy
Robots, drives, CNCUsually intactTheir own backupsCompare where tools allow; test motion before release

The trap is the engineering workstation. In many plants it is the only place the current projects live. If it is encrypted and nothing was kept offline, the plant can no longer prove what its controllers should contain, and every program becomes suspect.

The PLC ransomware recovery runbook

The order matters. Restoring a server onto a network the attacker can still reach invites re-infection, and reconnecting a controller before its program is verified risks running a tampered machine.

The OT ransomware recovery sequence Eight steps in two rows of four. Top row: contain and make safe, assess and scope, build a clean engineering workstation from offline media, and restore network and backup infrastructure. Bottom row: verify and restore controllers against the offline known-good version, rebuild SCADA and historian servers, restore HMIs, and restart in stages with monitoring. Below, a bar marks which steps depend on copies held offline before the incident. 1 Contain isolate, safe state 2 Assess what is hit, when 3 Clean workstation from offline image 4 Infrastructure network, backups 5 Controllers compare, restore 6 SCADA servers image or rebuild 7 HMIs image plus project 8 Staged restart test, monitor Orange outline: the step needs a copy that was offline before the incident. Steps 3 to 7 all depend on it. Without that copy, recovery becomes reconstruction.
Five of the eight steps depend on material held offline before the attack. That, more than any tool, decides whether recovery takes days or weeks.
  1. Contain and make safe. Isolate the OT network from IT and the internet at the boundary firewall. Decide with operations whether each line keeps running, which is often possible, or stops in a controlled way. Preserve evidence for the incident responders, and notify the people your plan names, including insurers. Joint U.S. agency guidance on primary mitigations for OT puts the ability to revert to manual operation at the centre of this step, and asks that it is tested before it is needed.
  2. Assess and scope. List which systems are encrypted, which are suspect and which are clean, and establish the earliest time the attacker may have been present. That time sets which backups can be trusted: anything taken after it is suspect.
  3. Build a clean engineering workstation from offline, known-good media, on an isolated network. Install the engineering software releases the plant needs from offline installers, with licences. Nothing else in the sequence can happen without it.
  4. Restore infrastructure. Verify switch and firewall configurations against saved copies, restore the backup server or change management repository from its offline copy, and confirm it is clean before connecting anything to it.
  5. Verify and restore controllers. Upload the running program from each controller and compare it with the last known-good version from before the incident window. Where they match, leave the controller alone. Where they differ, investigate, then restore the known-good version during a controlled stop. Start with the controllers whose process is most critical or most hazardous.
  6. Rebuild SCADA and historian servers from images that predate the incident window, or from installers and saved projects if no trusted image exists. Re-apply licences and certificates, and expect gaps in historian data.
  7. Restore HMIs, from a clean image plus the HMI project. Check that each screen’s write actions reach the right controller tags.
  8. Restart in stages. Bring lines back one at a time, with functional and safety tests, heightened monitoring and the IT connection restored last, through a reviewed firewall rule set. Then change every password and key the attacker may have seen.

Where the hours go. In practice step 3 and step 5 dominate. A plant that has to find installers for a 2012 software release, or review two hundred programs by eye, measures recovery in weeks. Our article on manufacturing downtime shows how quickly that cost compounds.

Before PLC ransomware recovery: what has to exist

ItemWhy it mattersWhere to keep it
Offline known-good programs for every controller, robot and driveThe reference for step 5Immutable or disconnected storage, plus an off-site copy
Clean workstation image with the engineering software installedStep 3 in hours rather than daysOffline media, refreshed after software changes
Installers and licence files for every engineering release in useOld releases may not be downloadable any moreWith the offline backups
Server and HMI imagesSteps 6 and 7Offline, with dates recorded
Passwords, keys and recovery keys (including BitLocker)Imaging and restores fail without themA sealed, offline credential store
Asset inventory with criticalitySets the restore orderPrinted copy plus the offline set
The runbook itself, rehearsedPeople follow the plan they have practisedPrinted, in the control room

An inventory sets the order. Without one, recovery starts with whatever is loudest. The OT asset inventory page explains how to build one that records criticality and backup status together.

Offline and immutable copies

An attacker with administrator rights can reach anything that is online. That includes the backup share, the change management server and any cloud sync that deletes when the source deletes. The copy that survives is one the attacker cannot alter:

  • Immutable storage, where backups cannot be changed or deleted for a set retention period, even by an administrator
  • Disconnected media, rotated on a schedule and stored away from the network
  • Separate credentials for the backup system, not shared with the Windows domain

This is the same rule CISA gives every sector in its #StopRansomware Guide: offline, encrypted backups, tested regularly, and maintained “golden images” of critical systems. In a plant the golden image is the clean engineering workstation with every software release it needs already installed.

Keep enough history. If the attacker was present for weeks before encrypting, the last clean backup may be a month old. Retain known-good versions well beyond the typical dwell time, and mark each one with the date it was verified.

Validating restored logic

Restoring a program is not the same as trusting it. Three checks close the gap:

  • Compare against the known-good version, not against the most recent backup. A compare tool that reads the device and shows differences rung by rung turns this into a review of exceptions. Tools differ here; see the software comparison.
  • Check data and configuration, not only logic: setpoints, recipes, safety parameters, I/O configuration and firmware.
  • Test the machine, including safety functions, before production runs on it.

Vendors in this market increasingly frame their products around recovery. AMDT reports that thyssenkrupp Presta says “restarts that used to take several hours can now be realised in just a few minutes” (AMDT case study), and Copia now describes itself as a “first responder” for OT incident recovery (TNW). Both are vendor claims; the point for a buyer is that recovery speed is now something to test in a trial. Our octoplant disaster recovery page looks at one approach in detail.

Rehearsing the runbook

A runbook that has never been rehearsed is a document, not a capability. The cheapest rehearsal is a tabletop exercise: operations, controls, maintenance, IT and the incident response lead in one room for half a day, walking through a scenario step by step.

Useful questions to put to the room:

  • The engineering workstations are encrypted. Where is the clean image, and who can build from it?
  • Which line restarts first, and who decided that?
  • The last clean backup of the packaging PLC is five weeks old. What changed since, and how would we know?
  • Who is allowed to download to a safety controller, and are they reachable tonight?
  • Who tells customers, and when?

Then run one technical drill a year: rebuild an engineering workstation from offline media and restore one non-critical controller from the offline copy, timing each step. The time it takes is the honest estimate of your recovery, and the gaps it finds are the plan for the following year.

Limits and open questions

  • Safety systems need their own restore and revalidation procedure, with the people qualified to sign them off.
  • Firmware tampering is rare but possible in targeted attacks; restoring a program does not restore firmware. Know how you would reflash a module.
  • Spares. A controller that cannot be trusted may need replacing. Hold or source spares at the right firmware.
  • Regulatory reporting obligations differ by sector and country. Know yours before the event.

How to evaluate your readiness

  • Is there an offline, immutable copy of every critical program, verified in the last quarter?
  • Could you build a clean engineering workstation without the network, today?
  • Do you hold installers and licences for every engineering software release in use?
  • Can a tool compare each running controller with a dated known-good version?
  • Is criticality recorded so the restore order is decided in advance?
  • Has the runbook been rehearsed with operations, controls and IT together?

Deeper

If the cost of an outage is the question in front of you, our OEE and downtime improvement work puts a number on it and ranks where recovery time is lost. Back to the automation change management hub.

Frequently asked questions

Does ransomware infect PLCs?

Most ransomware seen in manufacturing encrypts Windows systems: SCADA and historian servers, HMI PCs and engineering workstations. Controllers usually keep running. The plant still stops, because operators lose visibility and engineers lose the tools and files needed to change anything. Controllers can also be altered deliberately in a targeted attack, so their programs must be verified before production restarts.

What is the first step in PLC ransomware recovery?

Contain the incident and put the process in a known safe state. That usually means isolating the OT network from IT and the internet, deciding with operations whether lines keep running or stop in a controlled way, and preserving evidence for the incident responders. Restoring anything before the attacker's access is removed risks re-infection.

How do you know a restored PLC program is correct?

Compare it with the last known-good version held offline, before the incident window. Upload what is running in each controller, compare it with that version, and investigate every difference. Then test the machine through its normal sequence and safety functions before releasing it to production. A compare tool that reads the device turns this from days of manual review into hours.

Where should PLC backups be stored to survive ransomware?

At least one copy must be offline or immutable, so that an attacker with administrator rights on the network cannot encrypt or delete it. Keep it with the installers and licence files for the engineering software, and the images needed to rebuild a clean engineering workstation. Test that the offline copy can actually be restored, not only that it exists.

How long does recovery from OT ransomware take?

It depends almost entirely on preparation. A plant with offline backups, clean workstation images and a rehearsed runbook can often restore visibility in days. One that must rebuild servers by hand and reconstruct programs can take weeks. AMDT, for example, cites thyssenkrupp Presta reducing line restoration from several hours to minutes with automated recovery, a vendor-reported figure.

References and further reading

  1. #StopRansomware Guide · Cybersecurity and Infrastructure Security AgencyOffline, encrypted and tested backups, and golden images of critical systems.
  2. Primary Mitigations to Reduce Cyber Threats to Operational Technology · CISA, FBI, EPA and DOEIncludes keeping the ability to operate manually and testing recovery plans and software backups.
  3. IR 8374, Ransomware Risk Management: A Cybersecurity Framework Profile · National Institute of Standards and Technology
  4. SP 800-82 Rev. 3, Guide to Operational Technology (OT) Security · National Institute of Standards and Technology
  5. thyssenkrupp Presta: From standstill to restart in minutes · AMDTVendor case study; the restoration figure is the vendor's.
  6. GitHub for the factory floor: Copia raises $26M to protect the code that runs industry · The Next Web

← Automation Change Management

Have a project that needs technical ownership?

Send us the situation. If it is not something we should take on, we will say so.