Patch management is vital for effective cyber security

Protecting control systems against penetration requires an effective patch management process

Last month, my colleague Joe Doetzl blogged about six questions to test your preparedness for a cyber incident. After attending an educational session on penetration testing last week in Germany, I would like to offer some additional thoughts.

Cyber incidents normally occur when someone penetrates your system and installs malicious software on it. The way they get in is through exploits available within the systems you’re operating. The older the system, the more exploits exist. This means you can significantly reduce the risk of getting hacked by having new software running on all your systems (i.e., not only on your servers and workstations, but also on your switches, routers, firewalls and printers).

However, in the control system environment in which I work, we have long lived according to the rule, “Never change a running system.” As a result, there are still some old legacy systems securing the supervision and control of large networks supplying us with many of the necessities we have grown used to having: electricity, water, gas, etc.

If these systems were compromised, resulting in a major blackout, the results would be nearly unimaginable. I won’t go into the details here, but if you’d like to get a picture of what that world would look like, I recommend reading “Blackout” by Marc Elsberg, an Austrian journalist who did some excellent research on the topic.

A world without these necessities is a world we don’t want to experience — which is why everyone should take care to see that control systems are well protected against penetration. That does not mean that you only erect some firewalls around the systems and apply the right password policy. You must also have a good patch management process.

Unfortunately, a computer in a control system is different than your computer at home, which receives updates automatically over the Internet and requires only a simple reboot to install. In a 24/7 environment we want to omit reboots and certainly will not be directly connected to the Internet.

For that reason, many network control vendors, as well as some consultants who specialize in this matter, offer services which can help you to keep the lights on. ABB is the No. 1 network control vendor offering these services.

The process for patch management is quite clear: first you have to select those patches which are relevant (i) to your system and (ii) for security. In control systems computers and other devices are hardened (i.e., only the software which is absolutely necessary to run the control system is installed). In other words, no “games” should be installed on the computers, so there should be no patching required for them.

In the next step, you install the selected patches in a test environment and provide evidence that the system will run after the patching process. This is where the large installed base and product approach of an established company pays off. These tests have to be performed with many patches and are quite intensive. Just by testing our baseline system, ABB has a high coverage for many customers, which significantly reduces the average cost per customer. After the baseline testing, the project specifics have to be tested. This mostly has to be done by hand, so here it pays for your system to be closer to baseline.

When you know it is safe to install the patches at customers´ sites, you transfer them via secure mechanisms and agree on a patch schedule. You cannot switch off the entire system, so you have to use redundancies to patch one device after another.

I would love to hear your experiences and thoughts on this topic. Please share them with me in the comments below.