AWWA: Learning from the Oldsmar Attack

In February 2013, the American Water Works Association (AWWA) Water Utility Council started a project to address the lack of useful, step-by-step guidance for protecting water sector process control systems (PCS) from cyber-attacks.^[1]  The AWWA-recommended cybersecurity practices are broken down into fourteen (14) practice categories. The AWWA places significance on assessing and mitigating cyber security risks that could affect computer or other automated systems, including security impacting systems, monitoring practices, and the financial infrastructure of the system.  Utilities may have PCS and enterprise systems that are physically or logically connected or PCS that have access to the Internet.

In early 2018, the US Department of Homeland Security (DHS) and the Federal Bureau of Investigation (FBI) warned that foreign governments are specifically targeting the public utilities and other critical infrastructure sectors as part of a complex intrusion campaign.^[2]  On February 5, 2021, the City of Oldsmar detected an unauthorized person connecting to the PCS attempting to make changes to the chemical mixtures used to treat the water.  Fortunately, a plant operator noticed someone remotely changing the chemical mixture and was able to revert the settings back to normal.  The remote access happened twice, but the first time the plant operator thought a supervisor was remotely accessing the PCS computer.^[3]

GuidePoint Security has performed multiple AWWA Cybersecurity Risk and Resiliency assessments over the past year.  Here is a list of potential risks that were uncovered and can be used as a starting point in assessing and securing your infrastructure, PCS, and enterprise networks.

• Who manages the PCS network? Have you verified the configurations of the computers and network devices? If Internet access is allowed, how is it restricted? Has there been a network segmentation test or external penetration test verifying the security of the network?

• Are policies updated, distributed and known by all personnel? Is there an acceptable use policy? Are personal devices allowed to access enterprise or PCS networks? Do you perform annual Security Awareness training?

• Has your organization performed a risk assessment? Do you have an updated Incident Response Policy, Plan, or Playbook? Have you performed an Incident Response table-top exercise?  Do you back up sensitive data, protect backups, and test the backups for accessibility and operations? Do you have a valid business continuity plan? Has it been tested and validated?

• Are PCS assets managed the same way as enterprise assets are managed? If you use a contractor for managing the PCS network, do you verify configurations or authorize remote access to the PCS network?  Do PCS and enterprise computers start with a hardened baseline using an industry standard?

• If you only provide PCS with Internet access through your firewall, how do you know the PCS network is secured? How are the PCS computers updated? Who has remote access? Is multi-factor authentication used? Are USB devices authorized on PCS or enterprise computers? Is there an anti-virus solution used on PCS computers?

This is only a small list of items that were discussed during the assessments. Using these items as a starting point, your organization can start the process of improving the security of your PCS network and reduce the risk of an incident that could cause contamination, malfunction, outages resulting in illness, or worse. Improving enterprise (non-PCS) security can reduce the risk of delayed responses by first responders and increase the security of personally-identifiable information of your employees and customers.

[1] AWWA Cybersecurity Guidelines 2019

[2] U.S. Department of Homeland Security (DHS), US-CERT, Alert (TA18-074A), Russian Government Cyber Activity

Targeting Energy and Other Critical Infrastructure Sectors, March 15, 2018, revised, March 16, 2018, https://www.us-cert.gov/ncas/alerts/TA18-074A; U.S. DHS, US-CERT, Alert (TA18-106A), Russian State-Sponsored Cyber Actors Targeting Network Infrastructure Devices, April 16, 2018, revised, April 20, 2018, https://www.us-cert.gov/ncas/alerts/TA18-106A.

[3] https://www.tampabay.com/news/pinellas/2021/02/08/someone-tried-to-poison-oldsmars-water-supply-during-hack-sheriff-says/