By Fred Gordy, (CS)²AI Fellow, Director of Cybersecurity at Intelligent Buildings, LLC
October, 2020
Building automation systems have historically been installed with convenience in mind. Isn’t that why facility managers started using a browser instead of having to install a specific application on all the PCs that needed to access the BAS? Lower cost, greater portability, and ease of access.
Web access remotely was the next logical step for both the facility manager and the supporting vendors. Facility managers could check their systems through a web browser, and vendor cost could be reduced because they could access the system remotely through a browser and pre-diagnose or fix the issue before rolling trucks.
In the early days, IT really didn’t have or want to have anything to do with these systems, so BAS vendors took on the responsibility of running cable, installing unmanaged switches, and setting up remote access. Remote access was often accomplished either by using a public IP (a public IP is a globally unique address that can be accessed over the Internet) or forwarding a public IP to a private IP through a standard, internet service provider (ISP) router.
Public IPs were used in personal computers that ran the web service/application that enabled facility managers to view and interact with the BAS remotely. The vendor could also view, control, and reprogram the application through the public IP. Because the vendor could also program remotely, public IPs were extended to devices to facilitate programming. As a result, the system was totally exposed.
At the time, being exposed was okay to both the end user and the vendor because no one was actively seeking out these systems. That’s all changed. Devices are now in the hacker’s crosshairs.
Why has this changed? The simple answer is that devices offer the path of least resistance. Hackers may or may not be looking to compromise or destroy equipment. They may be looking for another way into the company network, and they know that control networks have little or no security and that these networks are not typically monitored for threats or intrusions.
Up until 2009, search engines were not specifically looking for Internet-connected devices. In 2009, Shodan was launched. It was the first search engine dedicated to searching for Internet-connected devices, also known as the IoT (Internet of Things). The intent was to catalog the number of devices (not websites) active on the Internet. Users could search free of charge for specific devices from specific manufacturers.
A byproduct of indexing Internet-connected devices was that now the bad guys could use this tool to find devices and probe for vulnerabilities.
In 2015, Censys was created at the University of Michigan and made available to the public for free. Censys, like Shodan, crawls the Web in search of Internet-connected devices. And like Shodan, both security researchers and hackers can use it.
Censys and Shodan both index and add tags to the devices. Why tags? It makes searching easier. You don’t have to know a query language, just know the tags. If you wanted to find all the building control system devices in the world that Censys has indexed, you would enter “building control” in the search box and in less than a second you would have a list. You could do the same for any of the tags shown. The large list of tags makes searching for devices easy to do and easy to refine.
Censys provides a lot of useful information for both good guys and bad guys, including the system version, the host ID/license, the host name, and the name of the building where the device resides. It also lists the geographic location of the device, manufacturer, OS version, ISP, etc.
After a device has been found, several software applications make compromising a device relatively easy. For example, if a hacker can find a BBMD (BACnet broadcast management device), they can have full command and control without having to crack the username and password. And the tool to do this is free. Anyone can download it from SourceForge.net.
When the first concerns about cybersecurity for control systems were raised, some in the field would ask, “So what if someone turns off the lights?” Today, there is widespread understanding that hackers can cause life safety issues, financial loss, and brand damage to companies.
Let’s unpack of few of these incidents. Most of them could have been prevented.
When the subject of cybersecurity for control systems comes up, the obvious thing that people think of first is loss or damage to equipment. If a generator were to be attacked and destroyed there is a cost of replacing the generator. (Here’s a Department of Homeland video of a staged generator attack.) But who would think that a printer connected to a parking system could cost a company six figures? One organization had an exposed printer and someone printed, “There is a bomb in building.” Nothing was damaged, right? Wrong. The high-rise building had to be evacuated, causing work to stop, yet salaries were still being paid. Emergency personnel were dispatched. And brand damage was inevitable and as of yet not quantified.
Another common situation involves the loss of front-end access. In one case, the front-end application was crippled, causing business cessation for 48 hours. Ransomware can also block front-end access. In 2017, there were numerous ransomware attacks on control systems. These caused stoppages in some cases and investigation in others, but for both the financial impact has not been fully realized due to ongoing review. In all the cases just noted, these attacks could have easily been prevented.
The biggest thing holding most facility managers back from securing their systems is fear of the unknown. How much will this cost? How hard will the change be? How inconvenient will it be? What else will it add to the daily to-do list?
The good news is that there is a lot of low hanging fruit — actions that will incur minimal cost. The first thing a facility manager can do is find out if the system has a public IP. If so remove it and get it behind a firewall. If remote access is needed, there are several low-cost remote access solutions. Just by doing that, the control system is now hidden from search engines like Shodan and Censys, making the system much harder to find.
As mentioned earlier, ransomware attacks are up — and relatively speaking, up significantly — on control systems. Ransomware has several delivery methods, but the most common is through email or social media. By restricting direct access to front-end, the facility manager will significantly reduce the probability of a ransomware attack. The front-end needs to be moved off the engineer’s desk, even if the engineer’s office is locked most of the time, and placed into a locked closet with the keyboard, mouse, and monitor removed.
It is essential to make sure every user, including your vendor, has a unique username and to make sure no one shares user credentials with others. This also means establishing access policies for both employees and vendors. The policy would require at a minimum the following:
· Least Privileges – Users of the system should have only the necessary rights to perform their duties.
· Admin – Most building systems only need 2 administrators.
· Vendor users should be disabled and only enabled during the time the vendor is performing work request by your company.
· Employee Access Cessation – If an employee is not no longer employed or no longer has need to access the system, they must be removed the last day of employment or last day of needed access
· Vendor Access Cessation – The vendor is required to inform FM with 24 hours that employee is no longer employed by the vendor or longer requires access to the system.
· Assess – Determine specific risk for by equipment type and function. Document the primary risk and recovery plan when an event occurs.
· Inventory – Make sure all the devices that are connected are supposed to be connected and how they are connected. Remember the printer example? Once that happened the company thought the wireless access had been disconnected from the printer. When we performed our audit is was still connected. Inventory is not a “one-and-done”. Establish a periodical inventory review and change management process.
There are many more things that can be done at low or medium cost to secure a facility. The list above will get you pointed in the right directions. The National Institute of Standards and Technology has established an elegant and simple principal in their cybersecurity framework that gives a high-level approach to securing systems:
• Identify: Learn to manage cybersecurity risks.
• Protect: Limit or contain the impact of cybersecurity events.
• Detect: Identify when a cybersecurity event occurs.
• Respond: Take action when a cybersecurity event is detected.
• Recover: Maintain plans for resilience and restore capabilities or services impaired due to a cybersecurity event.
The bottom line is that the days when control systems were not on any hacker’s radar are gone. Control systems are seen as easy targets. As a result, IT departments are beginning to be more involved in helping secure control systems, and that’s great, but it’s not enough. When a control system is being assessed, and a report/remediation plan is being created, three risks must be factored in. The obvious is the cyber risk.
A second risk is the financial/business risk. This one has components that are somewhat easy to quantify, but it is always tough to assign a dollar value to brand damage.
The risk that is typically not thought of is the operational risk if IT implements its own cybersecurity measures on the control system. Operational risks are anything that could impede or disrupt system performance. Some IT threat-monitoring or corrective technologies can literally cause control systems to fail or lockup or at the very least disrupt communication, which in turn could prevent sequences of operations from functioning. The remediation report must include what IT can and cannot do.
In addition to being easily found now with IoT/device search engines, common best practices that IT has used for years are not utilized in the OT (operating technology) world. Audits of systems in the United States and Canada — including commercial real estate, healthcare facilities, government buildings, retail facilities, public venues, and military bases — show in almost every case that the organizations have virtually the same vulnerabilities and bad practices. Here are some examples:
· The vendor controls user administration.
· The vendor has 24/7 access to the system.
· The vendor controls remote access and administers it.
· The vendor maintains the backups, and the facility
· manager has limited access.
· The front-end uses a public IP (some of the devices
· also have a public IP).
· Shared or common username for both the facility
· manager and vendors.
· Incomplete or unknown inventory of connected devices.
· No change management.
· No disaster recovery plan.
· Backups stored locally on the front end.
· The front-end is exposed where anyone can use it.
· The front-end is used to surf the Web, check email,
· and visit social media sites.
· The front-end operating system and some devices
· are past end of life.
· Patching is not up to date.
· No antivirus software is used.
· There is no access management policy.
· No cyber awareness training is utilized.
In more cases than not, audits find all those problems with building control systems, and more.
Facility managers have many responsibilities, and few want to add cybersecurity to the list. They already have their hands full with running the facility. But we are in a new era where everything is connected, which means it is time to stretch skillsets. Done correctly cybersecurity will not add an unmanageable amount of work to the load, and the facility will be safer.
Comentarios