Search Results

By Jay Gignac, Head of Global Sales & Marketing, Framatome Cybersecurity, Cyberwatch & Foxguard February 29, 2024 Ensuring smooth performance amidst various constraints is imperative in the complex landscape of industrial operations. Industrial plants often face operational challenges such as limited shutdown windows, critical process availability requirements, and demanding safety considerations. In such environments, implementing effective remediation measures becomes crucial to maintain operational integrity while addressing security vulnerabilities. Here, we dive into the strategic approach to address operational constraints and propose remediation actions. Many industrial plants operate with annual shutdowns or planned maintenance slots. These limited windows restrict the opportunity for implementing security updates or modifications to systems. Certain assets within industrial processes must operate continuously without interruption and the processes must be available at all times. Halting these processes for maintenance or security updates is not feasible due to the potential impact on production and operations. Finally, industrial systems often play integral roles in safety mechanisms. Any modifications to these systems should be approached with extreme caution to prevent compromising safety protocols or regulatory compliance. In some sensitive Industries like Nuclear or Pharma, any change in the system requires a new qualification phase which can be long and costly. There are some pre-implementation considerations to take. For example, it is critical to maintain up-to-date backups of critical systems and data to facilitate recovery in case of unforeseen complications or failures during remediation activities. It is important to develop documents and test comprehensive restart or fallback procedures, outline steps to revert changes or mitigate adverse effects if remediation measures result in disruptions or unforeseen consequences.  Make sure to verify the authenticity and integrity of patches to mitigate the risk of introducing malicious code into the industrial environment. With that in mind, acquiring patches and updates from a single trusted source and through secure channel could be beneficial and time saving. Then thoroughly validate patches and remediation measures in a controlled environment before deployment to production systems. Testing should encompass compatibility, functionality, and security impact assessments. Make sure to have your vendors approved and warranties contracted. In the specific case of patch management, after patches have been thoroughly tested and approved, the focus shifts to effectively rolling out these updates across the organization's network. Centralized patch management tools play a crucial role here, enabling administrators to orchestrate the deployment process efficiently. These tools provide a centralized dashboard where administrators can schedule deployment times, target specific groups of devices or systems, and monitor the progress of patch installations in real-time. Automated deployment mechanisms streamline the process, ensuring that patches are applied promptly while minimizing disruptions to normal operations. Additionally, organizations may employ techniques such as phased deployments, where patches are rolled out gradually to different subsets of devices or systems, allowing administrators to monitor for any unexpected issues and adjust deployment strategies as needed. In environments with air gap equipment, specialized methods such as offline patch distribution may be required, where updates are manually transferred to isolated systems via physical media or dedicated network segments. Throughout the deployment phase, careful coordination and communication are essential to ensure that patches are applied effectively across all endpoints, reducing the organization's exposure to security vulnerabilities. To summarize, navigating operational constraints in industrial environments requires a meticulous balance between security and continuity. By adopting proactive remediation measures such as patching, hardening, and asset isolation, industrial plants can fortify their defenses against evolving cyber threats while ensuring uninterrupted operations. However, a cautious and methodical approach to implementation, including secure patch acquisition, pre-implementation considerations, and robust fallback procedures, is indispensable for safeguarding both security and operational integrity in industrial settings. Learn more about:

By Jay Gignac, Head of Global Sales & Marketing, Framatome Cybersecurity, Cyberwatch & Foxguard February 22, 2024 Maintaining an accurate asset inventory in cybersecurity, particularly within industrial and critical sectors is a fundamental baseline. Despite the challenges posed by diverse and evolving assets, consolidating data from various sources, and contextualizing it is crucial for effective risk management. By implementing robust asset management strategies, organizations can enhance their cybersecurity posture, ensuring operational continuity and safeguarding critical infrastructure. Resource constraints and the need to prevent operational disruptions necessitate prioritizing corrective actions in cybersecurity. Not all vulnerabilities can be addressed simultaneously, making it essential to prioritize based on risk severity. By mitigating the most critical risk first, organizations can allocate resources effectively and minimize potential impacts. The complexity of industrial systems - the range of vendors, the fact that they are often geographically spread, and the obsolescence of some equipment - sometimes lacking cybersecurity capabilities poses significant challenges to prioritization. Additionally, regulatory requirements further add compliance constraints to prioritization efforts. A risk-based prioritization methodology is essential for effective cybersecurity management. When utilizing information gathered from asset inventories and vulnerability assessments, organizations can conduct risk assessments to identify critical systems and evaluate the likelihood of exploitation. This method correlates severity, exploitability, and criticality obtained through context. Following this methodology, designing a remediation plan involves determining what actions to take, in what order, and considering operational constraints. This ensures that resources are allocated efficiently, focusing on mitigating the most significant risk first. Therefore, adopting a risk-based prioritization approach is crucial for effectively mitigating cyber risks and ensuring the resilience of critical infrastructure. The next step is to implement a remediation plan, addressing vulnerabilities and threats based on their risk severity and operational impact. Learn more about:

By Jay Gignac, Head of Global Sales & Marketing, Framatome Cybersecurity, Cyberwatch & Foxguard January 23, 2024 In the realm of Operational Technology (OT), the identification and management of vulnerabilities and patches are pivotal to maintaining a robust defense against evolving cyber threats. This critical process goes beyond mere detection; it's about understanding the intricate details and implications of each vulnerability. The sophistication in identifying vulnerabilities lies in the ability to consolidate data from multiple, trusted industry sources. This aggregation provides a rich tapestry of information, painting a complete picture of each vulnerability’s potential impact and the urgency of its remediation. Evaluating a vulnerability’s exploitability and maturity is a nuanced exercise. It's not just about identifying the weaknesses but also understanding the likelihood of these vulnerabilities being exploited in the wild. This understanding is crucial in prioritizing which vulnerabilities need immediate action, such as the application of a patch, and which ones may need continuous monitoring for potential future risks. In the context of OT, patch management is a critical yet complex task. Each patch deployment must be carefully considered, balancing the need to mitigate vulnerabilities against potential operational disruptions. The key lies in understanding the specific operational context and the criticality of the systems involved. A rushed patch might solve one problem but create several others in an OT environment where system stability and availability are paramount. Moreover, the process of identifying vulnerabilities and managing patches must be continuous and dynamic, adapting to new information and evolving threats. It involves not only technical insight but also strategic foresight, ensuring that the defenses are not just reactive but also proactive, preparing for future vulnerabilities and threats. The role of cybersecurity in OT environments extends beyond protection; it's about enabling secure and uninterrupted operations. Organizations must therefore prioritize their efforts around vulnerability management, engaging with platforms and solutions that provide comprehensive data analysis and contextual insights. This approach ensures not only a more secure environment but also one that is resilient and prepared for the challenges of a connected industrial world. In conclusion, the journey towards a secure OT environment is ongoing and ever-evolving. Emphasizing the identification of vulnerabilities and the strategic management of patches is crucial in this journey. It's about building a cybersecurity posture that's not only robust today but also adaptable for the uncertainties of tomorrow. Learn more about:

By Gregory Dupuis, Global Head of Marketing and EU Sales Team Leader at Framatome Cybersecurity (IBCY) January 11, 2024 In the ever-evolving world of cybersecurity, Chief Information Security Officers and Operational Technology cybersecurity specialists face unique challenges. Among these, maintaining an up-to-date asset inventory stands out as a fundamental pillar for ensuring a robust security posture. This article delves into why an accurate and current asset inventory is critical, particularly in industrial or critical environments, and how organizations can effectively overcome the associated challenges of managing it. Managing an asset inventory in OT environments can be complex due to the diversity and constant evolution of assets. This task becomes even more challenging in industrial and critical sectors, with often siloed systems and a mix of old and new technologies. The first step involves acknowledging and addressing these unique challenges. An effective approach to maintaining an up-to-date asset inventory involves consolidating existing data. This means leveraging pre-installed tools that gather data on both IT and OT assets. For instance, using industrial network probes for mapping can provide significant visibility. However, another layer of information can be obtained from more traditional tools such as antivirus software, Endpoint Detection and Response (EDR) systems, firewalls, and network switches. Merely gathering data is not enough; its contextualization is imperative. Without this, it’s impossible to add value in threat management. A good asset mapping should include not just a complete view but also a clear context, covering the location, functionality, and criticality of each asset. The software used to maintain the asset mapping must be capable of interconnecting with various tools to extract and consolidate data. It’s also vital that it offers functionalities for classifying assets into different categories. This classification should align with the company’s risk management policy, allowing for a more precise and targeted risk analysis. Maintaining an up-to-date asset inventory is fundamental to a strong cybersecurity posture, especially in OT environments. By addressing the unique challenges of these environments, consolidating and contextualizing data, and using appropriate tools for asset classification and analysis, organizations can greatly enhance their ability to manage risks and respond effectively to threats. Investing in these processes is not just a precautionary measure; it’s a strategic necessity to ensure operational continuity and protect critical infrastructure. Learn more about:

By Brent Huston, MicroSolved, Inc., (CS)²AI Fellow December 21, 2023 Introduction: OpenPLC is an open-source Programmable Logic Controller (PLC) for industrial applications. Installing OpenPLC on a Raspberry Pi can provide a low-cost and compact PLC system that can be used to control and monitor industrial processes. The OpenPLC runtime has a built-in web server, allowing program upload and configuration via a web interface. Installation Checklist: 1. Ensure your Raspberry Pi is running a recent version of the Raspbian operating system. 2. Make sure git is installed on your Raspberry Pi. If it's not installed, run the command: `sudo apt-get install git`. 3. Clone the OpenPLC repository using git with the command: `git clone https://github.com/thiagoralves/OpenPLC_v3.git`. 4. Navigate to the cloned directory: `cd OpenPLC_v3`. 5. Run the installation script for Raspberry Pi: `./install.sh rpi`. 6. Reboot your device after the installation is complete. 7. Once rebooted, OpenPLC will start automatically. 8. Access the OpenPLC web server by typing the IP address of your Raspberry Pi at port 8080 in your web browser. 9. Login using the default credentials (username: openplc, password: openplc) and change the default username and password under the Users menu. 10. Under the "Hardware" section, select the appropriate Raspberry Pi driver from the popup menu and save changes. 11. For pin mapping and creating your first project, refer to the official OpenPLC project page. Use Cases: Prototype Development and Testing: Raspberry Pi combined with OpenPLC offers a cost-effective platform for developing and testing new Industrial Control System (ICS) protocols and applications. This setup allows researchers and developers to simulate real-world ICS environments and test their prototypes under various scenarios, enhancing the robustness of new technologies before they are deployed in actual industrial settings. Educational and Training Tool: The Raspberry Pi and OpenPLC can be utilized as an educational tool for training personnel in the fundamentals of ICS. This setup provides a hands-on experience in a controlled environment, allowing trainees to understand the workings of PLC systems and the intricacies of industrial automation without the risk of impacting real-world systems. Cybersecurity Research and Testing: With the increasing importance of cybersecurity in industrial environments, Raspberry Pi with OpenPLC serves as an excellent platform for cybersecurity research. Researchers can use this setup to simulate ICS environments to study the impact of various cyber threats, develop mitigation strategies, and test the effectiveness of security solutions in a safe and controlled setting. Remote Monitoring and Control Experiments: Raspberry Pi, when integrated with OpenPLC, can be used for remote monitoring and control experiments. This application is particularly useful for research teams looking to develop and test new methods of remote operation, data acquisition, and process control in industrial systems, offering a practical and scalable approach to innovation in remote ICS management. Cost-Effective ICS Solution for Small Scale Industries: For small scale industrial setups or start-ups, the combination of Raspberry Pi and OpenPLC provides a cost-effective solution for implementing basic ICS functionalities. This setup allows smaller operations to automate processes and integrate control systems without incurring the high costs associated with traditional ICS infrastructure. These use cases highlight the potential of OpenPLC on a Raspberry Pi in providing a low-cost and effective platform for ICS security research and education, enabling asset owners and security teams to understand better, secure, and manage their industrial control systems.

By Dan Clark, Director of OT Cybersecurity Architecture for Verve Industrial Solutions July 11, 2023 We hosted a (CS)²AI Online™ Seminar on June 21, 2023 that focused on Applying Network Segmentation to Secure OT Environments. The event was sponsored and led by subject matter experts from our Strategic Alliance Partner, Verve Industrial Solutions. Here is a bit about the event: As threats continue to rise and target industrial organizations, one of CISA’s consistent recommendations for effective OT security is network segmentation. But between unique challenges in industrial environments, aligning IT & OT teams, and understanding where to start to secure critical networks, many organizations struggle to put it into practice. How should IT and OT work together? How do we segment with the least impact on operational uptime? In this webinar, Verve CEO, John Livingston, and Director of OT Cybersecurity Architecture, Dan Clark, will share over 30 years of OT networking experience and discuss how to: • Implement network segmentation in OT systems • Effectively bridge IT & OT systems security • Achieve effective visibility of segmentation • Gain buy-in from your team and investors Do you want to have access to more content like this? Register for our next (CS)²AI Online™ seminar or symposium here, or consider becoming a member of (CS)²AI to access our entire library of recorded sessions. The Q&A portion below represents selected overflow questions and commentary from the event, and has been answered in detail by Dan Clark. ****************************** QUESTION: What’s your opinion on SDN vs VLAN? ANSWER: We are generally brought in to separate an IT (Corporate, Business) network from an OT (Process Control, Manufacturing) network. Typically, the networking equipment between the IT and OT networks are different and some of the routing, switching and firewalling equipment can be quite old, specially in the OT environment. It is not unusual to find switches and routers over ten years old. As I understand SDN, this is typically used in cloud computing for Internet Service Providers and is used to improve network performance and monitoring. The OT networks that we have been asked to segment have not matured to that point yet. However, overtime, I could see that SDN could be used for the OT environment of the future if SDN is still viable. QUESTION: How many engineer hours does it take to implement a new protocol-specific deep-packet-inspection engine? ANSWER: I don't know. How do you define implement (proof of concept, tested on any device that uses this protocol, tested on every device)? Does the protocol already exist and is it well defined? Does the engineer have all of the equipment that can be used to test the implementation of the deep-packet-inspection engine? Does the vendor of the device used to do this inspection support new protocols? Which vendor or vendors does this implementation need to support? QUESTION: Were you aware that by using a Tripwire-like firewall with a protocol-specific deep-packet inspection engine, we were able to mask existing vulnerabilities in devices on both the human interface side and the device-interface side? By doing this there were many patches which did not need to be applied. ANSWER: Many of the new NGFWs (Cisco, Fortigate, Palo Alto, e.g.) have the ability to look into the packets for ICS protocoIs (MODBUS, DNP3, e.g.) to make decisions on whether or not the commands meet the requirements for the protocol. Decisions can then be made to on allowing reads, writes, etc. We have had mostly successes when implementing but a few failures as well, but they are really nice when they work. The failures sometimes get fixed with patches and firmware upgrades as well. However, not everyone wants to implement these features. It may sound shocking, but this is all too common. QUESTION: Everyone is aware of DARPA's definition of the four pillars of security: Physical (spatial), Logical, Temporal, and Cryptographic Threats. What are you doing about the fifth pillar, Perceptive Threats? ANSWER: I was not aware of "DARPA's ... four pillars of security", and could never find on the DARPA website ... but I didn't make an exhaustive search. However, from the context of your question we do deal with physical, logical, temporal and cryptographic threats in various modes with our clients … network segmentation, training personnel, physical room and cabinet security, and others. Other than the training that we do for personnel and specifying what and details of the segmentation efforts like configuring firewalls, routers, and switches which directly addresses real not perceived threats, I am not aware of other any other methods that we use to address perceived threats. For what it is worth, our clients don't always follow all of our recommendations either. There are many practical business and operational requirements that impact what is actually implemented during a segmentation project and why. Many industries do not have governmental regulations for cybersecurity; so, compliance is optional and decisions are made on a risk/reward basis. QUESTION: Did you know there is a way to wrap zero trust around something like DOS or Windows NT? It is called Virtualization. My PhD advisor wrote the book on Virtual Machines. I helped with the book. If you would like to talk to me about it, contact me: https://cs.wisc.edu/~bezenek. The TRUST in this system comes from the newer technology VM running on a contemporary processor with a root of trust built into it. ANSWER: We are very familiar with Virtualization. OT people are very reluctant to change out systems that work and they know. Mostly they don't want to upgrade because it costs them money in production downtime, equipment cost, and training for production personnel. This is why we see pieces of a production system still running DOS6, Windows NT, Windows XP, Windows Server 2003, and many others that are obsolete. Many, but not all older systems, have upgrade paths to newer platforms including VMs. When moving an existing older system to a Virtual Machine the costs mentioned earlier are likely even higher. For VMs, you must have a host hardware and software which costs, development costs, and testing costs. After that you still have to add the other costs as well. We have also had issues with drivers on some of this old hardware specially systems that use serial communications. So typically, we don't have Virtualization as a practical option. By the way, last year I was involved with replacing a 1990's vintage Sun Unix computer with a Moxa Industrial PC. Serial communication was involved so we ported the software from Unix to Linux and used the serial port on that Moxa computer for the communication ... so it is doable. QUESTION: When discussing VLAN hopping, is hopping between VLANS on the same switch (intra-switch hopping) essentially the same vulnerability-wise as inter-switch VLAN hopping with homogeneous switches? Note: Define homogeneous switches as being the same model made by the same manufacturer running the same version of the management software on the same OS with all the same patches to all software. ANSWER: As I understand your question, yes. However, VLAN hopping (inter- or intra-) primarily occurs as a result of poor configuration and can be mitigated with a minimal amount of quality configuration standards … not using VLAN 1, defining a VLAN for each access port, filtering trunk ports based on VLAN IDs, and SHUTDOWN each unused port. QUESTION: DOS 6 that's hard-core (but probably simple too simple) ANSWER: There is nothing simple about making older systems work, because if it was, it would have already been replaced. We have had issues with motherboards dying when rebooting an old computer and getting the system back up and running is difficult, and it isn't always possible. However, there hasn't been a system so far, that we couldn't get running again ... but it never is easy. Sometimes we have to upgrade a system. QUESTION: Is it best to prevent SVI (switch virtual interface) completely or is there use cases for inter-VLAN comms....? ANSWER: Absolutely there are cases for inter-VLAN communication. Segmentation wouldn't be very effective we couldn't route between the VLANs. Ideally, there is a firewall device to negotiate when and how this communication is allowed. QUESTION: How do you get your customers to update their OT equipment to the latest version of firmware? Many of our customers are at very old and vulnerable releases of firmware. ANSWER: Mostly you don't until they have some type of cybersecurity event. It is easy to pick on the OT guys, but most of their issues are related to production and cost. They don't want to upgrade because it costs them money in production downtime, equipment cost, and training production personnel. This is why we see pieces of a production system still running DOS6 or Windows NT. One of the other questions suggested moving these old systems to a Virtual Machine. That still costs money for equipment, development costs, and testing costs. After that you still have to add the other costs as well. We have also had issues with drivers on some of this old hardware specially systems that use serial communications. QUESTION: Would an additional advantage in segmenting network be able to more readily isolate the targeted network if/when there's an impacted attack? ANSWER: Yes. QUESTION: How can segmentation effectively be implemented when OT environment is connected to cloud? ANSWER: Directly connecting from an OT network to the "cloud" has its challenges, but we do this in some manner in all segmentation projects. The key is to only make those connections that are required, limit the number of TCP/UDP ports being used, authenticate the traffic and do IDS/IPS filtering where possible. The reality for OT systems and networks requires that the OT(Manufacturing) network have access to the IT (Business) network. The Business sells something and the Manufacturing has to build what was sold. Without this connection, the entire Business would not likely exist. QUESTION: What is the best practice in segmentation when it comes to deploying Managed SW Vlans for different IT/ OT environment? ANSWER: The best practice would be to have IT manage the IT network and the OT manage the OT network and provide visibility between the networks to both management groups. Whenever this is not feasible to have two different management groups, use a single management group but still give visibility to both groups. Significant problems arise when with lack of transparency. QUESTION: Which are the main or widely used network segmentation techniques used (for example: VLAN ID tag was mentioned). Thanks ANSWER: I presented in the webinar the techniques that we typically use for segmentation. However, we are mostly led by what our clients desire. Some clients are looking for assistance on the segmentation how, but most want support in implementation of what they know they want. So for larger organizations that have 10, 20, 50, 100, or more sites, they may do a proof of concept for a few sites, then we are hired to implement their existing solution to the remaining sites. QUESTION: What percentage of environments that Verve sees follow the "Company Network A" model?(estimated are okay) ANSWER: I don't have an actual number. This percentage keeps changing as we do more segmentation projects, but I will throw out 10% as an estimate for the systems that we see in the industrial space that the IT/OT is combined into a single network without a firewall. However, most of the time, there is least some access control rules on the incoming router which typically include multiple VLANS so there is some protection. Nevertheless, there is no physical separation between IT and OT. I really want to say that I am exaggerating, but unfortunately I am not. Hopefully, others doing this work in the OT space see something better. QUESTION: Leasing from telecom carriers for ICS/OT? This is nearly the biggest achilles heal of anything mentioned this far. An owner can secure assets to n-th degree, but has very limited or no control over the infrastructure on which the information is being carried by a telecom carrier from a system configuration, physical protection and cyber-security aspect. Can a user control the telecoms patching procedures? Haven’t seen it other than penalties for not meeting uptime guarantees. ANSWER: I have not seen a situation where a user can control an ISP. Many of our clients are in remote areas and getting access to any internet connection can be a challenge. QUESTION: I’m from a newer world of things, and while being able to see the risk management and monitoring side of things; how would serverless cloud hosting procedures withstand the long tail of vulnerabilities? ANSWER: Directly connecting from an OT network to the "cloud" has its challenges, but we do this in some manner in all segmentation projects. The key is to only make those connections that are required, limit the number of TCP/UDP ports being used, authenticate the traffic and do IDS/IPS filtering where possible. The reality for OT systems and networks requires that the OT(Manufacturing) network have access to the IT (Business) network. The Business sells something and the Manufacturing has to build what was sold. Without this connection, the entire Business would not likely exist. QUESTION: In a situation like where you plugged in that cable, how did you resolve it? Was there like a spare system that was used to replace it? How did this downtime impact the company? Was there a need to update the OS with another system? ANSWER: The example I gave was during a planned outage. So, when the server failed, it did not impact operations at all. This is one of the reasons we are really uncomfortable doing implementation while the site is making product, or what we deem as "hot". If we were not already in an outage situation, the plant would have stopped making product. So, no downtime. Also, we had a about 40 hours before we were to bring this site back up to operation. It was also on Saturday, so we did not have access to many plant personnel. Many, but not all, process control systems have some form of plan to resolve equipment failures... backup or spare equipment. This particular site had neither. The first reaction to these situations is to panic, but don't, unless you are doing this hot. We asked if they had a spare computer for this process and the answer was no. This was a 2015 Dell computer running Windows XP Server. There was a BestBuy about 45 minutes away from the site. We knew that we wouldn't be able to get a computer to run Windows XP, but we "hoped" that we could get a new computer and run a newer version, Windows 10/11, version of the software. We are OT people so we know about many of the applications that run in the OT space. We also know about PLCs, DCSs, SCADA Systems, and many more. Luckily, I had multiple people that I could call for help. So, we sent one of our guys doing this implementation to Best Buy. We had a site support guy that thought they might have a spare XP machine at a sister facility about 20 miles away, so we sent him to check that option out. We did try to fix the computer, hoping that it was just a video card, but that failed. Unbelievably about three hours later, we got a spare computer from the sister facility that didn't throw that old computer away, swapped out the hard drive for the machine that failed and magically it worked. Success. We got lucky. It did not impact production because we did this during an outage. Most of our client's personnel didn't know how close we were to causing an outage. QUESTION: With the new movement to have IT and OT feeds on a single pane of glass, are there any differences in segmentation rules you are presenting? ANSWER: For sure the rules will have to change to allow IT (Business) network devices to communicate with OT devices. Many times this can be a single feed and technically you could put a data diode in the network for this feed as well to protect the OT network from the IT network. We have used firewall rules and data diodes. It just depends on what the client needs. QUESTION: Is the recommendation for complete physical segregation with a single point of contact, or a logically segregated OT and Enterprise network? ANSWER: We would suggest physical segregation of the IT and OT network zones and logical separation among the subnetworks in each zone. The only issue is budget. We have done both. QUESTION: How to achieve segmentation of assets which are inside the target network ANSWER: Sub-segmentation of the target network. We are working on another presentation to discuss the details and how to subsegment a process control network. QUESTION: At what point the process identifies vulnerability of assets in consideration & its possible mitigation? ANSWER: If we have a client that has a flat IT (business) network, we would want to start with segmenting and separating the OT (process control, manufacturing) network for the existing IT (business) network. Once separated, we would discover the assets for both networks using our Verve Software, identify asset vulnerabilities, and then use our software to mitigate those vulnerabilities. We also periodically do a survey at the beginning to determine the extent of existing segmentation and the types of IT/OT assets that exists. This survey can sometimes identify specific issues that may be mitigate prior to other work. QUESTION: What are the best practices in segmenting SIS from BPCS? ANSWER: Add another segment to the network for SIS and separate this network from the IT (business) and BPCS networks. I have seen fire systems isolated (as islands) from the network. QUESTION: Could you speak a little bit about the adoption of Zero Trust in the field of ICS/OT? Have you seen any adoption? and could you give us an example of full Zero Trust adoption in OT/ICS? ANSWER: We have seen some adoption of Zero Trust in ICS/OT, but so far it is not typical. I personally have not seen an example of full Zero Trust adoption. QUESTION: When it comes to endpoint protection, such as HMI and Engineering workstations, have you observed any technologies used other than antivirus and application whitelisting? Have you seen adoption to technologies like EDR (Endpoint Detection & Response) or orchestration? ANSWER: Mostly no, but we have seen some adoption of other technologies. QUESTION: What kind of tools you use for discovery? Passive or active? Nmap or Nazomi ? ANSWER: It depends upon the project and what is available, but we have used multiple tools for discovery. We have used our Verve product, which uses both and active and passive functions. When not directly using our product, we will query routers, switches and firewalls (active) through SSH to get configurations, status, mac addresses, routing information, interfaces, hit counts, etc. Once we get that information, we analyze that data to find process control equipment that lives on the IT network. Then we take that information and track it down once at the site for physical discovery. We also ask questions of site personnel to confirm information we find and get additional information that is not readily available through data analysis. QUESTION: How does segmentation give you and ROI? ANSWER: Segmentation helps prevent bad actors from getting into your network. Segmentation minimizes the ability of bad actors pivoting with the network to reduce damage from an intrusion. All of the "returns" are cost avoidances through process downtime and ransomware or protection of intellectual property. Our clients understand clearly the cost of process downtime and typically can quote a $/hour number. The client has to evaluate these "benefits" relative to the costs of segmentation. QUESTION: Statistically. using different vendors for the same type of device increases the likelihood of an exploitable vulnerability. Also you need to stay aware of more security announcements and staff must know more systems (increasing the likelihood of misconfiguration).Compare this to using just one vendor. ANSWER: I am not sure what specific statistics that you are referring to, but you make a very valid point. However, if IT manages the set of firewalls separating the IT (Business) systems from the DMZ and OT manages the set of firewalls separating the OT (Process Control, Manufacturing) systems then their should be no statistical difference from IT managing both IT and OT firewalls. Also, by separating management of the different zones (IT/OT) and giving visibility to the other management group, this gives the organization an additional set of checks and balances. QUESTION: What is your recommendation on implementing network segmentation on legacy systems? ANSWER: So far, we have done this on every segmentation project that we have been involved with, so we recommend it. Ultimately, network segmentation costs money and rarely, if ever, there is an unlimited budget for doing this work. We will normally put pieces of a legacy system on a VLAN by itself, or subsegment that system into multiple VLANs depending upon client requirements. QUESTION: What is your recommendation on implementing network segmentation on legacy systems and cost? ANSWER: All projects are based upon risks and costs. We work with our clients on finding the best network segmentation solution.

By Vivek Ponnada, Engineer, MBA, GICSP, Regional Sales Director at Nozomi Networks, (CS)²AI Fellow January 2, 2023 While attending another well-organized BSides in Edmonton last month (closing out the triumvirate in the Western part of Canada after Calgary and Vancouver earlier in the year), a student inquired how she might get started in OT Cybersecurity. This is a common question that is often asked in other forums including Social media, online webinars etc. While I answered her a bit in our conversation, I figured that publishing a more detailed article is helpful since the answer is slightly complicated. Quick background: OT is the term used for technology used in Industrial control applications, and includes purpose-built systems & protocols (e.g., Programmable Logic Controllers, Distributed Control Systems), general IT systems repurposed with specific software to configure or view data from those control systems (e.g., Windows based Human Machine Interface, or Engineering Workstations), or a combination of technologies (Firewalls that might be ruggedized or available with ability to parse industrial protocols). Definition here from NIST. A few things about OT Cybersecurity that trip up a lot of folks are: 1) OT domain knowledge: Some contend that no one can be in OT cybersecurity unless you’ve already had experience in industrial control systems, having worked in power plants, refineries etc. While that might be a valid expectation in several contexts, especially if you are billing a customer as a ‘experienced’ consultant, that stance is often is overplayed. Don’t get me wrong, a sure way for a consultant to be kicked out of a plant is to make a basic error in safety, such as not wearing the appropriate safety boots, or removing your hearing/eye protection where they are mandated. And if you go in with a swagger that you are better than the ICS personnel as you ‘know’ security better, you’ll burn bridges really quick. However, various IT Security skills are much needed in the OT world e.g., configuring firewalls & routers as part of network segmentation projects, evaluating secure remote access solutions, helping figure out patching options etc. While not every IT best practice is applicable to OT Security, a strong collaboration between those that have ICS experience and those that have IT skills can improve the overall security posture of the organization. People with IT security experience are essential for the improving overall OT Security. 2) Certs: Frankly, it’s a related topic to above, but certifications are almost never a starting point in OT security, or even the best way to be successful. Reputed organizations now offer ICS Security certs but they are fairly expensive and ideally paid by organizations, not individuals on their own. Your credibility as an OT Security practitioner is more established based on your experience, projects completed etc. rather than the most recent cert you spent time, money and effort on. Lots of communities exist where you can get involved – see below ‘Content, Connections and how you can get started!’ section - while you are building either your ICS knowledge or Cybersecurity skills. It never hurts to get real-world experience including plant visits whenever you can (that surely means safety orientations, general awareness of what’s important to plant personnel etc.). 3) Pace of technological change: If you are a keyboard ninja that’s excited about constantly updating your tools with docker, k8s etc., this industry, even the rare penetration-testing roles, might not be for you. OT technologies rarely update that quickly though learning about them might take a lifetime. Suffice to say that OT technology will be like your home iPad/tablet that’s been around for 5 years even while newer versions are available. Though you might typically update other devices like a Smartphone every year, you use the iPad/tablet longer because it works really well for the purpose you bought it for, and unless it fails, you find it hard to justify upgrading. OT has a lot of older equipment because the controllers and systems were engineered for a purpose they work for very well even after 15+ years, replacement is CapEx driven & resource intensive etc., so upgrading them based on security alone is rarely justifiable due to the outage time required that impacts business revenue. I sincerely hope you consider a career in OT Cybersecurity because it’s challenging, fun and rewarding. From a dispassionate point of view, the industry is fairly new, clearly needs significant additional resources, and the added public scrutiny leading to regulatory pressure is adding more jobs. For organizations, it’s not easy to find anyone with the ideal combination that is a mix of IT Security skills and operational knowledge. So, the more people cross-pollinate their skillsets and focus on OT Security, the better. Being in OT Security invariably leads to working towards protecting Critical Infrastructure, such as Power, Oil & Gas, Transportation and so many more verticals, which besides being economically attractive can be something you can be extremely proud of, a rare combination in a civilian job! Content, Connections and how you can get started! There are quite a few resources available these days that you can learn in your preferred audio/visual/in-person methods! 1) Podcasts (search in your favorite podcast store) a. CS2AI Podcast b. The Industrial Security Podcast c. Unsolicited Response Podcast d. Hack the Plant Podcast e. Many other vendor-sponsored ones (search for Industrial Security) 2) Videos/Webinars a. On-Ramp, Highway and Autobahn playlists in S4xEvents YouTube - https://www.youtube.com/@S4Events b. https://www.cs2ai.org/ - anyone can join the free webinars, membership required to watch recordings c. Sans Webcasts - https://www.sans.org/webcasts/ - several ICS/OT focused webinars including recordings from past years d. https://www.brighttalk.com/ has several ICS focused videos e. https://www.cisa.gov/uscert/ics/Industrial-Control-Systems-Joint-Working-Group-ICSJWG has several video recordings and content including training 3) Cybersecurity groups – whether it’s a local DefCon chapter, a BSides Committee, or Security meetup, explore the different options in your region. Obviously bigger cities have more options than in the rural areas, but you’ll be surprised how welcoming and close-knit the security members if you just reach out. While most might not have a significant OT content/focus, chances are they have someone or something that is in ICS/OT a. https://forum.defcon.org/social-groups b. http://www.securitybsides.com/w/page/12194156/FrontPage 4) Projects of special interest a. MITRE ATT&CK for ICS - https://attack.mitre.org/techniques/ics/ b. Secure PLC Coding Practices – https://www.plc-security.com/ c. Incident Command System for Industrial Control Systems - https://www.ics4ics.org/ d. Sign up to the mailing list at ISA https://www.isa.org/connectivity-and-cybersecurity - they regularly blog on ISA/IEC 62443 implementations, Risk management in OT etc.; if you want to get further involved, join an ISA chapter in your area, and you can get view access to the standards as well, or participate in the committees 5) Conferences – while ICS/OT focused conferences are few but famous, there are many others that are critical infrastructure focused, so naturally will have a cybersecurity topic or two that would be relevant and are much more affordable for individuals, and might even be free for students. While building your interest/value prop towards being able to attend the later ones of the list, start with your regional conferences that might have some OT content a. BSides (same link as above) b. Houston Security Conference - http://houstonseccon.org/ c. Cyber Security for Critical Assets https://www.cs4ca.com/ d. Cyber Senate - https://www.cybersenate.com/control-systems-cybersecurity-usa/ e. Sans ICS Security Summit – Orlando - https://www.sans.org f. API Cybersecurity Conference - https://events.api.org/ g. ICS Cyber Security Conference – Atlanta - https://www.icscybersecurityconference.com/ h. Industrial Security Conference – Copenhagen - https://insightevents.dk/isc-cph/ i. S4 Conference – Miami - https://s4xevents.com/ 6) Vendors – use a throw-away email to sign up to all major OT vendors’ mailing lists. They provide regular content on various topics including threat reports, recent vulnerabilities and exploits etc., and you can unsubscribe later to those that don’t align with your interests 7) Social Media a. There are some fantastic content creators and influencers; a recommended list is here https://mobile.twitter.com/i/lists/1549766676392165377 but be aware that due to recent shifts, many of these sources have moved to the federated universe. You can find many of them on LinkedIn as well, and they might be hosting content on YouTube, Github etc. b. CISA (US-CERT) have social media presence. Pick your platform to follow them (or email list https://www.cisa.gov/uscert/ics/advisories) 8) CTFs and Gamified learning a. https://store.steampowered.com/app/994670/ThreatGEN_Red_vs_Blue/ b. https://tryhackme.com/paths - not ICS focused but still relevant c. https://www.sans.org/mlp/holiday-hack-challenge/ - not ICS focused but still relevant 9) Books/Reading Material a. NIST - 800-82R2 - Guide to Industrial Control Systems Security https://csrc.nist.gov/publications/detail/sp/800-82/rev-2/final b. Industrial Network Security: Securing Critical Infrastructure Networks for Smart Grid, SCADA, and Other Industrial Control Systems - by Eric D. Knapp , Joel Thomas Langill c. Handbook of SCADA/Control Systems Security – Illustrated, by Burt G. Look , Robert Radvanovsky , Jacob Brodsky d. Hacking Exposed Industrial Control Systems: ICS and SCADA Security Secrets & Solutions –by Clint Bodungen, Bryan Singer, Aaron Shbeeb , Kyle Wilhoit , Stephen Hilt e. Industrial Cybersecurity: Efficiently monitor the cybersecurity posture of your ICS environment, 2nd Edition – by Pascal Ackerman f. Pentesting Industrial Control Systems: An ethical hacker's guide to analyzing, compromising, mitigating, and securing industrial processes Paperback – by Paul Smith g. Countering Cyber Sabotage: Introducing Consequence-Driven, Cyber-Informed Engineering (CCE)– by Andrew A. Bochman , Sarah Freeman h. Industrial Cybersecurity: Case Studies and Best Practices by Steve Mustard 10) Formal training a. https://www.abhisam.com/industrial-control-system-cybersecurity/ b. https://icscsi.org/training.html c. https://www.isa.org/certification/certificate-programs/cybersecurity d. https://www.sans.org/job-roles-roadmap/industrial-control-systems/?msc=job-roles-page

By Steve Mustard, PE, CAP, GICSP (CS)²AI Fellow October 8, 2022 I’ve recently had a book published, titled Industrial Cybersecurity Case Studies and Best Practices. The book is my attempt to summarize all that I have seen and learned about in industrial cybersecurity in the past two decades. I feel that we have made progress since the early 2000’s, but I still feel we have a way to go before we are fully managing our collective industrial cybersecurity risks. Some sectors are doing better than others, but every facility I’ve ever visited has multiple vulnerabilities. Even the most modern greenfield facilities include inherent vulnerabilities that could have been removed with the right processes and procedures in place. Much of what I have seen in my travels relates to failures of people and process. The cybersecurity profession responds to the challenge by offering more technology, but technology can only do so much to cover up failures in people and process. On a positive note, the industrial environment offers established practices and culture around safety that can be readily adapted to manage cybersecurity. The book covers several areas including: Measure to Manage Risk - Now that organizations have a better understanding of the difference between industrial cybersecurity and IT cybersecurity, there is an opportunity to apply existing proven industrial risk management practices. The use of statistical methods can provide a more reliable estimate of the likelihood of a cybersecurity incident. This more reliable estimate can be used to better identify the risk reduction needed to manage the risk to as low as reasonably practicable (ALARP). The use of existing tools such as bowtie diagrams can help to elevate the significance of controls needed to maintain a secure industrial facility. Standardized designs and Vendor Certification - I believe one of the biggest opportunities to improve cybersecurity in industrial facilities is with better design practices. The Purdue hierarchy has been a mainstay of automation and control systems for 30 years and is utilized in the ISA/IEC62443 series of standards when considering cybersecurity of these systems. In recent years the applicability of the hierarchy has been called into question. In fact, the Purdue hierarchy remains as essential to automation and control systems design as the OSI seven-layer model is to network design. Certification is not the only answer to effective cybersecurity, but it does drive improvements in design and development, and it does provide an independent level of assurance. The Pitfalls of Project Delivery - Despite the widespread awareness of the cybersecurity threat and the availability of standards, certified products, certified professionals, and collective experience, systems are still being deployed that lack the most basic security controls. In addition, the projects themselves create additional security vulnerabilities due to poor training, awareness, and oversight among personnel. In addition, a focus on efficiency and cost reduction means that many of the duties involved in managing cybersecurity are added to existing workloads, rather than to dedicated professionals with the right mix of skills and knowledge. What We Can Learn from the Safety Culture - Visit any OT facility today and you will likely find several obvious cybersecurity policy violations or bad practices. Even in regulated industries, compliance with cybersecurity regulations is, at the time of this writing, not where it should be. NERC, for instance, continues to fine companies that fail to follow its cybersecurity regulations. Human behavior must be understood if organizations are to provide good awareness training for their employees. Additional controls can be deployed to minimize the consequences of such mistakes, but effectiveness varies. Safeguarding Operational Support - Safety is a major concern in industrial environments, yet cybersecurity, despite being a potential initiating cause in these hazards, is not respected in the same way as safety is. Many organizations begin meetings or presentations with the refrain that safety is the number one concern. But in those same meetings, there may be comments to the effect that “We have more important priorities than cybersecurity.” Clearly, there is still much to do before cybersecurity receives the attention it requires in operational environments. I hope that the book adds to the body of knowledge and can help others with our collective mission of improving industrial cybersecurity. You can read more about the above topics in a series of posts on the ISAGCA blog.

By Bayron Lopez, Director of Operational Technology at Kilroy Realty Corporation (CS)²AI Fellow September 22, 2022 With the growing landscape of intelligent building systems being deployed into commercial real estate, asset owners must develop a cyber-physical strategy to meet the ever-changing threats. As we continue integrating access controls, cameras, smart lighting, and even intelligent irrigation at scale, we must ensure that we do not sacrifice security for accessibility. With the addition of new technologies into the space, we must be able to vet a solution's software components and hardware integration. As the Director of Operational Technology, I work with all verticals of our organization to deploy technology that meets site requirements and protects them. One of the biggest threats I see is the lack of seriousness surrounding these systems. Don't get me wrong, as an industry, we spend millions on the usage and development of technology to meet the needs of the properties. We have elevators that can track your phone and take you to the correct floor, a turnstile that can recognize your face and allow you into a building, and sensors that can tell you how busy the cafeteria is, so you don't miss that hot cup of coffee. The comforts are there, yet we still lack enough understanding of the hardware security that operates those systems. Many are still under the impression that because these are not your traditional "IT systems," they don't require as many security policies around them. Yet if someone hacks into the system that stores all the faces and names to that turnstile access control system, that would be a significant breach. We have seen threat actors expose camera vulnerabilities due to lax security policies. The industry keeps deploying technology to make the lives of both the occupants and operators easier, but it also opens the sites up to potential harm from others. For years, individuals have been screaming at the top of their lungs regarding these threats, yet we decided to focus our attention elsewhere. The buzz words of digital twinning, fault detection, and many others that I call bells and whistles overtook the industry. We became infatuated with having more tech, more systems, and more shiny things that we hoped would distract threat actors from really looking under the hood. There was an explosion of solutions, and if you had an idea, there was a vendor that would promise you that it was possible. As the layers of systems and data became deeper and deeper, cracks began to show on the foundation. We lacked the seriousness of deploying a cyber-physical foundation for these systems. Many believed that they were not as complex as the corporate side of the house and that there was no way that they could produce large amounts of data. Some deployed 4K cameras without thinking that those unmanaged 10/100 switches would never be able to handle the traffic. Some gave access to vendors to access their data via open internet connections, not understanding the potential threats they had created. Even I was unaware that we could produce more than 15,000 data points daily from a single occupancy system. It wasn't until someone got hacked that most of us started to pay attention to those individuals screaming for seriousness in control systems. Fortunately, those individuals never gave up, and now we are starting to understand the foundational implementations they have been pushing. I've been fortunate to chat with my fellow CS2AI Fellows on some of these topics and have learned the importance of re-sealing our foundations.

By Jaco Benadie, Partner, Technology Consulting at Ernst & Young Consulting (CS)²AI Fellow July 4, 2022 Operational Technology Cybersecurity – United and Strong We are living in a time where speed and connectivity is everything. The pace of digital transformation continues to accelerate, and the complexity of these technologies makes it extremely difficult to fully understand the vulnerabilities and risks it brings until it is sometimes too late. “Countering Cyber Sabotage” by Andew A. Bochman and Sarah Freeman opens a whole new perspective of combining traditional safeguards with modern day digital controls to protect our critical systems. Please check out my original article on LinkedIn to read about my view on how to implement Consequence-based, Cyber-informed Engineering. https://www.linkedin.com/pulse/operational-technology-cybersecurity-united-strong-jaco-benadie/

By Fred Gordy Director of Cybersecurity at Intelligent Buildings, LLC, (CS)²AI Fellow March, 2022 Building Control System WhisperGate Attack Post on LinkedIn https://www.linkedin.com/posts/intelligent-buildings_intelligentbuildings-smartbuildings-cre-activity-6912767514456305665-hEZY?utm_source=linkedin_share&utm_medium=member_desktop_web I have periodically monitored several Russian aligned ransomware groups’ dark websites, primarily focusing on Conti. If you are not familiar, Conti is by far the most successful ransomware group in operation today, routinely pulling in multi-million-dollar payments from victim organizations, and they publicly announced their support for Russia when they invaded Ukraine. They are not the only ransomware groups to announce support for Russia. Others include UNC1151, Zatoichi, Killnet, Stormous Ransomware, Digital Cobra Gang (DCG), Freecivillian, SandWorm, The Red Bandits, and Coomingproject. I have noticed an upwards spike in U.S. companies showing up on Conti’s site. As recent as today, March 24th, 2022, a U.S.-based mechanical engineering and construction firm that, according to their website, is a leader in the Washington, D.C. market. Their website says they work on complex commercial, government, and institutional design-build projects. Two days ago, a U.S.-based cancer diagnostics laboratory was ransomed. Data is up for sale for both these companies and others, and they have been locked out of their systems. These two examples are centered around data, but the building controls community is not exempt. We recently were able to stop an attack on several building systems from what we believe to be Russia. WhisperGate malware was found and contained before it could do any damage. WhisperGate is a sophisticated malware known for targeting multiple organizations in Ukraine. It has two stages that corrupt a system’s master boot record, displays a fake ransomware note, and encrypts files based on certain file extensions. Conti Post on LinkedIn https://www.linkedin.com/posts/fredgordy_cybersecurity-conti-ransomware-activity-6910592084202655744-Wn8P?utm_source=linkedin_share&utm_medium=member_desktop_web

By Robin Berthier, CEO & Co-Founder at Network Perception February 9, 2022 We hosted a (CS)²AI Online™ symposium on January 19, 2022 that focused on Cyber Security for Energy: Part 2 - Electric Sector. Here is a bit about the event: Part 2 of the Symposium on Control System Cyber Security for Energy will provide tangible recommendations and best practices for electric utilities to address current and upcoming compliance and cybersecurity challenges. First, attendees will gain a detailed understanding of the latest government regulations that have been pushed by recent changes in the threat landscape. Second, industry practitioners will share their experience on technology solutions and process improvements to mitigate risk faster and build a strong culture of cyber resiliency. The symposium will provide ample opportunities throughout the event to interact, ask questions, and leverage the shared expertise of the (CS)²AI community. Speakers: • Melissa Hathaway (President, Hathaway Global Strategies) - Keynote • Marc Rogers (VP of Cybersecurity at Okta): Hands-on experience on exploit • Ben Sooter (Principal Project Manager EPRI: Responding to High Impact Cyber Security events in Operations • Branko Terzic (Former FERC Commissioner): Challenges for electric utilities • Philip Huff (Univ. of Arkansas): Vulnerability Management for electric utilities • Todd Chwialkowski (EDF-RE): Implementing Electronic Security Controls • Saman Zonouz, Threats to Programmable Logic Controllers (PLCs) • Robin Berthier (Network Perception): NERC CIP Firewall Change Review Workflow As always, we encourage our audience to participate throughout the event by contributing feedback and questions for our speakers. We weren't able to answer all of your questions, so we have asked some of our speakers to answer a few of them. Below, are some answers to a few questions posed during the event. Do you want to have access to more content like this? Register for our next (CS)²AI Online™ seminar or symposium here, or consider becoming a member of (CS)²AI to access our entire library of recorded sessions. ****************************** QUESTION: Haven't we learned our lesson about companies claiming they're 100% hack-proof? ANSWER: Exactly, it is now well-established that 100% security is unrealistic. This is why organizations have to invest in cyber resiliency: designing systems, processes, and training to be able to keep operating despite being under attack. QUESTION: How are firewalls and DMZ’s validated for built-in backdoors? ANSWER: Through a combination of configuration verification and network traffic monitoring. Independent verification of firewall and router configuration files enables security team to validate that no backdoor access has been inserted. Network traffic monitoring enables security team to ensure that no process is subverting the access control implemented. QUESTION: Is this NP approach IT first, then proceeding to OT? not many firewalls in OT that need to be constantly tweaked or tuned. ANSWER: This depends on organizations. Some have many firewalls in their OT environments. Even if network changes are less frequent in OT compared to IT, we recommend starting with OT verification since that's where the most critical cyber assets are located, and then expanded into IT. QUESTION: What are the most electric companies getting incorrect witht their configurations? ANSWER: 1. Lack of egress access control 2. Lack of documentation 3. Overly permissive rules 4. Insecure services 5. Access list complexity QUESTION: Would all the pieces of the firewall analysis and monitoring done from inside the ESP, cloud, corporate side? ANSWER: We recommend to deploy the firewall analysis platform in the DMZ next to the ESP with unidirectional data retrieval. QUESTION: What is the major differences between traditional firewall system and your suggested firewall system that we can consider? ANSWER: Traditional systems rely on a single firewall management solution. We recommend to separate monitoring from management. Monitoring should be done independently from the management platform so it can be done read-only and reduce the risk of human error. QUESTION: What is best security way during OT connectivity with IT ? data diode or firewall ANSWER: Misconfigured data-diodes can be less secure than correctly configured firewalls, so the key to best security isn't one or the other, but the correctness and continuous verification of the device configurations.