A key to cyber security situational awareness is to watch a lot of “stuff.” Stuff includes things like system log (syslog) messages from servers; network traffic (netflow) data coming from networking devices; output from anti-malware scanners; application logs from EMS, DMS and SCADA systems; alarms about multiple failed login attempts; failed control actions; attempts to use insecure tools and utilities (for example, FTP and telnet); and network packets to and from unknown Internet protocol (IP) addresses, or to and from unexpected transmission control protocol (TCP)/IP ports/services. It also includes out-of-date virus signature databases, digital certificates nearing expiration and a long list of other items.

Security alarm categories.

communications protocol server

Though there are ways to find all this information without a new tool, there is no existing single solution that can gather this information and quickly and efficiently present it to a system operator in a language the operator will understand.

The goal is for CSM to be this comprehensive, understandable tool usable by a system operator. CSM does not replace other monitoring and forensic tools used by cyber security administrators. Rather, CSM distills information into actionable insights for control systems operators and control systems support analysts by creating and communicating the following:

  • Alarms. Sending these to its own web-based user interface and, if desired, to the control system itself for inclusion in alarm lists or other user interface displays.
  • Control system logs, alarm messages and SCADA data. The mechanisms used to obtain data from the control system permit bidirectional data flow.
  • Syslog messages. Cyber security best practice is to preserve system logs for use in post-event analysis and forensics, as well as for use potentially by law enforcement agencies for investigation and prosecution. To ensure complete logs are available, CSM writes information to a syslog server, such as any alerts or alarms it generates, summaries derived from netflow data, and intermediate information used in its decision-making processes.

Operator User Interface

CSM has a menu-driven, web-based user interface with more than 50 displays (pages). A small minority of these displays are intended for use by system operators. The others are for use by system engineers, support analysts and trainers as part of CSM’s built-in operator training simulator capability. CSM can act as a stand-alone operator training simulator, or it can be used in conjunction with the Siemens Spectrum power operator training simulator. Access to the displays is role based.

Following recommendations by the industry advisors, CSM places alarms into six categories from a security alarms table in the top-level display. Typically, if a control center alarm category is active — indicating there is at least one current situation that has an alarm — the user can drill down by clicking a detail button. The user can see additional detail about individual alarms.

Additional elements on displays can be used to view and enter information:

  • Recommended actions to be performed by an operator or support analyst; these are configured by the CSM owner to adhere to a utility’s policies and procedures as well as to use vocabulary familiar to the user
  • Contact information for on-call support personnel
  • Notes about what was done to handle a specific event
  • CSM sensitivity, which can be adjusted based on the current security posture of the utility’s or partner’s systems. For example, if a corporate local area network (LAN) or an inter-control center communications protocol (ICCP) communicating partner is known to be under attack, it may be desirable to increase CSM’s sensitivity to connection attempts through perimeter firewalls.

Operating Specs

CSM was developed to run under IBM’s AIX operating system and uses an Oracle database. Initial development and testing has been done with Siemens’ Spectrum Power 3 Energy Management System and the design is such that using CSM with another control system is intended to be easy. A port to Linux is currently under investigation to make CSM accessible to a wider audience.

CSM is designed primarily for use in a comparatively static networked system like a control system. It is configured to know what the components of the control system are and how they communicate among themselves and the outside world. CSM is not designed to operate in a dynamic network such as an office local area network, where changes are frequent.

Actionable Intelligence

CSM is not a stand-alone magic bullet for cyber security. However, it holds the promise of being able to collect information from a variety of disparate devices and distill that information into actionable intelligence for use by a control system operator. This allows the system operator to become an active participant in recognizing and responding to cyber security incidents.

Acknowledgement

The authors would like to express their appreciation to  Mark Flanary, PMP (Siemens project manager), Mike Stemper (CSM product manager) and Dave Taylor, CISSP (CSM principal investigator) for their assistance in preparing this article.

The authors also note that this article is based on work supported by the Department of Energy under award number DE-OE0000517.


Valentine A. Emesih (valentine.emesih@centerpointenergy.com) is director of the control systems department in CenterPoint Energy’s electric grid and market operations. He is responsible for various control systems used to securely monitor, manage and control advanced metering system meters, as well as electric distribution and transmission system field devices. Prior to joining CenterPoint Energy, he held engineering, system development and project management positions for electric utility automation systems vendors — Ferranti International Controls (now Ventyx -ABB ) and Johnson Controls (now ARINC Inc). Emesih has BSEE and MSEE degrees from The University of Texas at Austin and Auburn University, respectively, and is a professional engineer.

Bruce Oliver (bruce.oliver@smud.org) is the supervisor of energy management systems for the Sacramento Municipal Utility District’s grid operations and planning. He is responsible for systems used to control and monitor the generation, transmission, distribution and gas pipeline systems. He has a BSEE degree from California State University, Sacramento, and is a licensed professional engineer in California.

Companies mentioned:

CenterPoint Energy| www.centerpointenergy.com

Department of Energy| http://energy.gov

IBM| www.ibm.com

Linux| www.linux.org

New York Power Authority| www.nypa.gov

North American Electric Reliability Corp. | www.nerc.com

Omaha Public Power District| www.oppd.com

Oracle| www.oracle.com

Pacific Northwest National Laboratory| www.pnnl.gov

Sacramento Municipal Utility District| www.smud.org

Siemens| www.siemens.com

Westar Energy| www.westarenergy.com