THE LOS ANGELES DEPARTMENT OF WATER AND POWER (LADWP) is undertaking an ambitious substation automation (SA) project in 179 substations, as part of its larger Energy Control System Upgrade (ECSU) program. In mid-2004, LADWP (Los Angeles, California, U.S.) completed the comprehensive SA pilot phase validated by three system performance tests. The SA project has been in production for two years and will continue into 2007.

As is the case at many utilities, LADWP had a supervisory control and data acquisition/remote terminal unit (SCADA/RTU) system that was showing its age. Rapid detection and location of electric trouble was becoming more of a challenge. The original vendor of the system no longer supported the technology. In addition, some equipment was at the end of its service life, and legacy communication protocols were unable to extract needed information from modern intelligent electronic devices (IEDs). The relatively low bandwidth of those systems could not provide data throughput for modern applications. And, existing technology did not provide adequate reactive power control capability, a single factor carrying significant financial impact.

Likewise, data acquisition systems were not all compatible, so LADWP was not able to effectively manage its integrated resources. While there were information islands, each of which had value in and of themselves, the information could not be effectively shared. Furthermore, due to the system evolution, the islands also lacked a standard look and feel. LADWP ardently felt the need for improvements.

Faced with these system conditions, LADWP embarked on the ECSU program to develop a comprehensive solution that would remedy its immediate system shortcomings, as well as position it for the future. The ECSU program involves several elements of upgrade to the data-retrieval infrastructure, including replacement of the Energy Control Center (ECC) and automation of 179 substations.

The ECSU program, of which the SA systems are an integral part, brings several important benefits to LADWP, including improved system integrity and reliability, improved environmental quality and the establishment of a foundation layer for delivering critical data to the utility knowledge infrastructure. These benefits improve LADWP's service performance, organizational efficiency and financial performance.

When completed, LADWP anticipates bringing SCADA availability to 99.95%. ECSU also positions the SCADA to keep pace with system load growth and relieves the concern of a single SCADA master's susceptibility to outage due to natural disaster, accident or sabotage. Improved access to critical system parameters along with time stamping within 1 msec of important alarms is expected to yield a reduction in LADWP's customer average interruption duration index (CAIDI) by 10 minutes on monitored circuits. The ECSU program also provides expansion capacity, as well as capacity for increased monitor/control demands.

ECSU is leading to improved maintenance of critical system components with improved real-time data flow and improved data history. This significant impact is being enabled by providing an environment for on-time maintenance, as opposed to less-responsive periodic maintenance.

ECSU reactive power control capability will improve system efficiency through accurate capacitive control, thus requiring less generation to serve the same load. Reduced generation results in reduced emissions, which will further improve air quality in the Los Angeles basin.

ECSU also establishes a foundation layer for delivering critical data to LADWP's knowledge infrastructure and facilitates enterprisewide data sharing. The newly networked system will provide ready access to critical system data for engineering analysis and system-planning efforts, as well as provide an information bridge to the corporate enterprise.

The SA portion of ECSU includes replacing legacy REDAC RTUs with PLC/PC-based SA systems, replacing electromechanical relays with IEDs and replacing 1200-baud modems with OC-12 fiber-optic ring communication. A new fiber-optic communication ring now provides a wide-area network (WAN) retrieving data from all LADWP stations in this project. This WAN also allows remote loop-through access to all IEDs, reducing the need to send personnel to the stations.

The SA project includes the installation of human machine interfaces (HMIs), which incorporates items such as alarms, annunciator panels, control functions and station one lines into one display that is accessed by the operators in the stations. This reduces the need for panel space and provides one-stop shopping for most of the station functionality.

Other features of the SA project include the ability to automate selected control functions (at LADWP's discretion); to implement automation algorithms in the local PLC; to improve data acquisition, including oscillography and sequence of events (SOE); and to improve remote diagnostics. Updated communications with the stations through the newly installed fiber-optic WAN and the addition of loop-through provide communication with IEDs and the local station infoserver as though the engineer is in the station plugged into the IEDs' diagnostic ports. As of this writing, LADWP has ordered 120 systems (of the 179 stations in the entire project) and taken delivery of 82.

Teamwork makes this LADWP SA project work. LADWP has a dedicated owner/contractor team with the required skills and a local presence. From the beginning, the team makeup has been driven by project needs. Selecting the right contractor was accomplished through a rigorous qualification, proposal and evaluation process. This team consists of Enspiria Solutions Inc. (Greenwood Village, Colorado, U.S.) as the prime contractor and several subcontractors including Electrical Systems Ltd. (Corona, California, U.S.), Plan B Solutions (Dunedin, Florida, U.S.), Crown Electrical Systems (Los Angeles, California), Berg/ORSA (West Covina, California), PowerCon (Baltimore, Maryland, U.S.) and Configure-Data (La Jolla, California). In addition, LADWP assembled an in-house team with members from a variety of disciplines throughout the utility, including construction, system test, operations, maintenance, protective relaying and project management.

Project responsibilities are shared. The contractor builds and delivers complete systems, and LADWP is responsible for installation and site testing. To accomplish this, the contractor team provides integration management, proprietary equipment and software, factory testing, and detailed construction work packages to furnish and deliver complete factory-built systems along with system training. In addition to installation, site testing and startup, LADWP provides overall project management, source documents and standards, and operation and maintenance.

Considerable time was spent defining the roles of each team component and identifying hand-off points. Developing task assignments was a comprehensive effort by all parties. While arduous, it has proven well worth the effort. The roles of the various team members were further honed through the development of comprehensive project processes and standards. LADWP practices continual improvement of these project mechanics.

The RTU Points List Process is a good example and a critical path item. It is an essential element in developing the proper architecture for a specific station's SA system. The contractor initially developed the Points List for a given system with limited LADWP input. Early on, delays in the review process and last-minute changes resulted in significant production delays. This process was refined through team meetings and working groups. Process improvements included building LADWP input into the Points List early in the design process and thorough, timely review steps. What was once a significant challenge is now a streamlined and reliable process.

A rigorous pilot phase, conducted in 2002 and 2003, focused on refining the build-up and testing of seven SA systems prototypical of those expected to be seen in the balance of the project. This also involved the delivery of two test systems to provide an off-line environment as a proving ground to identify any desired changes in system parameters.

A comprehensive performance test was required of each of the three system types: receiving stations, industrial stations and distribution stations. These tests required all system components, including the associated relay racks, to be assembled in one area for a defined, structured test and then an unstructured test. Parameters of the structured testing were specified early in the pilot. The unstructured test provided the opportunity to push system operational limits.

The buildup, testing and initial deliveries of the pilot systems were particularly challenging from a technical standpoint: issues related to meeting specified technical requirements and issues from changing technical requirements as LADWP gained more familiarity with the systems. A prime example of meeting specs was system performance, as measured by CPU usage in the Infoserver. One of their largest systems with the maximum amount of IEDs and hardwired points was targeted for performance testing. When brought together for testing, this system had a higher CPU usage than specified. However, through a rigorous structured and unstructured testing process, CPU usage was improved and the system was ultimately approved and shipped.

The production phase brought another lesson. Previously, the team was focused on overcoming technical challenges to ensure a successful pilot system build and delivery. That focus precluded consideration of what would be required for production. As it turned out, a whole new set of challenges was encountered.