THE DAKOTA ELECTRIC ASSOCIATION (DEA) KNEW IT WAS TIME TO UPDATE ITS SCADA SYSTEM and that it needed a different approach to make sure the new system would work well with its people and processes. In the past, DEA (Farmington, Minnesota, U.S.) tended to take the traditional utility approach and purchased equipment systems that worked well from an operating standpoint, but didn't integrate well with existing or future information infrastructure. This time, DEA was determined to take more of an “own and operate” approach and ensure the SCADA system could be adapted, configured and expanded to meet its current and future needs.

DEA has used a Telvent SCADA system since 1996. This system consists of two workstations that function as hot and standby SCADA servers, two workstations that function as hot and standby history information servers, one workstation as an additional operator interface and one workstation as a development platform. All workstations were manufactured by Sun (Santa Clara, California, U.S.) using its Solaris operating system and networked on redundant Ethernet LANs.

The SCADA communications network is composed of an Ethernet radio wide-area network (WAN) and seven MAS-licensed radio channels. This network is used to communicate with more than 130 Telvent RTUs using the IDLC/SES92 protocol. Schweitzer Engineering Laboratory (SEL; Pullman, Washington, U.S.) IEDs are installed at each substation. The IEDs are used to protect the substation transformer, main bus and feeders, and also to collect substation analog and status data. At some locations, the IEDs are interfaced to RTUs through RTU serial ports. At other sites, DEA uses SEL-2030 concentrators to interface the IEDs to the RTUs. At all locations, data from the IEDs is mapped to the associated RTUs. Due to several problems with this system — including failing hardware, limitations in productivity enhancement opportunities, difficulties using a unique software version and limitations of a proprietary communications protocol — DEA determined in the fall of 2002 that it should begin planning an upgrade to the system in 2003.

Therefore, beginning early in 2003, DEA developed a request for proposal (RFP) and sent it out to vendors, five of which responded. After further evaluation, three vendors were selected for detailed investigations, including factory visits and interviewing existing customers. During the last week of January 2004, DEA finally selected one of the vendors and began negotiating a statement of work.

REQUIREMENTS REFINED

It was during the detailed negotiations of the statement of work that DEA realized the importance of the collective requirements of the new system and determined that the SCADA system had to possess the following qualities:

  1. The new SCADA system had to be based on software that has a large, diverse user base, yet also caters to the unique needs of the electric distribution industry. A large user base is the best guarantee the product will be available in the future. In serving the needs of diversified industries, more resources can be made available by the supplier to add new features and ensure high product quality.

  2. The new SCADA system had to be based on familiar technology. For DEA, this meant that the new system must use Intel-based computers, the current release of the Microsoft Windows operating systems, Microsoft SQL Server for any relational database needs, Microsoft Visual Basic for a scripting language and Microsoft Active Directory technology for securely connecting to DEA's existing networks.

  3. The new SCADA system had to have extensive and flexible product support options so that DEA is able to adjust the level of support as its needs change. Key areas of product support DEA wanted provided were:

    • Support personnel located in the Twin Cities metro area, where DEA is headquartered. Local support reduces on-site response time as well as travel and per diem costs.

    • Training programs located in the Twin Cities metro area. Local training reduces travel and per diem costs and is less disruptive for the maintenance staff.

    • 24/7 emergency support.

    • A standard product upgrade and maintenance program that provides access to the latest product improvements, enhancements, tools and features so that DEA can upgrade the product without external resources.

  4. The new SCADA system had to be flexible so that DEA can use the tools within the system to meet future challenges.

  5. The system had to be capable of sharing information with other key systems. The new system did not need to provide outage management and engineering modeling functionality because DEA decided to continue its best-of-breed approach in these areas. This was a major turning point in the project, because, without this constraint, many more options became available.

  6. The new system had to be flexible enough to accommodate DEA's desire to play an active role in every aspect of the configuring, testing and commissioning of the new SCADA system. DEA intended to participate in the following tasks:

  • Project planning and standardization

  • Hardware selection

  • Network wiring and design

  • Installation and licensing of Microsoft Office products, Microsoft Internet Explorer and anti-virus software

  • Modifications to network equipment and domain configurations

  • Conversion of the existing RTUs to the DNP 3.0 protocol

  • Connection of the new system to the existing WAN for testing

  • Participate in the setup and configuration of the provided equipment for testing

  • Participate in Witnessed Factory testing

  • Attend weekly progress meetings

  • Attend monthly project status meetings.

THE APPROACH

DEA was trotting down an unfamiliar path, so every area needed close examination. To begin, DEA needed to find out if the requirements as set forth in its RFP could be met.

Step 1 — Investigation: The first step DEA took was to investigate which shrink-wrapped software products have been used in the electric distribution industry. Not surprisingly, DEA discovered that each of the two largest makers of SCADA system software — Invensys and General Electric — had successful installations of their products.

Wonderware — a product of the global conglomerate Invensys — has installations in several municipal applications. The closest application was at an electric utility in Wisconsin and its power supplier. The utilities were replacing their existing traditional master station with Wonderware. Furthermore, both utilities were installing the software themselves and using the tools available within the package to customize the application to meet their needs.

iFIX — a product of the global conglomerate GE Fanuc — has been installed in several applications, including Sulpher Springs Valley cooperative in Arizona. While investigating iFIX, DEA discovered that a New Zealand-based company called Catapult had developed a product called iPower that provided electric utility functionality necessary to integrate with iFIX to form a complete electric distribution SCADA solution. Sulfur Springs Valley Cooperative was an early adopter of iFIX and iPower, and in a phone interview, indicated that they were more satisfied with their system than they were with the traditional system that their G&T provided for their use.

Step 2 — Select and Pilot: Based on preliminary examination of the iPower and iFIX products, discussions with the staff at Sulfur Springs Valley Cooperative and the fact that iPower was a product tightly integrated with iFIX, DEA decided to take the next step and began a pilot project in early May 2004 to determine if the iPower and iFIX products lived up to their sales information and could meet DEA's requirements. The key items that were demonstrated during the pilot included:

  • Reliable, robust communications between the SCADA Master Station and the RTUs using DNP 3.0 protocol.

  • Select-before-operate control.

  • Graphic displays for two substations and two distributed generators similar to those of the existing system. Functionality similar to the current system was demonstrated and many new features identified in the RFP were also demonstrated.

  • Database development tools.

  • Alarm and event processing and functionality.

  • Security features.

  • SCADA server redundancy.

Step 3 — Evaluate Pilot Results: Throughout the pilot, DEA staff confirmed the compliance of the software with most of the key requirements set forth in the RFP. However, DEA also confirmed that the current product did not meet four of the requirements:

  • No command failure monitoring function

  • No momentary change detect function

  • No real-time connection to Cannon Technologies Yukon system

  • The ability to dynamically generate lists of points with data was deemed too slow.

Catapult quoted prices to perform the work that would be required to bring the product into compliance. Catapult further committed that the additional functional requirements DEA had requested were reasonable and applicable to other users, and incorporated them into the next release of its iPower product.

There was a price for aligning with a standard product. The proposed system was different than the existing system in many ways, mostly good but some not so good. The team spent a great deal of time learning and understanding these differences so that they could evaluate their true impact on DEA.

Step 4 — Develop a Go-Forward Plan: Having witnessed that the iPower/iFIX software met most of its key requirements and having received assurances that those requirements not tested also comply or would be brought into compliance, DEA began efforts to consolidate the entire approach into a proposal. Since the approach required that software from several manufacturers be integrated together to perform all of the required tasks, a large portion of the planning effort was focused on who should do the integration.

Many integration options were examined, including using Industrial Network Systems (the local GE Fanuc and iPower sales representative), using Catapult, using a local iFIX integrator, using a U.S.-based integrator already familiar with iPower, DEA staff doing it themselves or a combination of all of these options. DEA decided to lead the integration effort and augment their skills with the experience of the local sales representative and Catapult.

INTEGRATION

In September 2004, system hardware and software were ordered, received and configured. The DEA engineers responsible for the integration attended product training. The scope of work documents for Catapult's four enhancements were negotiated and implemented.

In October, work began on developing the “factory” test plan, which provided the framework for the integration work that needed to be accomplished and tested before the end of the year. Items to be tested included all functionality required in the RFP. Special attention was given to those items not tested during the pilot, especially alarm throughput processing.

Later in the month, work shifted from system set-up tasks to configuring the database and displays. DEA decided to build the graphic displays and navigation methods to resemble the existing SCADA system. This was done to encourage the acceptance of the new system and to make it easier for staff to begin using the new system with minimal training.

Work continued throughout November with significant effort spent on testing the new iPower product. By the end of the month, the system was being prepared for the factory test, which began in early December and was completed by the end of the month with satisfactory results.

PARADIGM SHIFT

Having completed the factory testing and learning the ins and outs of the new product capabilities, DEA took a step back to access whether it should continue to use these new tools to create a user environment that was just like the old system. New navigation and standardized single-line diagram templates were developed that had a look and feel more like Windows. Both interfaces were presented to the system controllers, and DEA decided to move away from the style of its familiar SCADA system and instead represent the information more like the other personal computer applications it was using. This marked another major turning point in the new SCADA system, where users were now beginning to think of the new system as a source of information and a tool to be used to make them more productive.

CUTOVER

DEA dual ported each substation RTU to take advantage of WAN and RTU capabilities. This allowed DEA to continue to use the existing SCADA system while individual substations were commissioned.

At the beginning of this process, a new tool was developed in-house to standardize SCADA database point development. The tool allowed DEA to create and configure a library of standard points, including information for the system controllers on exactly how to respond to a particular alarm. The engineer responsible for the database then mapped each standard point into each RTU and the tool used this map to create a database file that was loaded into the SCADA servers.

Having remedied the past problems of database consistency with this new tool, developed graphic templates for the three different styles of substations and changed work procedures, DEA began developing databases, graphic displays and performing point-to-point checkouts on all of the substations in April.

System controller training was conducted in September and October, and DEA went live with the new system monitoring substations on December 14th. Cutover to the new system went very smoothly. Planning and standardization of generation database and displays are currently underway and it is anticipated that cutover of all remaining RTUs will be completed by June 2006.

RESULTS

DEA found that the SCADA software was flexible and well supported. The biggest lesson DEA learned is that integrating an open SCADA software is still a large task and consumes significant resources. It took more time than expected to learn the new system, develop testing materials, develop training materials, procure equipment, set up equipment and develop administration documentation. DEA provided feedback to Catapult that future integration enhancements would help prospective utility users.

To date, new standard software versions have been received and installed. The DNP driver software has recently been upgraded and the new version of iFIX from GE should arrive soon. So far, the shrink-wrap approach has been a success, and DEA is accomplishing its goal of keeping the system updated with in-house expertise.

Len Jewell joined Dakota Electric in 2000 and is currently the technical system manager responsible for the operation of load management, data communication, graphical information, OMS and SCADA systems. He is a registered control system engineer in Minnesota and a project management professional. Between 1990 and 2000, he designed and implemented SCADA systems for large water and wastewater treatment facilities throughout the United States. ljewell@dakotaelectric.com