Geospatial Information Systems and Big Data

The laugh I got recently when, in light of a marketing tagline, I was prompted to look up the definition of the word “version” provides our lead-in for this IdeaXchange:

version   (noun)

1. a particular form of something differing in certain respects from an earlier form or other forms of the same type of thing.
2. an account of a matter from a particular person's point of view.

Now think about the following marketing tagline of a goal many of us have heard when it comes to utility’s asset and network model-building and related enterprise software solution integration work involving geospatial information systems (GIS):   

“To achieve one version of the truth.” 

While the quest to get to one version of the truth may seem admirable, when I think about Big Data issues I wonder whether “one version of the truth” should go right up there with “jumbo shrimp” on a list of self-contradicting phrases.

I invite our Xperts to respond to any of the following short excerpts from the six Xperts’ replies regarding the Big Data question John H. Baker asked in November 2014: “Is BIG DATA a Big Bust?”

Our new add-on question is: Can a GIS-based “one version of the truth” goal meet all our Big Data-related concerns?

"Big Data WILL be a big bust if we don't..."

1. “…find ways for our utilities to manage, protect and extract value from BIG DATA [with] a holistic, standards-based approach to grid modernization [and] open architecture.” — John McDonald
2. “…address data errors [and] do a lot of work and planning to get to answers that are based on good data [for numerous systems that] are going to depend on getting these foundations right... Fixing the underlying data, grid topologies, grid models, electrical characteristics and other foundation pieces will not be cheap.” — Doug Houseman
3. “….[avoid Big Data’s] potential to be a significant cost and operational burden…. [and think] enough about how to leverage the data to create insight [which includes] investments not only in tools but in people, particularly those that can see the implications of what the information is telling them.” — Stewart Ramsay
4. “Figure out what insights (outcomes) are important to you, decide how to store the data [optimally] and when the business case is justified, implement the specific analytics outcomes, [which] includes people and processes…” — Mani Vadari
5. “….[if we don’t address] what makes this unsolvable — the boundaries we refuse to push out.”— Michael Heyeck
6. “…advocate a well-planned strategic approach that provides the end user(s) with clear, concise, actionable reports that contain useful data.” — Richard Ladroga

Brad Schmidt

The concept of one version for Big Data issues regarding utility operations is by all measures and concepts, a misnomer at best from two aspects: a) the manner in which the data is captured (i.e. the different software platforms) and b) the manner in which the data is used.

In today’s ever-changing world of technology, it’s virtually impossible for one software platform to excel in every area of utility operation regarding Big Data (BD); that’s just not realistic. What is realistic is the absolute necessity of those various platforms being able to communicate with one another so that data that might be common to them is available, doesn’t have to be re-captured or worse, re-entered. Thus data-sharing between platforms is an absolute. Any time data has to be duplicated, the process is ripe for errors, and errors lead to massive problems when it comes to data.

BD utilization, the second aspect, is also changing rapidly. Customer-service, accounting, engineering, etc. are all finding new opportunities to do their work better, more efficiently, and with greater meaning than ever before through access to BD. Here at Cass County Electric Cooperative (CCEC) there are numerous examples where AMI data has been proven within the first year of availability, to resolve numerous problems/issues/challenges that may at some point have been identified or maybe not. Issues like unauthorized interconnection of a DG system to the distribution grid; stuck regulator controls that are causing voltage problems for the end-user; creating virtual ‘circuit-feeder’ metering for engineering analysis are all real-life examples where BD has become the foundation for doing that work.

In the customer service world, resolution of high-usage issues, precise meter reading issues for billing cut-offs, and abilities to verify load management compliance performance for special rate considerations are all other real-world examples where BD is the key. Again, CCEC has numerous examples where this has been used with impressive results.

Everything CCEC ever hoped to achieve with BD has been realized and surpassed in the short time we’ve enjoyed the technology…and we have yet to dip below the tip of our data iceberg.

If there is any potential for a ‘one version of the truth’ for BD, it might be in the management of that data once it’s collected, whether that be a lone-software management concept, or joint ownership through various software platforms. Therein lies a daunting challenge to verify the authenticity of the BD, which is only second to assuring every minute of every hour of every day is accounted for with BD for those time elements.

BD is bigger than mere words can begin to describe…literally…in terms of how much data will be captured by utilities. Again, any question as to the accuracy or authenticity of BD will negate any of the best attempts to use the data to build any sort of foundation from which the utility employee does their work.

A most challenging aspect of data in today’s utility world, aside from protocol standards, is the communications connection between the data systems. As security concerns mushroom in our daily work worlds, systems are becoming less tolerant of non-compliance when it comes to software security, or are becoming ever more demanding for the same, with little or no grey lines between the go and no-go decision to accept data from outlying sources. These systems simply ostracize any non-complying entity. The ever-present concern by software firms to not have a finger pointed at them for allowing the ever-dreaded virus vampire into the BD storage vat will only grow as the threat of losing or poisoning any BD that’s been captured, verified, and logged is guaranteed safe.

Lee Willis

This worries me a lot. We’ve all heard it in fiction, fact, from the government (hard to know which it is there), institutions, philosophers and researchers: humans evolved to look for patterns. Apparently this is why we have so many superstitions and urban myths.  We see patterns and trends even when they really aren’t there. Big data and big data analytics are potentially useful, maybe even valuable (I have my doubts, but . . . okay, potentially). But if you go looking for patterns, you will find them, and all the math and the big data illusion that your knowledge base is based on “knowing everything” give you the illusion it is real and important. I think more than anything it gives consultants something to sell, and people who want to believe there is an edge they can get, something to believe it, and ultimately, a person who is looking for a unicorn, something to tell them they have found unicorn tracks . . .

Dr. Mani Vadari

The utility power system — the core system that an electric utility manages — is one of the strangest networks. For the most part, this network still functions based on the fundamental principles of physics as defined by Ohm’s law, Maxwell’s Equations and a host of other similar principles and laws. Now, what does this actually mean?

Among may other things, this means the following:

• This is still a real-time network. Generation and load still need to be balanced on an instantaneous basis and the difference is demonstrated as a frequency deviation from the norm. This will probably stay until we see electric energy storage become a reasonably strong force both in capability and price.
• Power still flows where it is directed by physics and not directable by controls and valves. This may change in the future with the advent of Smart inverters, FACTS and other similar devices which can alter the characteristics of a line forcing power to flow in the direction needed.
• Power consists of two components — real and reactive, one that converts to real energy and the other that is required to enable the flow of power from one location to the other.
• Transmission is still network-based, and distribution is still radial for the most part in most parts of the world. This specific aspect drives the design of the grid all the way through switches, fault current calculations, relays/protection equipment and their settings.
• And they are many more.

So what? Why is this even important?

This dependence on physics means that the flow of power from one place to the other depends on the physical characteristics of the grid. What are they?

• The type of component: This could be a switch, line, transformer or something else. Some of these are power system in nature (as defined above) and others could be non-power system in nature. The latter category includes components such as wood-poles, bus-bars, cross-bars and so on. A third category of component is more of a grouping; an example of this is a substation.
• Its name: What is it called?
• Its location: Where is this component located? This information tends to drive any distance-related component such as (transmission or distribution) line length and hence its resistance and so on.
• Its connectivity: Which components (if more than one) is this component connected to? This is important because it determines how the power will flow in the network.
• Its characteristics: In the case of a transmission line, what are its resistance, inductance and so on?

In addition, depending on the situation, some of these change:

• Characteristics can change with temperature, season and so on.
• Connectivity can change with the opening and closing of switches.
• And others.

This brings an extra dimension to the discussion on “one version of the truth.” What is that?

The concept of “one version of the truth” in reality is more complicated that it appears.  Let’s explain this based on some examples:

As-designed model: This is the designed model as defined by the planners and designers. This is where it all starts. Once the design is completed, this model is fixed.

As-built model: The crew that takes the design to the field for construction or modification may take field-level designs that will impact the actual implementation. Examples such as connecting the load to a different field transformer, connecting the load to a different phase, putting a pole at a different location and so on. These changes will impact how the power flows from the source to consumption. Once the build is completed and the equipment energized, this model is fixed.

As-operated model: As switches open and close (either due to planned or unplanned operations), the connectivity changes and as a result, the as-operated model changes.  This is a dynamic model and will keep changing.

This means that there are several versions of the truth, and the right one depends on what is being done with the model.

Now, let us come to Big Data and what we can do with it?

Utilities are gathering more and more data than ever in the past and this is coming from smart meters (multiple data points every 15 minutes or so), distribution automation (SCADA scan rates — 4 to 8 seconds per reading) and synchrophasors (multiple data points 30 to 60 times a second) and others. This is a lot more data then utilities have ever collected before and as more sensors get installed, the amount of data being collected will continue to increase even possibly exponentially. 

One key point needs to be kept in mind about all of this data: They are not all in one place. Given that various custom systems collect the data, they are all stored in different places and most times in very different formats. Much of the data is dependent on the connectivity model of the system that existed at that time.

While, the amount of data may not qualify it to be called Big Data, it is important to note that there is a lot of intelligence within this data that can be divided into five main categories:

• Operational data
• Asset data
• Customer data
• Meter data
• Other assorted data

What can we conclude from this?

While some intelligence can be gained from the data itself, the smart move is in crossing the data between the categories identified above and correlating them to the “correct” version of the truth, which could be either the as-designed model, as-built model or the as-operated model. For this to happen, it is important to note that the appropriate version of truth is not only accurate and up-to-date, but also available everywhere.

I believe a well-maintained and accurate version of the truth is not just important but critical to (1) proper functioning of the power system and (2) to derive good intelligence from the data. 

What do you think? Please comment in the space below.