Utility managers, regulators and even customers are consistently asking vegetation managers challenging questions. Managers ask if we are getting the best reasonable productivity from field workers and if work crews consistently follow specifications and performance expectations. Customers often want to know if the work is being performed properly. Regulators and managers want to know if we have an effective, documented contract monitoring and enforcement tool, and objective, accurate measurements of productivity and work quality to support the performance of our program.

Many utilities have not adequately developed the tools or processes to answer most of those questions with true confidence. Many feel that they know how well crews are performing through their routine field visits and observations, but if pressed, they could not provide objective, documented, meaningful measurements of productivity, work quality and contract compliance to support their opinions. Often, closer examination or problems that surface unexpectedly usually find many unrecognized or unaddressed challenges.

David Van Bossuyt of Portland General Electric said, “You don't get what you expect, you get what you inspect.” This motto has been proven time and again when audits are conducted for programs that did not regularly conduct credible, comprehensive field audits and enforce audit findings. Initial field audits find avoidable and often significant costs, unnecessarily high tree-related outage rates, preventable accidents and injuries, and an unreasonably high number of justifiably dissatisfied customers.

To be successful, a field-auditing program must include field-based reviews using an effective field-auditing tool conducted by qualified auditors at an appropriate frequency and intensity, properly focusing on the audit as a positive process.

A well-designed, executed and documented field audit system will provide many benefits, including:

• Clear communication of performance expectations to workers

• Timely and effective identification and resolution of performance challenges before they become significant or public issues.

• Consistently satisfactory or exceptional field performance.

• Visible goals and targets that will energize workers and improve morale.

• A strong foundation for performance-based payments.

• A reliable means to track and document performance data that can help identify and support resource needs.

Field-Based Reviews

Audits must be conducted through on-site, field observations and measurements. Many utilities audit field performance from the office using data collected from crew time sheets, outage reports, expenditures or other records. Although these records provide some measure of performance, they do not fully identify field challenges in a timely manner nor do they distinguish the root causes of program success or failure.

Auditing restricted to office-based record review without comprehensive, accurate field validation can be deceiving. Many work crews are notorious for poor record keeping. While they may be great tree workers, they are often not interested or proficient recorders. In some instances, especially if they believe there will be no field validation, crews have been known to falsify records to enhance perceived performance. Tree-related outage performance can be affected by line construction modifications, weather or other external factors that have no direct relationship to the performance of the vegetation management program. Program expenditures can be affected by several external factors, such as rate variations between contractors, which do not accurately measure performance. Expenditures do not measure work-quality factors and can be easily manipulated to provide a desired productivity result.

Effective Field Auditing Tool

An effective, accurate field audit tool includes the following components:

• Clearly defined rating items directly focused on program goals and objectives. Primary vegetation management goals and objectives are safety, electric service reliability, reasonable cost-containment and customer satisfaction.

  Once key vegetation management goals and objectives have been identified, meaningful field results must be established that clearly measure their successful accomplishment.

• Clear objective quantitative measurements for each rating item. Each criterion must be able to be reasonably observed and clearly measured in the field. Subjective measurements must be avoided. Rating item criterion can include such things as proper use of personal protective equipment, regrowth time measurements of the distance between vegetation and conductors, the number of trees pruned each day, the number and magnitude of improper pruning cuts, and errors in written records.

• Defined thresholds of satisfactory and exceptional performance. The minimum level of satisfactory compliance must be established to clearly communicate performance expectations and provide a basis for assessment. Defining a level of exceptional performance provides workers with a higher goal for which to strive.

  Thresholds must reflect fair standards based on reliable historical performance and measured average performance of satisfactory work crews. Many items can be based on established, unambiguous contract specifications, accepted industry standards or practices, and laws/regulations. Others, notably productivity, require a more collaborative approach where the utility and the field work forces develop and manage a mutually acceptable standard based on reasonable, supported performance levels.

  When establishing thresholds, it is important to ensure that variable field conditions are considered and gauged during field reviews. For example, productivity can be significantly impacted by a number of factors, including but not limited to access, obstacles under trees, the size of trees, travel time, traffic, weather and sensitive customers. An experienced, qualified arborist can develop the appropriate modifiers required to effectively account for these factors.

• Appropriate weight on each rating item relative to its contribution and importance to program goals and objectives. To fully and most effectively measure performance, several items must be assessed and evaluated. Although each item must be important, each does not contribute equally to accomplishing the goals and objectives of the program.

The items that most impact the achievement of a utility's vegetation management goals are productivity, the amount of line clearance achieved and the use of proper arboricultural techniques. These factors should receive a higher weight within the rating scale than other important measurements, such as communication and site cleanup. To be considered satisfactory or superior performers, crews should always be required to perform at least at the minimum satisfactory level for all rating items. Although safety is not a primary performance factor, any unsatisfactory measurements should be addressed.

Conducting Audits

Auditors must be experienced and knowledgeable in the tree and vegetation management operations being audited. Auditors should be International Society of Arboriculture Certified Arborists. Auditors should demonstrate that they are qualified through experience in performing the work being audited and that they possess an understanding of and can apply proper work methods and techniques under the variety of conditions encountered in the field.

Actual auditing frequency will vary according to the experience and performance of each crew. New crews, training crews or crews that are encountering performance challenges should be audited on a weekly basis until performance is consistently satisfactory. Experienced crews that consistently demonstrate satisfactory performance should be audited at least once per month.

Approximately 10% of a crew's work during the audit period should be reviewed and measured. The variability of tree populations and work conditions are such that a 10% sample will provide a reasonable cross-section of work and a relatively high confidence level in the result.

A Positive Approach to Audits

Audits are typically met with resistance and negativity from the recipient. This is because most audits are focused on identifying primarily what is wrong, don't appear to balance the positive results, and frequently result in penalties. Few audits seem to identify good or exceptional performance or provide rewards.

Auditing must not be used as a weapon against crews or contractors. Although it must be clear that consistently poor performance is unacceptable and must result in consequences, an audit system is best used as a communication and training tool to meet and exceed program expectations. Consistently exceptional performance should result in rewards.

An audit program with a positive approach will more effectively improve performance and increase field crew morale. Performance is improved through the audit's ability to quickly identify and focus on specific challenges. Morale is enhanced as a result of clear communication from management and a regularly focused attention on each crew's work product resulting in the crew's awareness that its contribution is important and that management cares about the crew's work. Morale is also enhanced by the establishment of measurably higher standards for exceptional performance that challenge workers and provide them with meaningful targets that result in higher job satisfaction and then recognition of consistent exceptional performance.

Regular, comprehensive and objective performance auditing by both the utility and contractor will ensure maximum, reasonable levels of electric service reliability while reducing the risk and occurrence of preventable accidents. A well-developed and executed auditing program also minimizes costs, enhances customer satisfaction and increases crew morale.

Kevin K. Eckert is president and managing director of Arbor Global LLC, which provides arboriculture and vegetation management consulting services. Eckert previously served as system forester for Hawaiian Electric Co. and Eastern Utility Associates, and right-of-way forester for New England Electric System. Eckert has degrees in Forestry and Park Management from West Virginia University. He is a past president of the ISA Utility Arborist Association, chair of the Hawaii Urban and Community Forestry Program, and chair of the Western Chapter Arborist Certification Committee. He is a past chair and vice chair of the EEI Vegetation Management Task Force. keckert@arborglobal.com