2014-2018 Assessment of Automated M&V Methods

Summary of Work

With the methods used to date, evaluation, measurement and verification (EM&V) constitutes a significant portion of the total costs of energy efficiency projects and programs. It also requires time-consuming data acquisition (building, system and operational data as well as the actual energy used). Confounding the difficulties of current practice further, multiple approaches have produced a spectrum of savings calculation approaches, with some relying more heavily on measured data and others relying more heavily on estimated, modeled, or stipulated data. These latter approaches, while often speeding results or reducing costs, may not accurately reflect the actual project or program and thereby introduce other uncertainties into the results.

There is a rising availability of energy usage data from "Smart" meters and devices providing energy usage data of at least hourly time resolution. The cost of technologies for metering energy usage and sensing other key parameters affecting energy use in buildings is falling rapidly, and the prevalence of high-resolution meters is growing. These technologies and continued increases in the availability of inexpensive computing power open the door to collect and analyze the data needed for M&V more quickly and at dramatically lower cost, with comparable or even improved accuracy. Today, commercially available and open-source analytics tools can be used to streamline or automate the process to determine baseline energy consumption and calculate savings relative to that baseline.

Streamlined or automated M&V strategies provide a means to estimate savings for whole-building-focused energy efficiency approaches. There is growing recognition that whole-building-focused approaches to energy efficiency hold great promise in realizing deep and persistent energy savings in commercial buildings. Owners, property and facility managers, and utility incentive programs are increasingly adopting and piloting multi-measure strategies that move beyond traditional component-based or one-time commissioning or retrofit interventions; the industry is seeing a movement toward continuous energy improvement practices that may include efforts such as ongoing commissioning, strategic energy management, and operational optimization, as well as retrofits and the implementation of advanced control and information technologies.

Although today's tools hold great promise in reducing the cost and time required for M&V in the commercial buildings sector, several questions relating to savings quantification remain to be answered: What performance testing procedures and metrics should be used to assess the accuracy of these tools? How can tools be compared and contrasted? With what levels of uncertainty and confidence can today's tools quantify gross savings, and what will industry require as indication of a good result? How can practitioners leverage automation to streamline and scale the M&V process, while still using their expertise to retain a quality result — what workflows and processes can be established?

To answer these questions, LBNL is pursuing the following:

  1. Develop, apply, and publish a test procedure to compare and contrast the predictive accuracy of automated tools (M&V2.0 tools) to determine gross energy savings. The test procedure establishes metrics and benchmarks to assess the general robustness of both proprietary and open M&V algorithms.

    The test procedure is described and applied to monthly and interval open source models in [Granderson et al. 2015 and ACEEE Summer Study Presentation 2014; it is replicated with a more diverse test data set, consensus-based metrics, and both open and proprietary tools in [Granderson 2016a and 2015 webinar presentation].

  2. Demonstrate the use of promising M&V tools on historic program data in partnership with partners from the utility program administration, regulation, and implementation communities. These demonstrations include quantification of uncertainty due to model error, and project-specific as well as portfolio-level aggregated results. In this work we also begin to determine labor time savings and results with respect to 'traditional' M&V. Findings were published in [Granderson 2017]. Case studies on Efficiency Vermont's and BC Hydro's work in this area is also available.
  3. In collaboration with industry stakeholders in diverse regions across the US, work to identify documentation and reporting requirements for the use of these analytic-based approaches to M&V [Draft guidance document and summary presentation]. These materials reflect ongoing dialogue to determine accuracy requirements, as well as qualitative guidance on transparent documentation for evaluation of results; they will be updated as best practice evolves.
  4. Following successful demonstrations on historic program data, provide technical assistance to design and execute more structured pilots. A pilot in Connecticut is being conducted in collaboration with CT DEEP, Eversource, and United Illuminating. A pilot in Seattle is being conducted in collaboration with Seattle City Light and Bonneville Power Administration. The pilots are designed to more formally test the value proposition associated with these tools. They are described further in a Fact Sheet.

Additional Project-Specific Documents

The State of Advanced Measurement and Verification Technology and Industry Application
Granderson, J., S. Fernandes. The Electricity Journal 30 (2017) pp.8-16.
PDF, 586 KB
DOE M&V 2.0 Project Overview, 2017 PDF, 1.7 MB
The Status and Promise of Advanced M&V (Rocky Mountain Institute; co-authored by LBNL), 2017 PDF, 12.5 MB
Will the Measurement Robots Take Our Jobs? An Update on the State of Automated M&V for Energy Efficiency Programs
Granderson, J., Touzani, S., Taylor, C., Fernandes. S., Lawrence Berkeley National Laboratory, August, 2016. LBNL-1006283.
PDF, 1.2 MB
Presentation on Industry and R&D Needs: Defogging Key Issues in M&V 2.0, 2016 PDF, 4.63 MB
Presentation on testing procedure and accuracy of 10 public and proprietary baseline models, 2015 PDF, 1.1 MB


For more information contact .