2014-2017 Assessment of Automated M&V Methods

Summary of Work

With the methods used to date, evaluation measurement and verification (M&V) constitutes a significant portion of the total costs of energy efficiency projects and programs. It also requires time-consuming data acquisition (building, system and operational data as well as the actual energy used). Confounding the difficulties of current practice further, multiple approaches have produced a spectrum of savings calculation approaches, with some relying more heavily on measured data and others relying more heavily on estimated, modeled, or stipulated data. These latter approaches, while often speeding results or reducing costs, may not accurately reflect the actual project or program and thereby introduce other uncertainties into the results.

There is a rising availability of energy usage data from "Smart" meters and devices providing energy usage data of at least hourly time resolution. The cost of technologies for metering energy usage and sensing other key parameters affecting energy use in buildings is falling rapidly, and the prevalence of high-resolution meters is growing. These technologies and continued increases in the availability of inexpensive computing power open the door to collect and analyze the data needed for M&V more quickly and at dramatically lower cost, with comparable or even improved accuracy. Today, commercially available and open-source analytics tools can be used to streamline or automate the process to determine baseline energy consumption and calculate savings relative to that baseline.

Streamlined or automated M&V strategies provide a means to estimate savings for whole-building-focused energy efficiency approaches. There is growing recognition that whole-building-focused approaches to energy efficiency hold great promise in realizing deep and persistent energy savings in commercial buildings. Owners, property and facility managers, and utility incentive programs are increasingly adopting and piloting multi-measure strategies that move beyond traditional component-based or one-time commissioning or retrofit interventions; the industry is seeing a movement toward continuous energy improvement practices that may include efforts such as ongoing commissioning, strategic energy management, and operational optimization, as well as retrofits and the implementation of advanced control and information technologies.

Although today's tools hold great promise in reducing the cost and time required for M&V in the commercial buildings sector, several questions relating to savings quantification remain to be answered: What performance testing procedures and metrics should be used to assess the accuracy of these tools? How can tools be compared and contrasted? With what levels of uncertainty and confidence can today's tools quantify gross savings, and what will industry require as indication of a good result? How can practitioners leverage automation to streamline and scale the M&V process, while still using their expertise to retain a quality result — what workflows and processes can be established?

To answer these questions, LBNL is pursuing the following:

  1. Develop, apply, and publish a test procedure to compare and contrast the predictive accuracy of automated tools (M&V2.0 tools) to determine gross energy savings. The test procedure establishes metrics and benchmarks to assess the general robustness of both proprietary and open M&V algorithms.

    The test procedure is described and applied to monthly and interval open source models in [Granderson et al. 2015 and ACEEE Summer Study Presentation 2014; it is replicated with a more diverse test data set, consensus-based metrics, and both open and proprietary tools in [Granderson 2016a and 2015 webinar presentation].

  2. Demonstrate the use of promising M&V tools on historic program data in partnership with partners from the utility program administration, regulation, and implementation communities. These demonstrations include quantification of uncertainty due to model error, and project-specific as well as portfolio-level aggregated results. In this work we also begin to determine labor time savings and results with respect to 'traditional' M&V. Preliminary findings were published in [Granderson 2016b] and expanded in a presentation at the 2016 ACEEE Summer Study. Case studies on Efficiency Vermont's and BC Hydro's work in this area is also available.
  3. In collaboration with industry stakeholders in diverse regions across the US, work to identify documentation and reporting requirements for the use of these analytic-based approaches to M&V. Currently underway, this includes dialogue to determine acceptable levels of statistical confidence and uncertainty, as well as qualitative guidance on transparent documentation for evaluation of results.
  4. Following successful demonstrations on historic program data, provide technical assistance to design and execute more structured pilots in key locations across the US. The pilots will be designed to more formally test the value proposition associated with these tools. For example through side-by-side comparison with traditional approaches, as projects are implemented, we will evaluate trade offs between time, cost, accuracy, and the value of continuous savings feedback.

Additional Project-Specific Documents

Presentation to NEEP EM&V Forum, June 2016 PDF, 4.63 MB
Presentation on testing procedure and accuracy of 10 public and proprietary baseline models, 2015 PDF, 1.1 MB
Presentation at ACEEE Intelligent Efficiency 2014 PDF, 735 KB
Presentation at ACEEE Summer Study 2014 PDF, 868 KB


For more information contact .