Advanced Measurement & Verification (M&V) Research

Advanced Measurement & Verification (M&V) Research

Energy management and information systems icon

Summary of Work

The growing availability of smart meter energy data and energy analytics software presents tremendous promise to accurately measure project energy savings in near real-time using advanced M&V methods. The last decade has seen significant progress to develop advanced M&V tool testing protocols, set guidance around accuracy and documentation, and to demonstrate the practical application of advanced M&V through pilots. Recent research has begun to explore new opportunities and remaining technical challenges, including: 

  • How effectively can advanced M&V methods capture the timing and location of energy impacts for emerging demand-side management strategies such as load shed (demand response) and load shift, at the building level and at grid level?
  • How can advanced M&V tools and guidelines account for energy impacts in buildings that are unrelated to the interventions targeted for M&V (so-called “non-routine events”)?
  • With what levels of uncertainty and confidence can today's advanced M&V tools quantify savings, and what will industry require as indication of a good result?
  • How can practitioners leverage automation to streamline the M&V process while using their expertise to attain a quality result using accurate M&V tools?

Below find summaries of Berkeley Lab's past and ongoing research to address these questions:

1. Grid services support: Berkeley Lab collaborated with Sacramento Municipal Utility District (SMUD) to characterize the impact of SMUD’s energy efficiency program portfolio at grid level. The results of the research demonstrated that advanced M&V methods can quantify hourly savings shape at substation/feeder level and seasonally (documented in a 2020 ACEEE Summer Study paper and a journal article). Follow up work has explored the accuracy of advanced M&V methods in calculating a baseline for load shed or demand response applications (documented in a public webinar deck; journal article under development).

2. Non-routine events: In collaboration with industry stakeholders, Berkeley Lab has been working on research, guidance, and tools to handle non-routine events (NREs). A 2020 ACEEE paper addressed fundamental questions around frequency, magnitude, and directionality of NREs (2020 ACEEE Summer Study paper and presentation). We also collaborated with the Efficiency Valuation Organization (EVO) on an application guide for identifying and accounting for NREs (publication coming soon). We also developed and tested algorithms for detecting NREs and semi-automated non-routine adjustments (journal article).

3. M&V tool testing: Berkeley Lab developed a test procedure to objectively compare and contrast the accuracy of advanced M&V tools. The test procedure establishes metrics and benchmarks to assess the general robustness of both proprietary and open M&V algorithms. The test procedure is described and applied to monthly and interval open source models in (Granderson et al. 2015 and ACEEE Summer Study Presentation 2014); it is replicated with a more diverse test data set, consensus-based metrics, and both open and proprietary tools in (Granderson 2016a and 2015 webinar presentation). In 2019 we collaborated with the Efficiency Valuation Organization (EVO) to launch an online portal incorporating the test method.

4. Accuracy, documentation, and reporting requirements: In collaboration with industry stakeholders in diverse regions across the US, Berkeley Lab worked to identify documentation and reporting requirements for the use of these analytic-based approaches to M&V (guidance document and summary presentation). More recently, we collaborated with EVO to develop a 2020 white paper that provides a snapshot of the current state of advanced M&V practice (white paper), which preceded the co-development of an EVO advanced M&V application guide (publication date TBD). We are also studying methods for quantifying uncertainty of advanced M&V savings estimates (journal paper published, and more recent work ongoing).

5. M&V 2.0 pilots: Following successful demonstrations on historic program data, Berkeley Lab provided technical assistance to design and execute more structured pilots:​

  • A commercial pilot in Connecticut is being conducted in collaboration with CT DEEP, Eversource, and United Illuminating (Factsheet). The project also included a residential M&V pilot (Factsheet coming soon).
  • A pilot in Seattle is being conducted in collaboration with Seattle City Light and Bonneville Power Administration (Project Final Report here).
  • A pilot in California is being conducted in collaboration with the California Public Utilities Commission (Factsheet).
  • A pilot in Sacramento is being conducted in collaboration with Sacramento Public Utilities District (Factsheet). A February 2020 webinar deck on grid-level analysis results from this pilot can be downloaded here.

The pilots were designed to more formally test the value proposition associated with these tools. Additional case studies documenting advanced M&V application collaborations with Efficiency Vermont and BC Hydro are provided in the table below.​​

6. RM&V Reference Tool: In 2017, Berkeley Lab published the open source M&V tool "RM&V," a reference implementation of advanced M&V analytical best practices. RM&V allows for multi-site analysis, model goodness of fit  screening, and support to visualize savings as they accrue and identify potential non-routine events. The tool offers two modeling options: a piecewise linear regression that uses time of week and temperature ('TOWT') as  driving variables, and a machine learning method. More detail on the TOWT modeling method can be found in this 2011 report, and a link to a paper on the gradient boosting machine learning method is included in the table below.

Additional Resources

Title and Description Resource Type Date

Spatio-temporal impacts of a utility’s efficiency portfolio on the distribution grid

This work demonstrates that advanced measurement and verification modeling techniques can be used to estimate the spatio-temporal grid impact of a portfolio of energy efficiency programs. It extends measurement-based methods to an entire Demand Side Management portfolio and uses a single model to predict annual as well as seasonal building energy use with near-zero bias.

Journal Article 2020

Detecting the undetected: Handling non-routine events for meter-based savings approaches

This paper documents the state of the art in NRE quantification and analysis. The results of research to quantify the frequency, nature and direction of NREs, and methods and metrics for determining a trigger threshold for taking action on NREs are presented. The paper also documents the latest technical guidance on application of NRE detection and adjustment methods.

Conference Paper 2020

Meter-Based Assessment of the Time and Locational Benefits of a Large Utility’s DSM Portfolio

This paper presents an analysis of interval data across more than 25,000 customers and twelve substations, from the Sacramento Municipal Utility District. The results show for different locations on the grid: achieved savings and the impact on grid consumption; hourly savings shapes for DSM program participants and non-participants, and how those shapes vary with season; and the impact of the programs on peak demand

Conference Paper 2020

Realizing high-accuracy low-cost measurement and verification for deep cost savings

This report summarizes the key outcomes of a multi-year project, including a literature review of M&V tools, development of data-driven techniques for identifying and adjusting for non-routine events, pilot deployment of advanced M&V methods in the Pacific Northwest, and engagement with regulatory bodies on the topic of advanced M&V.

Project Report 2019

Evaluation of Methods to Assess the Uncertainty in Estimated Energy Savings

This paper evaluates the accuracy of methods used to quantify the uncertainty of measurement & verification (M&V) savings estimates. The findings of this work indicate that standard methods for  estimating savings uncertainty over the post-installation period tend to underestimate the uncertainty, and that this tendency is stronger for hourly models than for daily models.

Journal Article 2019

Statistical Change Detection of Building Energy Consumption: Applications to Savings Estimation

A key challenge when applying advanced M&V is to identify and adjust for non-routine events (NREs), i.e., changes in building energy consumption unrelated to the project for which savings are being quantified. In this work, a data driven methodology was developed to (partially) automate, and therefore streamline the process of detecting NREs in the post-retrofit period and making associated savings adjustments.

Journal Article 2019

Gradient Boosting Machine for Modeling the Energy Consumption of Commercial Buildings

The gradient boosting machine is a machine learning algorithm that is gaining considerable traction in a wide range of data driven applications. In this journal article an energy consumption baseline modeling method based on a gradient boosting machine was proposed and assessed.

Journal Article

2018

The State of Advanced Measurement and Verification Technology and Industry Application

Article characterizes the features of sixteen automated M&V software tools, and reports on status of utility programs implementing whole building savings approaches.

Journal Article 2017

DOE M&V 2.0 Project Overview

Powerpoint presentation to industry stakeholders, providing an overview and recent results of Berkeley Lab's research on advanced M&V.

Presentation 2018

The Status and Promise of Advanced M&V

The paper identifies the benefits, methods, and requirements of advanced M&V and outlines key technical issues for applying these methods. It presents an overview of the distinguishing elements of M&V 2.0 tools and of how the industry is addressing needs for tool testing, consistency, and standardization, and it identifies opportunities for collaboration.

Paper 2017

BC Hydro to Deliver M&V 2.0 to Customers and Program Administrators (Case Study)

In working to develop a customer-facing M&V 2.0 tool, BC Hydro partnered with researchers at Berkeley Lab to analyze a set of historic program data.

Case Study 2017

Efficiency Vermont Takes Action to Apply M&V 2.0 (Case Study)

With a desire to develop an M&V 2.0 platform for program delivery, Efficiency Vermont worked with researchers at Berkeley Lab to test the performance of M&V 2.0 open-source algorithms against a small set of historic program data. Parallel analyses were conducted by VEIC and the Berkeley Lab team to ensure repeatability of results and correctness of code implementation

Case Study 2017

Will the Measurement Robots Take Our Jobs? An Update on the State of Automated M&V for Energy Efficiency Programs

This ACEEE Summer Study paper details metrics to assess the performance of advanced M&V approaches, a framework to compute the metrics, and recent test results. We also discuss the accuracy, cost, and time trade-offs between advanced and traditional M&V. The paper also includes early results of advanced M&V pilot.

Report 2016

Presentation on testing procedure and accuracy of 10 public and proprietary baseline models

Presentation gives an overview of the test method developed by Berkeley Lab to assess baseline modeling capability for M&V software tools. The test method was applied to ten modeling methods/tools with results presented here. Additional links are provided in item 1 above.

Presentation 2015

Contact

Mechanical Staff Scientist/Engineer
Program Manager
Program Manager
Senior Scientific Engineering Associate