Summary of Work
The growing availability of smart meter energy data and energy analytics software presents tremendous promise to accurately measure project energy savings in near real-time. Advanced M&V (also known as M&V 2.0) can also capture changes to a building's hourly loadshape, of growing importance as regulators, utilities, and owners recognize the time-varying value of energy efficiency.
Advanced M&V software tools have been developed and demonstrated on a small scale, but questions remain to be answered before reaching scaled market adoption.
- What performance testing procedures and metrics should be used to assess the accuracy of these tools?
- With what levels of uncertainty and confidence can today's tools quantify savings, and what will industry require as indication of a good result?
- How can practitioners leverage automation to streamline the M&V process while using their expertise to attain a quality result? Below find summaries of Berkeley Lab research to address these questions:
- M&V tool testing: Berkeley Lab developed a test procedure to objectively compare and contrast the accuracy of advanced M&V tools. The test procedure establishes metrics and benchmarks to assess the general robustness of both proprietary and open M&V algorithms. The test procedure is described and applied to monthly and interval open source models in [Granderson et al. 2015 and ACEEE Summer Study Presentation 2014; it is replicated with a more diverse test data set, consensus-based metrics, and both open and proprietary tools in [Granderson 2016a and 2015 webinar presentation]. We are currently collaborating with an industry partner as they work to launch an online portal incorporating the test method.
- Accuracy, documentation, and reporting requirements: In collaboration with industry stakeholders in diverse regions across the US, Berkeley Lab is working to identify documentation and reporting requirements for the use of these analytic-based approaches to M&V [Draft guidance document and summary presentation]. These materials reflect ongoing dialogue to determine accuracy requirements, as well as qualitative guidance on transparent documentation for evaluation of results. They will be updated as best practice evolves.
- M&V 2.0 pilots: Following successful demonstrations on historic program data, Berkeley Lab is providing technical assistance to design and execute more structured pilots:
- A pilot in Connecticut is being conducted in collaboration with CT DEEP, Eversource, and United Illuminating (Factsheet).
- A pilot in Seattle is being conducted in collaboration with Seattle City Light and Bonneville Power Administration (Factsheet coming soon).
- A pilot in California is being conducted in collaboration with the California Public Utilities Commission (Factsheet).
- A pilot in Sacramento is being conducted in collaboration with Sacramento Public Utilities District (Factsheet).
- The pilots are designed to more formally test the value proposition associated with these tools. Additional case studies documenting advanced M&V application collaborations with Efficiency Vermont and BC Hydro are provided in the table below.
- RM&V Reference Tool: In 2017, Berkeley Lab published the open source M&V tool "RM&V," a reference implementation of advanced M&V analytical best practices. RM&V allows for multi-site analysis, model goodness of fit screening, and support to visualize savings as they accrue and identify potential non-routine events. The tool offers two modeling options: a piecewise linear regression that uses time of week and temperature ('TOWT') as driving variables, and a machine learning method. More detail on the TOWT modeling method can be found in this 2011 report, and a link to a paper on the gradient boosting machine learning method is included in the table below.
|Title and Description||Resource Type||Date|
Statistical Change Detection of Building Energy Consumption: Applications to Savings Estimation
A key challenge when applying advanced M&V is to identify and adjust for non-routine events (NREs), i.e., changes in building energy consumption unrelated to the project for which savings are being quantified. In this work, a data driven methodology was developed to (partially) automate, and therefore streamline the process of detecting NREs in the post-retrofit period and making associated savings adjustments.
Gradient Boosting Machine for Modeling the Energy Consumption of Commercial Buildings
The gradient boosting machine is a machine learning algorithm that is gaining considerable traction in a wide range of data driven applications. In this journal article an energy consumption baseline modeling method based on a gradient boosting machine was proposed and assessed.
The State of Advanced Measurement and Verification Technology and Industry Application
Article characterizes the features of sixteen automated M&V software tools, and reports on status of utility programs implementing whole building savings approaches.
DOE M&V 2.0 Project Overview
Powerpoint presentation to industry stakeholders, providing an overview and recent results of Berkeley Lab's research on advanced M&V.
The Status and Promise of Advanced M&V
The paper identifies the benefits, methods, and requirements of advanced M&V and outlines key technical issues for applying these methods. It presents an overview of the distinguishing elements of M&V 2.0 tools and of how the industry is addressing needs for tool testing, consistency, and standardization, and it identifies opportunities for collaboration.
BC Hydro to Deliver M&V 2.0 to Customers and Program Administrators (Case Study)
In working to develop a customer-facing M&V 2.0 tool, BC Hydro partnered with researchers at Berkeley Lab to analyze a set of historic program data.
Efficiency Vermont Takes Action to Apply M&V 2.0 (Case Study)
With a desire to develop an M&V 2.0 platform for program delivery, Efficiency Vermont worked with researchers at Berkeley Lab to test the performance of M&V 2.0 open-source algorithms against a small set of historic program data. Parallel analyses were conducted by VEIC and the Berkeley Lab team to ensure repeatability of results and correctness of code implementation
Will the Measurement Robots Take Our Jobs? An Update on the State of Automated M&V for Energy Efficiency Programs
This ACEEE Summer Study paper details metrics to assess the performance of advanced M&V approaches, a framework to compute the metrics, and recent test results. We also discuss the accuracy, cost, and time trade-offs between advanced and traditional M&V. The paper also includes early results of advanced M&V pilot.
Presentation on testing procedure and accuracy of 10 public and proprietary baseline models
Presentation gives an overview of the test method developed by Berkeley Lab to assess baseline modeling capability for M&V software tools. The test method was applied to ten modeling methods/tools with results presented here. Additional links are provided in item 1 above.