The use of unstructured data combined with machine learning and data fusion techniques, has the potential to enhance today’s energy analysis tools and provide asset information for public and private sector building portfolios. This project will develop automated approaches to determine building characteristics, and retrofit and operational efficiency opportunities.
State of the art analytics software and modeling tools can provide valuable insights into efficiency opportunities. However prior research has shown that key barriers include relatively limited data sources (smart meters and weather being most common in commercial tools), or reliance upon user-provided inputs for which default values may be the fallback. There is great opportunity to apply techniques based on multi-stream data fusion and machine learning to overcome these challenges. Secondarily, these techniques can be applied to provide enhanced asset information for public and private sector building portfolios.
This project is developing automated approaches to determine building characteristics and efficiency opportunities using unstructured data from a subset of the following types:
1) Drone-based RGB images
2) Drone-based thermal images
3) Satellite or aerial images
4) Building footprint GIS data
Year 1 Research Outcomes are detailed in this deck and summarized below
1) Opportunity analysis to apply machine learning to enhance building energy-efficiency measure identification.
We gathered information regarding data availability, providers, and methods of collection for ~27 novel data sources (public, proprietary, private, or researcher datasets) and assessed how these data can be used to enhance identification of ~18 building energy use and asset features (such as equipment, envelope, interior, context, occupancy, utility cost etc.), and ~70 building energy efficiency measures.
The analysis surfaced satellite/aerial data, building footprints GIS data, and drone-based visible and thermal images as the most promising data sources to pursue via machine learning methods.
2) Replicable, open solutions for ML-based satellite/aerial image feature extraction
The open source Automated Building Footprint Extractor codebase is available at Github with three modules:
- Module to generate training data for deep reinforcement learning algorithms to determine building footprints from satellite data
- Module for deep learning semantic segmentation
- Module to post-process building footprints into GIS-Format (i.e., GeoJson)
3) Foundation for Drone-based Generation of 3D Geometry
We designed a drone system, defined the data acquisition methodology, and developed a workflow to automatically extract building 3D geometry.
First Fuel, NYSERDA, Signetron, TRC