About the Client
The client is one of the largest electric and natural gas utilities in the US with service territory across five states. They have a long history of offering innovative energy efficiency programs and are a leader in program design and implementation.
The ESRPP Program is a nationally-coordinated, midstream program design aimed at influencing retailers to alter their product assortment and to sell, promote, and demand more energy-efficient models of home appliances in specific product groups (e.g., clothes washers, dryers, and refrigerators). Our client and other organizations (“Program Sponsors”) across the U.S. have partnered to develop and implement ESRPP. Each participating Program Sponsor pays participating retailers per-unit incentives for every program qualified unit they sell in each program category. The program theory holds that, by increasing the sales of energy-efficient models over less efficient models, the ESRPP Program will generate energy and demand savings for utility customers in the short-, mid-, and long-term through participating retailers, while also transforming the overall market towards higher efficiency in the long-term.
EMI Consulting was asked to provide a utility with key information to inform decisions about how to administer ESRPP, specifically by providing an assessment of the prospects for cost-effectiveness in each product group. The utility had developed seven possible scenarios, ranging from “conservative” to “aggressive” for certain key program elements, such as incremental measure cost, sales volume increases, and unit savings, in an Excel-based tool.
To help the utility develop a deeper understanding of the potential benefits of the program, EMI Consulting conducted a Monte Carlo simulation of the cost-effectiveness for each product group. To achieve this, EMI Consulting first replicated the utility’s cost-effectiveness calculations as an R-based tool, then simulated market outcomes by using variation in the sales data, the measurement uncertainty in sales increase rates, and empirically-based scenarios of program effects over time to conduct a simulation of the distribution of outcomes. The Monte Carlo analysis mimicked traditional analyses previously done by using cost-effectiveness assumptions generated by the utility, but it allowed for greater nuance in the scenarios by using random draws of sales volume simulations in each product category and by allowing program costs to vary at different rate to understand a full range of possible program variation.
By running over one million individual simulations across product groups, EMI Consulting was not only able to provide the utility with a single estimate of cost-effectiveness, we were able to provide both an average estimate of cost-effectiveness and information about how much uncertainty there was in that estimate so that the utility could understand the full range of outcomes and the risk associated with each product group. And because there were so many different inputs and scenarios, we also developed an R-based analysis tool to help the utility investigate the results and understand the distribution of outcomes for each scenario or set of scenarios.
This Monte Carlo simulation provided the utility client with the information required to meet a wide array of needs, including the ability to answer specific questions that executives would have about the impact of the program on the overall portfolio, based on real-world insight about the market. This type of approach could be useful any time there is uncertainty about how program and measure costs are going to change and when the extent of market uptake is unknown, such as launching new energy efficiency pilots or programs, or expanding electric vehicle charging infrastructure. For program evaluation, a Monte Carlo simulation could support the development and analysis of baseline market scenarios. Although the degree to which the results reflect reality is dependent on what is known about how the values will change, even coarse ranges of inputs can provide more meaningful bounds on the realm of possible outcomes than the simplistic approaches of the past that have lacked reasonable variation in important market characteristics.