About the Client
The company is one of the largest electric power holding companies in the United States, providing electricity to several million retail customers in six states. They are focused on transforming their customers’ experience, modernizing their energy grid, generating cleaner energy, and expanding their natural gas infrastructure to create a smarter energy future.
Our client was interested in extending their demand response (DR) programs to include small businesses—a segment they had not involved in these programs to-date. Small business participation would provide a positive impact on the grid, while also providing participants with the same incentives being offered to residential customers and large businesses engaged in similar DR programs.
The company developed a program that provided small business customers with a free programmable, two-way Wi-Fi thermostat (or load switch device, based on customer preference or thermostat compatibility issues) as well as financial incentives for participating in demand response events called Conservation Periods (summer and winter). Participants earned a bill credit for participating in Conservation Periods at the end of each season, related to one of three Cycling Levels they chose when they signed up. The Cycling Levels of 30%, 50%, and 75% represent the percent reduction in air conditioning operation during a Conservation Period, with higher bill credits for higher Cycling Levels.
As part of a multi-firm evaluation team, EMI Consulting was brought in to conduct the first process evaluation of this innovative new program.
We began by cataloguing questions about program design and delivery from our client and their implementers to determine what information would be most helpful in ensuring the success of the program. As this was a new program that had not been implemented by other utilities at the time of our evaluation, many of these questions centered around the customers’ experiences throughout the program: What motivates customers to enroll in the program? Do customers understand how to use their smart thermostats? Do customers understand the purpose and requirements of the program when they enroll? How satisfied are customers with the program and program elements? And what barriers prevent customers from participating? Once we understood the needs of the program, we then created our research methodology, which included:
- Primary Data Collection: (1) program staff interviews, (2) early participant interviews, (3) a participant survey, (4) non-participant interviews, (5) unenrolled (“drop-out”) participant interviews
- Secondary Data Review: (1) program tracking data, (2) review of program documentation and collateral
From this research, we uncovered a number of important lessons, which are both unique to this audience and important to recruitment and implementation. For instance, program staff found it easier to recruit if they explained the energy savings and other benefits of the free thermostat before moving on to explain Conservation Periods and Cycling Levels. While this strategy was successful in helping the program exceed enrollment goals, customers possessed lower levels of awareness and understanding of the demand response elements of the program, including Cycling Levels and events, relative to thermostat and energy efficiency program elements. For example, most participants were unaware of their Cycling Level.
Also, while participants were highly satisfied with their thermostats and the program itself, it seemed significant that some customers did not fully understand the DR elements of the program. Customers who understood events and cycling better reported better experiences with the program and were more likely to continue to participate in the future. On the flip side, customers who dropped out of the program reported a poorer understanding of events and cycling than those who continued participating in events. These customers reported that increased indoor temperatures during events had too great of an impact on their comfort.
Despite the challenges some businesses had with events, participants tended to value the functionality of the smart thermostat, including the ability to control multiple thermostats at the same time or to control temperatures remotely in the event an employee overrode a setting. Most participants signaled that they were satisfied with the program overall.
Our research also suggested the benefit of using a WiFi thermostat—which customers already understand and value—as an effective tool for introducing customers to unfamiliar demand response programs. However, there is risk in relying too heavily on energy efficiency and smart thermostats without fully explaining demand response or vetting participants. Namely, the program may enroll customers who are not a good fit, do not fully understand how to pick the appropriate Cycling Level, or are unaware of behavioral strategies to help mitigate the effects of events on their comfort.
The program was a first step for our client to bring DR programs to small businesses, and our work clearly demonstrated both the customers’ impressions of the program and lessons that can be applied going forward. Our research has allowed our client to keep in front of this kind of offering by identifying and accounting for complications as they arise. Importantly, we helped our client understand the need for improving awareness and education of demand response, cycling levels, and behavioral strategies that could help those customers whose comfort was affected during events.