The utility industry stands at a critical crossroads, grappling with outdated regulations and aging infrastructure while facing unprecedented challenges. As climate adaptation looms and surprises, community resilience wavers and utilities are torn between innovating ahead of disruptions and conserving cash for rainy days and investor returns. The majority see the need for increased flexibility and resilience, yet are hamstrung by regulatory frameworks ill-equipped for today's realities. And then, there is this:
Less than third of over 400 surveyed utility company leaders think that solution and innovative technology providers will help them navigate what is coming. Just over half think their regulators and governments are more worthy partners. This kind of finding makes my head spin, until I remember - oh, we have allowed monopolies to be monopolies.
Be that as it may: I think utilities are worrying too much about regulators and thus are ill-equipped to grasp how quickly they could adopt software, hardware, and data management tools in their backyard, fostered through the massive innovation drive of decentralized SaaS and customer choice providers, whether it is smart meter data access to support energy efficiency programs or figuring out how much headroom is on a transformer so that interconnecting DERs and EV charging stations can be located sensibly and managed in real-time as load flexibility assets.
According to Guidehouse Insights forecasts, “at least $1 billion will be invested annually in advanced distribution management systems by 2035 to improve grid reliability in North America.
At the same time, innovations in CE/CX technologies point to a digitally enabled flexibility transformation across the grid edge ecosystem.
Investments in behavioral and structural energy efficiency technologies that interact with behind-the-meter technologies like electric vehicles represent an exciting opportunity for utilities to better tailor incentives for their clients.
An estimated 90% of North American homes will be served by customer engagement and experience analytics programs by 2030, increasing from around 65% today.”
you will greatly identify with pretty much all of this.
Snapshots of PUF Guidehouse Survey - 2024 of over 400 Utility CEOS
400+ Surveyed Utility CEOS in 2024: Here is What They Think
Regulation:
42.7% believe the current utility regulatory landscape is "Behind the times, outdated and misaligned with today's system needs and investments."
67% think utilities can best serve customers by "Lead by example: Mobilize the public sector and regulators to innovate out in front of looming threats."
Stakeholder Prioritization:
56.4% of utilities prioritize partnering with "State and local government, including regulators" to navigate the Energy Transition.
Funnily enough, tech solutions providers and large C&I customers are two oddball friends in the same bucket of less-important allies.
Flexibility:
61.4% believe utility infrastructure investments should prioritize "Increasing flexibility to improve energy system resilience."
Resilience:
40% see "Aging infrastructure and decline in overall quality of life / community livability" as the most disruptive threat facing US utilities.
57.1% would like to see utilities build a "Storm-hardened, self-healing, and resilient delivery system providing 24/7 uninterrupted power" as the infrastructure-of-the-future vision.
Electric distribution utilities face significant challenges in adopting and integrating new technologies to modernize the grid and improve operations. Regulatory constraints, organizational inertia, procurement processes, and funding models impede technology deployment. There are however emerging opportunities and strategies to accelerate adoption, including regulatory reform, new partnership models, and innovative financing approaches.
PURPA Culture & California’s Dereg Frenzy Led to Enron, Left Utilities Behind on Software Tech
In a little twist, PURPA began an era of attitudinal shift around what utilities should be able to do as innovators vs. third party innovators: this was not productive in that it precluded the essential hauling of monopoly energy providers into a modern era. Sometimes, a historian can tell it best. For this story, it is electricity historian Dr. Gretchen Bakke and the particular leave-them-out effort in question, is California’s deregulation mandate in 1998, a piece of legislation that was “litlte more than a remarkable series of loopholes twisted through with some regulatory language…”
Bakke writes that “PURPA cracked open the utilities’ monopsomy control of the grid” and intitiated a grand phase of the end of of their writ-large natural monopoly over the means of production and delivery of electricity. However, “it did not change…the prevailing attitude” of lawmakers and the utilities that the only way to exert control over the behavior of the utilities was “through requiring absolute adherence to whatever happened to get legislated. Over time, this culture of obligation has had the unforunate effect of making the utilities even less capable of adapting to the rapidly changing electricity lanscape…”
When California implemented their pre-EPACT deregulation state bill, it obligated utilties to strict regulatory compliance and they also encountered the dual whammy of the internet and non-utility companies cow-boying into the technology software space which boomed alongside. The lopsided adoption of technology innovation freedoms provided by California law conferred benefits to third-parties but not to utilities; which allowed Enron to become a premier name in online energy futures and spot market commodity trading. Their schemes, as Dr. Bakke writes, include fun games with power like “Death Star”, “Ricochet”, and “megawatt laundering.” Aside from Enron, “smaller companies in San Francisco were doing the same or very similar kinds of transactions….”. Dr. Gretchen Bakke, The Grid.
I’ll agree with Dr. Bakke, that Enron’s exposure led to quiet shutdowns of far smaller tech company operations doing essentially the same thing. Also, California deregulation was not the only party at fault for Enron enablement. “In December 2000, Senator Gramm helped push a bill through Congress that deregulated trading in energy. Enron's electricity trading business swelled, and some of the firm's only real profits were made. Without owning a single California power plant, Enron came to control the state's market. Rolling blackouts became the norm, prices skyrocketed, and the state racked up billions in debt. Phil Gramm blamed environmentalists for the crisis. Finally, price controls were imposed and the bubble burst. Deprived of its cash cow, Enron hit the rocks a few months later. But of course the biggest profiteers were the synergists themselves, the investment banks and venture capitalists (VCs).” After the New Economy: The Binge . . . And the Hangover That Won't Go Away by Doug Henwood.
Another op-ed historian writes, “Mr. Gramm, a Texas Republican, is one of the top recipients of Enron largess in the Senate…Wendy Gramm has been influential in her own right. She, too, is a demon for deregulation. She headed the presidential Task Force on Regulatory Relief in the Reagan administration. And she was chairwoman of the U.S. Commodity Futures Trading Commission from 1988 until 1993. In her final days with the commission she helped push through a ruling that exempted many energy futures contracts from regulation, a move that had been sought by Enron. Five weeks later, after resigning from the commission, Wendy Gramm was appointed to Enron's board of directors. According to a report by Public Citizen, a watchdog group in Washington, ''Enron paid her between $915,000 and $1.85 million in salary, attendance fees, stock options and dividends from 1993 to 2001. [***] An article in yesterday's Times noted the extensive links between Enron and the powerful Texas congressman Tom DeLay. Mr. DeLay became unhappy when Enron wooed a Democrat -- a senior treasury official in the Clinton administration -- to run its Washington office. ''Still,'' the article said, ''whatever the tensions last year, Mr. DeLay and Enron had a natural alliance. In his days in the Texas Capitol, Mr. DeLay was called Dereg by some because of his support of business. And in Congress he has been a longtime proponent of energy deregulation, an issue dear to Enron.''” Enron and the Gramms, New York Times.
(In a rare moment, realizing Texas hopefully apologized to California for how many Texas politicians backed federal and state policies which enabled Enron’s conduct. Did California apologize to Californians?)
In its chaotic reversal of a dangerous and clumsy effort at deregulation, California forgot once more to bring utilities along for the ride in technology adoption and modernization. Dr. Bakke writes that “the state required the utilities to comply to its poorly crafted legislation while smaller, more flexible, more innovative companies ran rings around them both. This, was the result of the culture of PURPA, which presumed the stodginess and unadventurousness of the utilities to be inherent and thus unreformable characteristics of the industry. The utilities could be regulated, wisely or foolishly, but they would not be asked to learn adaptability, flexibility, or creativity themselves; instead they would be told what to do and they would be expected to do it. “
In a moment of thunderous revelation for myself, like all the stars in my head finally aligned themselves into the Biggest Dipper, Dr. Bakke concludes: “One result of this is that as the holes PURPA rent in the system have grown larger over the decades, the electrical utilities in their hamstringed attempts to reinvent themselves all too often remain an impediment in the task of reimagining and remaking our grid.”
Excerpts and Research Citations: Dr. Gretchen Bakke, The Grid, Chapter 4, pp. 112-114.
Enron led to Commercial and Industrial Grid Defection in 2001 - a Missed Opportunity for CA Utilities to Serve Large Loads and Advance their Grid Tech
In 2000-2001 as California faced the reckoning of these market manipulation impacts and rolling blackouts, the folks with the most to lose (the largest Value of Lost Load numbers), began to take action to create self-reliance. Bakke writes that “many new economy businesses, like Apple and Cisco Systems, as well as other electricity-dependent undertakings such as military bases and prisons, began to think about ways they might detach themselves from grid-provided power.”
Bakke explains that institutional grid defection in the wake of 2000-2001 signaled that smart companies were engineering ways to “use the grid as a backup power system rather than as something upon which they must rely regardless of how poorly it was managed or how sporadically its product was delivered.”
Excerpts and Research Citations: Dr. Gretchen Bakke, The Grid, Chapter 4, pp. 114.
Eyeballs Wide in 2024: So Many Constraints, So Little Time
Technology is advancing at the speed of sound compared to the rate of government regulation of technology. This makes catch-up hard. What if I told you, that utilities’ staff believe that they are far behind even government? For them, technology innovation is occurring at the speed of light, and catching up is not a reality. Better to catch the light, and toss out the oil lamp.
Throwing out the old stuff and bringing in the new, is incredibly difficult. A small history lesson on California alone, should bear on the thoughts you develop on this topic. I describe it as a Catch-22: damned if you do, damned if you don’t. Let’s dig into the challenges.
Regulatory Constraints
Utilities operate in a highly regulated environment that can discourage innovation and risk-taking. Key regulatory challenges include:
Rate of return regulation that favors capital investments over operational expenditures
Limited ability to earn returns on software-as-a-service (SaaS) solutions (capital investments)
Lengthy approval processes for new technology investments
Lack of incentives for performance improvements and innovation (performance based standards are not prophylactic; they are punitive, hard to view them as motivating, inspiring)
@Mission:Data, a coalition of technology companies advocating for customer-friendly data access policies, highlights the importance of regulatory reform in enabling technology adoption so that the speed of light can be experienced by more Americans.
Michael Murray, President of Mission:Data explains that "outdated regulatory frameworks often hinder utilities from embracing innovative technologies that could benefit both the grid and consumers. We need policies that incentivize utilities to adopt solutions that enhance energy efficiency and grid reliability."
Organizational Inertia
Many utilities have organizational cultures and structures that are resistant to change. Barriers include:
Risk-averse culture focused on reliability over innovation usually discussed as an issue of “safety first” and “everything else later.”
Siloed departments that impede cross-functional collaboration and single points of accountablity over major cross-department initiatives, i.e., the lack of an “ombudsman” to herd cats in many departments.
Lack of digital skills and technology expertise, fed by regulatory reviewers encouraging prudency over cow-boyish behavior, which immediately demotivates the young and bright from trying anything new.
Resistance to new ways of working is deeply ingrained in culture at the top of key departments, even if utility leaders are feeling gung-ho about the direction of the company.
Age gaps and knowledge gaps without change management: utility workers in the past 15 years who retired from positions, left wide gaps behind them and did not train up replacements who knew as much or more than them. New people coming in are talented, bright, and want change: but they are doing the jobs of 12 people in some cases and equally terrified of regulators treating them like bad actors.
Procurement Processes
Traditional utility procurement practices are often ill-suited for acquiring new technologies:
Emphasis on lowest cost over long-term value (because monopolies capitalize and earn a profit, and earning the right profit depends on showing the regulator their expenses were as frugal as possible).
Difficulty evaluating new technologies and vendors.
Inflexible contracting models (contracting process for a new tractor and a new DERMs just might be the same, even if the use cases make comparisons senseless).
describes this experience: PowerClerk is a software platform for utility program management. Andrew eplains that the lengthy procurement cycles in the utility industry can be a significant barrier for innovative technology providers. By the time a contract is signed, the technology landscape may have already shifted.
Funding Models
Utilities face challenges in securing funding for new technology investments:
Constraints on operational expenditures limit adoption of SaaS solutions are notorious and well-recognized, and plenty written about.
Difficulty justifying returns on emerging technologies comes primarily from a confluence of chaos created by attempts to integrate old systems together in a patchwork that relies on one outdated software stack, which means replacing one component for one utility area could “break the rest of it”. (My personal example of this is Austin Energy’s billing system, which connects utility waste services billing to the same engine that bills customers for their electricity consumption and exports).
Limited R&D and innovation budgets, also spurred on by deregulation constructs which demotivate incumbents: added to this, capital expenditures are already so inflated, there is quite literally no motivation in many utility businesses to cut the fat in those places and invest in innovation. The motto of the day continues to be: run what you have into ground, because your Commission/Regulator will approve a replacement for something that is already broken, depreciated, or both.
Pressure to keep rates low while investing in modernization is a tremendous Catch-22. It gets all of us in trouble: like the time Michigan’s PSC tried to build a “fast track pilot program” series of efforts with DTE, and third-party vendors filed comments asking DTE to delay its work plan until a public comment period additional to the existing drawn-out calendar for these pilots to get developed - the public comments request came with an assertion that DTE’s pilot proposals were not actually innovation issues.
Emerging Opportunities
Despite these challenges, several trends are creating new opportunities for utilities to accelerate technology adoption:
Regulatory Reform
Some regulators are implementing reforms to encourage innovation, such as:
Performance-based regulation that rewards utilities for outcomes
Regulatory sandboxes to test new technologies and business models
Pre-approval processes for certain technology investments independent of the rate case behemoth
Allowing returns on cloud computing and SaaS expenditures
New Partnership Models
Utilities are exploring new ways to collaborate with technology providers:
Innovation partnerships and co-development initiatives
Startup engagement programs and accelerators
Joint deployments with other utilities to share costs/risks
Outcome-based contracting models
A recent example of such a partnership is Edge Zero's collaboration with Vermont Electric Cooperative (VEC). As reported by EIN Presswire, "Edge Zero has been chosen to deliver real-time grid monitoring to Vermont Electric Cooperative." This partnership aims to enhance VEC's grid visibility and operational efficiency through Edge Zero's advanced monitoring technology. Edge Zero's technology exemplifies the potential of digital transformation in the utility sector for something as basic as “can we make sure the loads that want to interconnect at point X should actually be here versus point Y, and can we save on infrastructure build costs and reduce sagging voltage by relocating this request to a different spot that the customer will like as much as we do.” Their real-time low voltage (LV) network monitoring cloud-connected transformer-mounted sensors provide utilities with visibility into their low-voltage distribution networks, enabling proactive maintenance and improved operational efficiency.
Innovative Financing
New approaches are emerging to fund grid modernization:
Government grants and incentives for smart grid technologies
Digital Transformation Initiatives
Leading utilities are launching enterprise-wide digital transformation efforts:
Appointing chief digital officers and innovation teams
Workforce retraining and digital upskilling programs
Agile development and product management approaches
Data platforms to unlock value of grid and customer data
What to Do and Who Should Do It
For Utilities:
Develop comprehensive digital transformation roadmaps
Create dedicated innovation teams and budgets
Partner with startups and tech companies to co-develop solutions
Implement agile development and product management practices
Invest in workforce digital skills development
For Regulators:
Implement performance-based regulatory models
Allow returns on cloud and SaaS investments
Create regulatory sandboxes for piloting new technologies
Streamline approval processes for certain grid modernization investments
For Technology Vendors:
Develop utility-specific SaaS and subscription models
Offer flexible contracting and risk-sharing approaches
Partner with utilities on pilots and co-development initiatives
Provide implementation services and change management support
Come into utility conversations knowledgeable about the status quos of that customer, including things like the health of their relationship with a regulator, biggest pain points, and what they are most concerned about -and also, what their biggest and/or most vocal customers are complaining about.
Special Bonus: Leveling Up Data Access, Tech, and Energy Efficiency
Access to energy usage data is crucial for enabling innovative energy efficiency solutions and empowering consumers to make informed decisions about their energy consumption. I talked above about Mission:data, a coalition that has been engaged in many states on improving data accessibility and standardization across utilities.
Mission:data's Recommendations
Michael Murray, President of Mission:data, emphasizes the importance of data access: "Outdated regulatory frameworks often hinder utilities from embracing innovative technologies that could benefit both the grid and consumers. We need policies that incentivize utilities to adopt solutions that enhance energy efficiency and grid reliability" [1].
Mission:data has put forth several key recommendations for policymakers and utilities:
Standardized Data Formats: Advocate for the widespread adoption of Green Button — Connect My Data (CMD) as a standardized format for energy data sharing. This allows third-party service providers to easily integrate with multiple utilities, reducing costs and accelerating innovation [2].
Real-Time Data Access: Encourage utilities to enable real-time data access through Home Area Network (HAN) interfaces, allowing consumers and authorized third parties to receive energy usage information with minimal delay [3].
Streamlined Authorization Process: Implement a simple, streamlined process for customers to authorize third-party access to their energy data. This could include online authorization portals and the ability for third parties to initiate the authorization process on behalf of customers [4].
Data Quality and Completeness: Ensure that utilities provide complete, accurate, and timely data to authorized third parties, including historical usage, billing information, and rate structures [5].
Privacy and Security Safeguards: Implement robust privacy and security measures to protect consumer data while still enabling authorized access for innovative energy services [6].
Expanding Utility Data Usage
Mission:data also recommends that utilities expand how they use and share data to drive energy efficiency improvements:
Disaggregation Services: Utilities should consider offering or partnering with providers of energy disaggregation services, which can break down total energy usage into individual appliances or end uses. This granular information can help consumers identify specific opportunities for energy savings [7].
Benchmarking and Comparative Analysis: Utilities should provide anonymized, aggregated data for building benchmarking and community-wide energy analysis, enabling property managers and local governments to identify efficiency opportunities [8].
Time-of-Use (TOU) Rate Analysis: Utilize interval data to help customers understand the potential benefits of switching to TOU rates, and provide tools for optimizing energy usage under these rate structures [9]. ** Interestingly, at NARUC, I was in the audience when commissioners from a few states including Washington, DC said that customers “resist” time of use rates. That’s interesting.
Targeted Energy Efficiency Programs: Use detailed energy usage data to identify customers who would benefit most from specific energy efficiency programs or interventions, allowing for more effective allocation of resources [10].
PowerClerk's Role in Streamlining Energy Efficiency Programs
, Director of Business Development at Clean Power Research, explains: "PowerClerk is being used by over 60 utilities, mostly IOUs, to streamline their energy efficiency and distributed energy resource programs." [11].
Key features of PowerClerk that enhance energy efficiency program workflows include:
Automated Application Processing: PowerClerk automates the intake and processing of program applications, reducing administrative burden and expediting approvals [12].
Integration with Utility Systems: The platform can integrate with utility customer information systems, billing systems, and other databases to streamline data flow and improve accuracy [13].
Customizable Workflows: Utilities can design program-specific workflows that align with their unique requirements and processes [14].
Real-Time Reporting: PowerClerk provides utilities with real-time insights into program performance, uptake, and budget utilization [15].
Customer Portal: A user-friendly interface allows customers to track their application status and submit required documentation easily [16].
By leveraging platforms like PowerClerk, utilities can significantly reduce the administrative costs associated with energy efficiency programs. As Andrew Price notes, "Some utilities have reported administrative cost reductions of up to 50% after implementing PowerClerk for their energy efficiency programs" [17].
The combination of improved data access, as advocated by Mission:data, and streamlined program administration through platforms like PowerClerk, presents a powerful opportunity to accelerate energy efficiency initiatives. By embracing these technologies and recommendations, utilities and policymakers can create a more dynamic, responsive, and efficient energy ecosystem that benefits both consumers and the grid as a whole.
[1] Convo with Michael Murray, Mission:data Coalition, 2023.
Share this post
Push and Pull: how do we help regulated utilities adopt distribution grid tech better and faster?
Share this post
A BIG THINK ON TECH, DATA, ELECTRIC UTILITIES
The utility industry stands at a critical crossroads, grappling with outdated regulations and aging infrastructure while facing unprecedented challenges. As climate adaptation looms and surprises, community resilience wavers and utilities are torn between innovating ahead of disruptions and conserving cash for rainy days and investor returns. The majority see the need for increased flexibility and resilience, yet are hamstrung by regulatory frameworks ill-equipped for today's realities. And then, there is this:
Less than third of over 400 surveyed utility company leaders think that solution and innovative technology providers will help them navigate what is coming. Just over half think their regulators and governments are more worthy partners. This kind of finding makes my head spin, until I remember - oh, we have allowed monopolies to be monopolies.
Be that as it may: I think utilities are worrying too much about regulators and thus are ill-equipped to grasp how quickly they could adopt software, hardware, and data management tools in their backyard, fostered through the massive innovation drive of decentralized SaaS and customer choice providers, whether it is smart meter data access to support energy efficiency programs or figuring out how much headroom is on a transformer so that interconnecting DERs and EV charging stations can be located sensibly and managed in real-time as load flexibility assets.
According to Guidehouse Insights forecasts, “at least $1 billion will be invested annually in advanced distribution management systems by 2035 to improve grid reliability in North America.
At the same time, innovations in CE/CX technologies point to a digitally enabled flexibility transformation across the grid edge ecosystem.
Investments in behavioral and structural energy efficiency technologies that interact with behind-the-meter technologies like electric vehicles represent an exciting opportunity for utilities to better tailor incentives for their clients.
An estimated 90% of North American homes will be served by customer engagement and experience analytics programs by 2030, increasing from around 65% today.”
If your name is:
400+ Surveyed Utility CEOS in 2024: Here is What They Think
Regulation:
42.7% believe the current utility regulatory landscape is "Behind the times, outdated and misaligned with today's system needs and investments."
67% think utilities can best serve customers by "Lead by example: Mobilize the public sector and regulators to innovate out in front of looming threats."
Stakeholder Prioritization:
56.4% of utilities prioritize partnering with "State and local government, including regulators" to navigate the Energy Transition.
Funnily enough, tech solutions providers and large C&I customers are two oddball friends in the same bucket of less-important allies.
Flexibility:
61.4% believe utility infrastructure investments should prioritize "Increasing flexibility to improve energy system resilience."
Resilience:
40% see "Aging infrastructure and decline in overall quality of life / community livability" as the most disruptive threat facing US utilities.
57.1% would like to see utilities build a "Storm-hardened, self-healing, and resilient delivery system providing 24/7 uninterrupted power" as the infrastructure-of-the-future vision.
Survey Full Link: PUF Publication 2024
Introduction
Electric distribution utilities face significant challenges in adopting and integrating new technologies to modernize the grid and improve operations. Regulatory constraints, organizational inertia, procurement processes, and funding models impede technology deployment. There are however emerging opportunities and strategies to accelerate adoption, including regulatory reform, new partnership models, and innovative financing approaches.
PURPA Culture & California’s Dereg Frenzy Led to Enron, Left Utilities Behind on Software Tech
In a little twist, PURPA began an era of attitudinal shift around what utilities should be able to do as innovators vs. third party innovators: this was not productive in that it precluded the essential hauling of monopoly energy providers into a modern era. Sometimes, a historian can tell it best. For this story, it is electricity historian Dr. Gretchen Bakke and the particular leave-them-out effort in question, is California’s deregulation mandate in 1998, a piece of legislation that was “litlte more than a remarkable series of loopholes twisted through with some regulatory language…”
Bakke writes that “PURPA cracked open the utilities’ monopsomy control of the grid” and intitiated a grand phase of the end of of their writ-large natural monopoly over the means of production and delivery of electricity. However, “it did not change…the prevailing attitude” of lawmakers and the utilities that the only way to exert control over the behavior of the utilities was “through requiring absolute adherence to whatever happened to get legislated. Over time, this culture of obligation has had the unforunate effect of making the utilities even less capable of adapting to the rapidly changing electricity lanscape…”
I’ll agree with Dr. Bakke, that Enron’s exposure led to quiet shutdowns of far smaller tech company operations doing essentially the same thing. Also, California deregulation was not the only party at fault for Enron enablement. “In December 2000, Senator Gramm helped push a bill through Congress that deregulated trading in energy. Enron's electricity trading business swelled, and some of the firm's only real profits were made. Without owning a single California power plant, Enron came to control the state's market. Rolling blackouts became the norm, prices skyrocketed, and the state racked up billions in debt. Phil Gramm blamed environmentalists for the crisis. Finally, price controls were imposed and the bubble burst. Deprived of its cash cow, Enron hit the rocks a few months later. But of course the biggest profiteers were the synergists themselves, the investment banks and venture capitalists (VCs).” After the New Economy: The Binge . . . And the Hangover That Won't Go Away by Doug Henwood.
(In a rare moment, realizing Texas hopefully apologized to California for how many Texas politicians backed federal and state policies which enabled Enron’s conduct. Did California apologize to Californians?)
In its chaotic reversal of a dangerous and clumsy effort at deregulation, California forgot once more to bring utilities along for the ride in technology adoption and modernization. Dr. Bakke writes that “the state required the utilities to comply to its poorly crafted legislation while smaller, more flexible, more innovative companies ran rings around them both. This, was the result of the culture of PURPA, which presumed the stodginess and unadventurousness of the utilities to be inherent and thus unreformable characteristics of the industry. The utilities could be regulated, wisely or foolishly, but they would not be asked to learn adaptability, flexibility, or creativity themselves; instead they would be told what to do and they would be expected to do it. “
Excerpts and Research Citations: Dr. Gretchen Bakke, The Grid, Chapter 4, pp. 112-114.
Enron led to Commercial and Industrial Grid Defection in 2001 - a Missed Opportunity for CA Utilities to Serve Large Loads and Advance their Grid Tech
In 2000-2001 as California faced the reckoning of these market manipulation impacts and rolling blackouts, the folks with the most to lose (the largest Value of Lost Load numbers), began to take action to create self-reliance. Bakke writes that “many new economy businesses, like Apple and Cisco Systems, as well as other electricity-dependent undertakings such as military bases and prisons, began to think about ways they might detach themselves from grid-provided power.”
Bakke explains that institutional grid defection in the wake of 2000-2001 signaled that smart companies were engineering ways to “use the grid as a backup power system rather than as something upon which they must rely regardless of how poorly it was managed or how sporadically its product was delivered.”
Excerpts and Research Citations: Dr. Gretchen Bakke, The Grid, Chapter 4, pp. 114.
Eyeballs Wide in 2024: So Many Constraints, So Little Time
Technology is advancing at the speed of sound compared to the rate of government regulation of technology. This makes catch-up hard. What if I told you, that utilities’ staff believe that they are far behind even government? For them, technology innovation is occurring at the speed of light, and catching up is not a reality. Better to catch the light, and toss out the oil lamp.
Throwing out the old stuff and bringing in the new, is incredibly difficult. A small history lesson on California alone, should bear on the thoughts you develop on this topic. I describe it as a Catch-22: damned if you do, damned if you don’t. Let’s dig into the challenges.
Regulatory Constraints
Utilities operate in a highly regulated environment that can discourage innovation and risk-taking. Key regulatory challenges include:
Rate of return regulation that favors capital investments over operational expenditures
Limited ability to earn returns on software-as-a-service (SaaS) solutions (capital investments)
Lengthy approval processes for new technology investments
Lack of incentives for performance improvements and innovation (performance based standards are not prophylactic; they are punitive, hard to view them as motivating, inspiring)
@Mission:Data, a coalition of technology companies advocating for customer-friendly data access policies, highlights the importance of regulatory reform in enabling technology adoption so that the speed of light can be experienced by more Americans.
Michael Murray, President of Mission:Data explains that "outdated regulatory frameworks often hinder utilities from embracing innovative technologies that could benefit both the grid and consumers. We need policies that incentivize utilities to adopt solutions that enhance energy efficiency and grid reliability."
Organizational Inertia
Many utilities have organizational cultures and structures that are resistant to change. Barriers include:
Risk-averse culture focused on reliability over innovation usually discussed as an issue of “safety first” and “everything else later.”
Siloed departments that impede cross-functional collaboration and single points of accountablity over major cross-department initiatives, i.e., the lack of an “ombudsman” to herd cats in many departments.
Lack of digital skills and technology expertise, fed by regulatory reviewers encouraging prudency over cow-boyish behavior, which immediately demotivates the young and bright from trying anything new.
Resistance to new ways of working is deeply ingrained in culture at the top of key departments, even if utility leaders are feeling gung-ho about the direction of the company.
Age gaps and knowledge gaps without change management: utility workers in the past 15 years who retired from positions, left wide gaps behind them and did not train up replacements who knew as much or more than them. New people coming in are talented, bright, and want change: but they are doing the jobs of 12 people in some cases and equally terrified of regulators treating them like bad actors.
Procurement Processes
Traditional utility procurement practices are often ill-suited for acquiring new technologies:
Long, complex RFP processes favor incumbent vendors.
Emphasis on lowest cost over long-term value (because monopolies capitalize and earn a profit, and earning the right profit depends on showing the regulator their expenses were as frugal as possible).
Difficulty evaluating new technologies and vendors.
Inflexible contracting models (contracting process for a new tractor and a new DERMs just might be the same, even if the use cases make comparisons senseless).
Funding Models
Utilities face challenges in securing funding for new technology investments:
Constraints on operational expenditures limit adoption of SaaS solutions are notorious and well-recognized, and plenty written about.
Difficulty justifying returns on emerging technologies comes primarily from a confluence of chaos created by attempts to integrate old systems together in a patchwork that relies on one outdated software stack, which means replacing one component for one utility area could “break the rest of it”. (My personal example of this is Austin Energy’s billing system, which connects utility waste services billing to the same engine that bills customers for their electricity consumption and exports).
Limited R&D and innovation budgets, also spurred on by deregulation constructs which demotivate incumbents: added to this, capital expenditures are already so inflated, there is quite literally no motivation in many utility businesses to cut the fat in those places and invest in innovation. The motto of the day continues to be: run what you have into ground, because your Commission/Regulator will approve a replacement for something that is already broken, depreciated, or both.
Pressure to keep rates low while investing in modernization is a tremendous Catch-22. It gets all of us in trouble: like the time Michigan’s PSC tried to build a “fast track pilot program” series of efforts with DTE, and third-party vendors filed comments asking DTE to delay its work plan until a public comment period additional to the existing drawn-out calendar for these pilots to get developed - the public comments request came with an assertion that DTE’s pilot proposals were not actually innovation issues.
Emerging Opportunities
Despite these challenges, several trends are creating new opportunities for utilities to accelerate technology adoption:
Regulatory Reform
Some regulators are implementing reforms to encourage innovation, such as:
Performance-based regulation that rewards utilities for outcomes
Regulatory sandboxes to test new technologies and business models
Pre-approval processes for certain technology investments independent of the rate case behemoth
Allowing returns on cloud computing and SaaS expenditures
New Partnership Models
Utilities are exploring new ways to collaborate with technology providers:
Innovation partnerships and co-development initiatives
Startup engagement programs and accelerators
Joint deployments with other utilities to share costs/risks
Outcome-based contracting models
A recent example of such a partnership is Edge Zero's collaboration with Vermont Electric Cooperative (VEC). As reported by EIN Presswire, "Edge Zero has been chosen to deliver real-time grid monitoring to Vermont Electric Cooperative." This partnership aims to enhance VEC's grid visibility and operational efficiency through Edge Zero's advanced monitoring technology. Edge Zero's technology exemplifies the potential of digital transformation in the utility sector for something as basic as “can we make sure the loads that want to interconnect at point X should actually be here versus point Y, and can we save on infrastructure build costs and reduce sagging voltage by relocating this request to a different spot that the customer will like as much as we do.” Their real-time low voltage (LV) network monitoring cloud-connected transformer-mounted sensors provide utilities with visibility into their low-voltage distribution networks, enabling proactive maintenance and improved operational efficiency.
Innovative Financing
New approaches are emerging to fund grid modernization:
Green bonds and sustainability-linked financing
As-a-service models for grid infrastructure
Non-wires alternatives leveraging third-party investments
Government grants and incentives for smart grid technologies
Digital Transformation Initiatives
Leading utilities are launching enterprise-wide digital transformation efforts:
Appointing chief digital officers and innovation teams
Workforce retraining and digital upskilling programs
Agile development and product management approaches
Data platforms to unlock value of grid and customer data
What to Do and Who Should Do It
For Utilities:
Develop comprehensive digital transformation roadmaps
Create dedicated innovation teams and budgets
Partner with startups and tech companies to co-develop solutions
Implement agile development and product management practices
Invest in workforce digital skills development
For Regulators:
Implement performance-based regulatory models
Allow returns on cloud and SaaS investments
Create regulatory sandboxes for piloting new technologies
Streamline approval processes for certain grid modernization investments
For Technology Vendors:
Develop utility-specific SaaS and subscription models
Offer flexible contracting and risk-sharing approaches
Partner with utilities on pilots and co-development initiatives
Provide implementation services and change management support
Come into utility conversations knowledgeable about the status quos of that customer, including things like the health of their relationship with a regulator, biggest pain points, and what they are most concerned about -and also, what their biggest and/or most vocal customers are complaining about.
Special Bonus: Leveling Up Data Access, Tech, and Energy Efficiency
Access to energy usage data is crucial for enabling innovative energy efficiency solutions and empowering consumers to make informed decisions about their energy consumption. I talked above about Mission:data, a coalition that has been engaged in many states on improving data accessibility and standardization across utilities.
Mission:data's Recommendations
Michael Murray, President of Mission:data, emphasizes the importance of data access: "Outdated regulatory frameworks often hinder utilities from embracing innovative technologies that could benefit both the grid and consumers. We need policies that incentivize utilities to adopt solutions that enhance energy efficiency and grid reliability" [1].
Mission:data has put forth several key recommendations for policymakers and utilities:
Standardized Data Formats: Advocate for the widespread adoption of Green Button — Connect My Data (CMD) as a standardized format for energy data sharing. This allows third-party service providers to easily integrate with multiple utilities, reducing costs and accelerating innovation [2].
Real-Time Data Access: Encourage utilities to enable real-time data access through Home Area Network (HAN) interfaces, allowing consumers and authorized third parties to receive energy usage information with minimal delay [3].
Streamlined Authorization Process: Implement a simple, streamlined process for customers to authorize third-party access to their energy data. This could include online authorization portals and the ability for third parties to initiate the authorization process on behalf of customers [4].
Data Quality and Completeness: Ensure that utilities provide complete, accurate, and timely data to authorized third parties, including historical usage, billing information, and rate structures [5].
Privacy and Security Safeguards: Implement robust privacy and security measures to protect consumer data while still enabling authorized access for innovative energy services [6].
Expanding Utility Data Usage
Mission:data also recommends that utilities expand how they use and share data to drive energy efficiency improvements:
Disaggregation Services: Utilities should consider offering or partnering with providers of energy disaggregation services, which can break down total energy usage into individual appliances or end uses. This granular information can help consumers identify specific opportunities for energy savings [7].
Benchmarking and Comparative Analysis: Utilities should provide anonymized, aggregated data for building benchmarking and community-wide energy analysis, enabling property managers and local governments to identify efficiency opportunities [8].
Time-of-Use (TOU) Rate Analysis: Utilize interval data to help customers understand the potential benefits of switching to TOU rates, and provide tools for optimizing energy usage under these rate structures [9]. ** Interestingly, at NARUC, I was in the audience when commissioners from a few states including Washington, DC said that customers “resist” time of use rates. That’s interesting.
Targeted Energy Efficiency Programs: Use detailed energy usage data to identify customers who would benefit most from specific energy efficiency programs or interventions, allowing for more effective allocation of resources [10].
PowerClerk's Role in Streamlining Energy Efficiency Programs
PowerClerk, a software platform developed by Clean Power Research, has been instrumental in improving the workflow and administration of energy efficiency programs.
Key features of PowerClerk that enhance energy efficiency program workflows include:
Automated Application Processing: PowerClerk automates the intake and processing of program applications, reducing administrative burden and expediting approvals [12].
Integration with Utility Systems: The platform can integrate with utility customer information systems, billing systems, and other databases to streamline data flow and improve accuracy [13].
Customizable Workflows: Utilities can design program-specific workflows that align with their unique requirements and processes [14].
Real-Time Reporting: PowerClerk provides utilities with real-time insights into program performance, uptake, and budget utilization [15].
Customer Portal: A user-friendly interface allows customers to track their application status and submit required documentation easily [16].
The combination of improved data access, as advocated by Mission:data, and streamlined program administration through platforms like PowerClerk, presents a powerful opportunity to accelerate energy efficiency initiatives. By embracing these technologies and recommendations, utilities and policymakers can create a more dynamic, responsive, and efficient energy ecosystem that benefits both consumers and the grid as a whole.
[1] Convo with Michael Murray, Mission:data Coalition, 2023.
[2] Mission:data Coalition, "Data Access Recommendations," 2022.
[3] Ibid.
[4] Mission:data Coalition, "Streamlining Third-Party Authorization," 2021.
[5] Mission:data Coalition, "Energy Data Quality Standards," 2022.
[6] Mission:data Coalition, "Privacy and Security Best Practices," 2023.
7] Mission:data Coalition, "Expanding Utility Data Usage," 2022.
[8] Ibid.
[9] Ibid.
[10] Ibid.
[11] Convo with Andrew Price, Clean Power Research, 2024.
[12] Clean Power Research, "PowerClerk Features," 2024.
[13] Ibid. [14] Ibid. [15] Ibid. [16] Ibid.
[17] Convo with Andrew Price, Clean Power Research, 2024.