[Note that this article is a transcript of the video embedded above.]

In June of 2000, the power shut off across much of the San Francisco Bay area. There simply wasn’t enough electricity to meet demands, so more than a million customers were disconnected in California's largest load shed event since World War II. It was just one of the many rolling blackouts that hit the state in the early 2000s. Known as the Western Energy Crisis, the shortages resulted in blackouts, soaring electricity prices, and ultimately around 40 billion dollars in economic losses. But this time, the major cause of the issues had nothing to do with engineering. There were some outages and a lack of capacity from hydroelectric plants due to drought, but the primary cause of the disaster was economic. Power brokers (mainly Enron) were manipulating the newly de-regulated market for bulk electricity, forcing prices to skyrocket. Utilities were having to buy electricity at crazy prices, but there was a cap on how much they could charge their customers for the power. One utility, PG&E, lost so much money, it had to file for bankruptcy. And Southern California Edison almost met the same fate.

Most of us pay an electric bill every month. It’s usually full of cryptic line items that have no meaning to us. The grid is not only mechanically and electrically complicated; it’s financially complicated, too. We don’t really participate in all that complexity - we just pay our bill at the end of every month. But it does affect us in big ways, so I think it’s important at least to understand the basics, especially because, if you’re like me, it’s really interesting stuff. I’m an engineer, I’m not an economist or finance expert. But, at least in the US, if you really want to understand how the power grid works, you can’t just focus on the volts and watts. You have to look at the dollars too. I’m Grady, and this is Practical Engineering.

Electricity is not like any normal commodity we buy and sell. You can’t really go to the store and pick up a case of kilowatt-hours. It can’t really be stored or stockpiled on an industrial scale, which means it has to be created at essentially the exact instant it's needed. And the demand is fairly inelastic. We want our lights, stoves, air conditioners, and devices to turn on no matter the time of day. That requires the supply side to handle incredible volatility, ramping up or down to meet demands in real-time. And the whole business is incredibly capital-intensive: you need very expensive infrastructure for pretty much every step of the process. The only reason it can work is that we all share that infrastructure, spreading out the costs. Call me a nerd, but I think all of this creates some fascinating challenges, both on the technical side for engineers and the organizational side for the policymakers, regulators, and all the companies that participate in the electric power industry.

It wasn’t that long ago that the electric utilities did it all. As the pros say, they were “vertically integrated.” Each utility owned and controlled the three major pieces of the grid within their service areas: generation (or power plants), transmission lines (which carry electricity at high voltage across long distances), and a distribution system (which delivers electricity to most customers at lower voltages). That meant they had a monopoly. Customers couldn’t choose where their power came from or how they got it. And that meant that electric utilities had to be carefully regulated to make sure that, without any competition, they were still offering customers a reasonable price for power.

Over time, utilities realized the value of interconnecting so they could help each other in times of need. Electricity is a true commodity, even if it has some unusual properties. For the job it does, it mostly doesn’t really matter who made it - a kilowatt is a kilowatt, no matter where it came from. If one utility’s power plant went down or bad weather hit, they could work out a deal to share power with a neighbor and keep demand satisfied. As the practice grew more common, power pools developed where multiple utilities would interconnect and agree to share power. Every system is different; subject to different risks, different weather conditions, and outages at different times. It just made economic sense to spread out that variability and risk. Eventually, huge parts of North America were interconnected by transmission lines, creating the “grids” we know today. The major interconnections in the US and Canada are the Western, Eastern, Quebec, and Texas.

Historically, the wholesale price one utility would pay another for power was regulated just like the rates utilities charged their retail customers. It was usually based on the actual cost of generating that power, so the big utilities couldn’t price gouge smaller companies. But a lot of that changed in the 1990s when the federal government opened the door for deregulation. The idea is simple on the surface: if power can move fairly freely on the grid, there’s no need for major utilities to be the only ones producing it, and there’s no need to regulate the prices for which it’s bought and sold. Let’s let market forces drive the decisions. It will increase competition and efficiency, driving down prices, and the investment risks will fall to the investors, not the customers.

Quite a few states took the opportunity to deregulate the production of power, and quite a few didn’t. In fact, right now, it’s roughly half and half, but there’s a lot of variety between states when it comes to who produces power and how it’s bought and sold between utilities and other companies, and even big differences within individual states. In truth, the process of deregulation has been anything but simple, and actually created a whole new set of interesting challenges. Companies trying to game the system, like what happened in California, is only part of it. In fact, power professionals often say that certain states aren’t deregulated; they’re just differently regulated. But how does all this really work in practice? Let’s set up an analogy.

Say I live on one side of a big lake. The water isn’t mine, but there is a water company on the other side. If I want to buy some water, they could load it on a truck and haul it to me. Or they could just put it in the lake, and I could take out the same amount. It’s probably not the same water, but it doesn’t really matter. In this analogy, water is water. Let’s scale it up. Now, hundreds of people need water, and hundreds of companies are selling it. Each person can hire any company they want to provide their water. The distance between buyer and seller doesn’t really matter. Every seller puts the amount of water they’ve sold in the lake, and each buyer takes as much out as they’ve bought. As long as everyone keeps track of how much they buy and sell, the lake stays full, and basic laws of physics will sort out how the actual water flows. In one way, you know exactly where you get your water: the company you paid to provide it. But in another way, you have no idea. All the water from all the companies is comingled in the lake. This is what happens on the grid. In a way, the power coming to your house comes from the power plant or plants that your utility paid to create it. But the electrons themselves probably didn’t. Just like the water in the lake, electricity flows according to the laws of physics from high potential to low, sloshing and flowing according to what everyone on the system is doing.

This is a lot like how a deregulated grid works. Utilities that supply electricity to their customers don’t generate the power themselves. They enter contracts to get it from wholesale power providers, separate companies who only generate electricity. But you can see a challenge here. If I’m on my big lake wanting some water, I may not want to coordinate with every water company to see who’s got the best price, especially if my need for water varies day by day or even second by second, and honestly, I’m not even sure exactly how thirsty I’m going to be. And if I’m a water company on the lake, it’s a lot of overhead work to deal with all these customers and their different needs. It makes a lot more sense if there’s a marketplace. So it is on the grid.

Like I mentioned, this varies quite a bit depending on where you are, so I’ll try to be as general as possible. Outside of those direct contracts between one buyer and one seller, most wholesale electricity in deregulated areas is bought and sold on the day-ahead market. Wholesale purchasers like utilities submit bids with estimates of how much electricity they’ll need for each hour of the next day. And generators submit their offers to sell a specific amount of electricity for a given price that’s based on their production costs, availability of fuel, and operational constraints. The facilitator of each wholesale market takes all the bids for every hour and matches the supply and demand to get the right amount of energy on the grid at the right times for the lowest cost. Here’s a basic example of a single hour of the auction:

Let’s say four generators submit bids to provide electricity during this hour: A nuclear plant bids 1200 MW for a price of $20/MWh. A natural gas peaker plant bids 400 MW for $100/MWh. A coal plant bids 500 MW for $30/MWh. And a wind farm bids 400 MW for $0. Wind and solar can submit very low bids because they have no fuel costs. There’s pretty much no way for them to lose money if they’re connected to the grid, especially because many get outside incentives for every megawatt-hour they generate. They even submit negative bids in some cases, meaning they’re willing to pay money to stay connected to the grid. In any case, electricity generators in our hypothetical hour have offered 2,500 megawatts of power to the market.

Let’s say purchasers submitted 2,000 megawatts of demand for this hour. We arrange the generation bids in order of least cost to satisfy demand. This concept is known as economic dispatch. We buy power at the lowest cost possible. We’re going to dispatch the wind farm and nuclear plant, dispatch the coal plant at 80% of the capacity they bid, and we don’t need the peaker plant at all. The clearing price is the cost of the last unit of supply to be dispatched. In this case, it’s $30 per megawatt-hour. Every producer gets paid that price for the power they put on the grid for that hour, even if they bid lower, and every buyer pays that price for wholesale electricity. This is why wind and solar are incentivized to bid 0 dollars. They essentially guarantee that they’ll make the cut.

It seems like a simple process in our hypothetical hour, but in reality there’s a lot more to it. For one, many types of power plants can’t just be toggled on and off with the flip of a switch. They need significant lead time to start up and shut down. They have minimum and maximum output levels. And their costs can vary a lot depending on how long they run. So the market has to take those factors into consideration. Also, we can’t perfectly predict the future, even for the next day. There are always going to be differences in the day-ahead forecasts. Demand varies, equipment has problems, and other unforeseen events like sabotage or solar storms happen all the time.

So another market runs in real-time, sometimes with auctions every 5 minutes, to make up those differences and keep the supply in check with demand. For example, if a wind farm overproduces what they bid into the day-ahead market, they can sell the extra on the real-time market. And if they underproduce, they may need to buy power in the real-time market to make up for the shortfall. And if things really get tight with not enough reserves, the real-time markets usually include a way to boost prices upward, even beyond what the clearing price would be, to make sure they’re more closely reflecting the actual value of electricity. That includes the cost to society if people lose power, or put another way, the cost they would be willing to pay to avoid a disruption in electrical service. This concept is called the value of lost load, and it’s something that the generators usually aren’t taking into account in their bids.

But that’s not all the markets. Many areas have a capacity market intended to make sure there are enough generators available to meet demands over the long term. These auctions happen only once a year or so, and generators bid to create new capacity within three years. All the generators that win in the auction are rewarded for adding capacity to the grid, no matter how much of that capacity actually gets used in the future. This doesn’t happen everywhere though. Texas doesn’t use a capacity market and instead relies on prices in the day-ahead and real-time markets to encourage generating companies to make long-term investments in capacity

Many areas also have markets for so-called ancillary services, basically services needed to keep the grid stable and reliable. There are auctions for regulation, which accounts for very short-term fluctuations in supply and demand to keep the frequency stable. There are also auctions for reserves that can keep plants ready to get on the grid quickly if another resource trips offline. Other services to keep the grid stable are often contracted directly instead of using auctions. Reliability-must-run contracts pay for power plants that are on the verge of retirement to stay in service until the capacity is replaced. Inertia services pay to keep a certain amount of rotating mass connected to the system. I have a video on that topic if you want to learn more. Black start contracts pay for some generators to have the ability to go from a total shutdown to operational without assistance from the grid. I also have a video on that topic. And reactive power contracts help maintain the stability of the voltage on the grid. And, I have a video on that one too.

A potentially surprising thing about many of these markets is that it doesn’t just have to be generation resources bidding into them. The overall goal is just to get the supply to meet demand, and there are two ways to do that. You can increase the supply or decrease the demand. I said earlier that electricity demand is fairly inelastic, but there are a lot of situations where customers can reduce demand, especially if they’re compensated for doing it. Large industrial power users like refineries can shift schedules around or even turn on their own generators if resources on the grid are getting scarce. This is how you get wacky news stories about cryptocurrency miners making more money participating in electricity markets than in Bitcoin. There are even companies that will gather up a bunch of smaller power users who have some flexibility in their demands, package them up, and sell that demand reduction as a service in the wholesale electricity market. And some utilities coordinate similar demand response programs with their customers, offering credits on your bill if you have a smart thermostat. Deregulation of wholesale electricity markets just opens up this world of possibilities in how we manage the grid.

But there is one big way my lake analogy from earlier breaks down. Because that lake symbolizes the transmission and distribution lines that carry power between the buyers and sellers. And in reality, they’re not really like a lake, but more like a series of interconnected canals. And they didn’t just appear. Someone has to build them and maintain them, often at great cost, so those costs need to be covered by the rates we pay for electricity on top of the generation. In this case, there’s really no way to deregulate those costs. It doesn’t make sense to build parallel, competing networks of transmission and distribution lines. It would cost too much, and we’d just have too many wires across the landscape. So regulators oversee the rates that transmission and distribution companies charge utilities to use their wires to move power between users and generators. And of course, there’s a whole host of complex financial systems in place to make this happen. Wholesale purchasers not only have to buy power they need and the power that will be lost along the way, but also reserve capacity on the transmission system for that power to travel, and pay the transmission and distribution system operators for the privilege.

Confusingly, the flow of power isn’t really controlled on a line-by-line basis or sometimes even on a system-by-system basis. Power flows where it flows once it’s released on the grid, and there’s no simple way to keep track of who made it or who bought it at individual points on the network. Transmission reservations and tariffs are the law of the land, but the actual electrical power follows the laws of physics. So unlike at your house where you pay one-to-one for the actual power that flows through your meter, payments to transmission operators aren’t always a perfect reflection of how each buyer’s power moves through their system. Still, it’s the best mechanism we have to ensure electricity moves reliably across the grid and that the owners of the transmission assets are fairly compensated.

The other thing is that those canals don’t have infinite capacity. They can only move so much water, just like the transmission system can only move so much power. So in managing the wholesale electricity market, you don’t only have to consider what’s the next cheapest source of power, but also whether you can actually get that power to where it needs to go. Grid operators have to account for congestion, like rush hour for electrons. They usually do this by allowing prices to vary from place to place, an idea called Locational Marginal Pricing. You can see on this map of Texas how significantly prices can vary across the state, reflecting a difference in where the demand is versus where the generators are and the congestion on the transmission system that results. And hopefully at this point you’re seeing how complicated all this really is. Grid operators have to take into consideration all these details - power flows, weather, limitations of every kind of generator, second-by-second changes in the system - in order to match supply with demand at the lowest cost possible. And it gets even more complicated when you add distributed generation sources, like home solar installations, that put energy on the grid from the other side of the meter.

And this is only on the wholesale side of the grid. Even though most of those dollars moving around came out of our pockets, the end-users of the electricity, you and I really don’t participate in this segment of the grid. For many of us, the company we pay for electricity (the retail provider) didn’t generate that electricity, and in many cases, doesn’t own the infrastructure that it traveled along to reach our house or place of work. And for around a quarter of the US, the retail market is deregulated to the point where you can choose which company you buy your power from. So what do they actually do?

In essence, retail providers just buy power on the wholesale market and sell it to you. They’re middlemen, the car dealerships of electricity. They navigate all that complexity we just discussed so you don’t have to. Retail providers all provide essentially the same thing, but they can differentiate themselves by offering different kinds of rates that suit their customers better. One provider in Texas, Griddy Energy, famously offered their customers the real-time wholesale price, exposing them to the incredible volatility of the market. Unsurprisingly, Griddy filed for bankruptcy after the winter storm in Texas when their customers couldn’t pay the exorbitant bills. The other thing retail providers can do is connect your dollars to specific sources of generation like renewables. Instead of buying power in the auction, where you have no control over the sources, they contract directly with wind, solar, and other generators to purchase it directly on your behalf.

So next time you get your power bill, take a look at those line items. Maybe there’s a base rate set by your provider that covers all the various costs of operating the grid from generation to transmission to distribution. Or maybe they’re broken out according to all the various costs that it actually takes to run the bulk power system. Do you pay a separate rate for the distribution service? Does your bill have an adjustment for the variability in the wholesale market? Is there a charge for the Public Utility Commission or whatever agency oversees this whole financial web of complexity? Every bill looks a little different, but I hope this video clears up some misconceptions and encourages you to think about what the price you pay for electricity actually accomplishes on the grid.