This video explains exactly how and why I used keyword arguments in my Python functions. If you want to learn how to make your scripts easier to read, this 9 minute video is for you.

If you want to see more tutorial posts like this: Tell us what topic you’d like covered using this form

Transcript

Hi there power system engineers, last video we talked about how to make a full newton rapheson solution with Python and PSSE. And during that example I showed how to use keyword arguments to your function. Inside my fnsl function I’ve used options1=0 and options5=0 to turn off taps and switched shunt adjustments.

The old way

You might be familiar with the alternate form of calling a function with the _i in it. I’m going to show you how I wrote a function with options1 and options5 keywords based on the documentation.

You might be more familiar with the _i arguments that PSSE provides. WHen you record a macro into a Python file you might find something that looks like this:

1

fnsl([0, _i, _i, _i, 0, _i, _i])

Now this equivalent to the one above, options1 is set to zero and options5 is set to zero. However then you need to fill in the intermediate options with these default integers which PSSE takes to mean don’t set this option leave it as the default. You’ll get something like this from the PSSE record macro.

Why change from the older method?

What I do is I replace this list of integer arguments with just two keyword arguments. Just the ones I want to change. The reason I do this is because it is easier for me to read later on. I can just look and see that I’m only changing options 1 and 5. I think that later on for me at least it is easier to read.

How to find keyword argument names

I’ve looked up the documentation to find how to define each of these things. The way I find the documentation is inside PSSE 32, in the help topics there is a PDF. There is a line in there called Application Program Interface or API. Which is the Python connection to PSSE.

If I look inside the menu here on the side and go down to fnsl, this is the definition of that function in the documentation. I can see the Python syntax fnsl and a named argument called options. A unique quality of PSSE is that you can refer to each of the option list elements as a keyword argument. So 1,2,3,4,5,6,7 and 8 each of these options can be addressed individually. Python keyword arguments cannot have brackets (). So I write options1 all lower case. It corresponds to the tap adjustments flag. The docs tell you what values 0, 1 and 2 mean. I’ve used value 0 which means disable tap stepping, the default is 1 (tap stepping enabled) and another value is 2 (direct tap stepping).

options5 is a switched shunt adjustment flag. I’ve disabled the flag which is value 0.

These keyword arguments start at 1 not zero. Normally in Python indexing starts at 0, however the PSSE keyword arguments start at one.

The keywords are not always the word options

I’ll just show one more function, its the machine data changing function. If we go to Power Data Flow Changing. Let’s head to machine_data_2, this function has four arguments. This one intgar and realar are lists of arguments. intgar is an array of 6 elements. If I wanted to change the machine status I might say intgar1=0. The intgar ones are normally integers and the realar ones are floating point numbers.

So thats how I found the keyword arguments for the fnsl function.

If you have any questions about setting up keyword arguments for your PSSE functions, get onto https://psspy.org/ and ask a question (it’s free).

This video will show you exactly how to solve a PSSE loadflow using the full newton raphson solution.

If you want to see more tutorial posts like this: Tell us what topic you’d like covered using this form

Transcript

Hi there power systems engineers, today I’m going to show you how to solve a load flow in Python using PSSE. In the last lesson we looked at how to set up your PSSE script so that you could run PSSE from Python and not the other way around.

So I’m going to build on that script and add the capability to run a load flow and then say how many iterations that last load flow took to solve.

I’ve set up the script in the same way as the last video. We’ve changed the python path so that python knows where to find the psspy library. and I’ve redirected PSSE output to python.

Initialise PSSE

Just run psseinit to initialise psse. If you run the program you find that you get that psse initialise text.

Load a saved case

To solve a case you first need one loaded in memory. To load the saved case use the case function. I’m just using the example saved case that comes with PSSE savnw.sav.

If I run this again we see that the savnw case is loaded in memory and was correctly loaded.

Solve the saved case with fnsl

The next step is to solve it, and print a little message to the screen. That says the number of iterations in the last solution. I’m using the Full Newton Raphson solution.

I’m using the iterat function. The iterat function gives the number of iterations the last solution took to solve. And we substitute this value into the string with the percent ‘s’ (%s).

Here we can see there were 6 iterations, and my last print statement shows the number of iterations was 6.

Conclusion and sneak peak at the next video

That is how easy it is to run a full newton raphson solution. I can run with other options, I can run with taps stepping disabled and I can disable switched shunt adjustments as well.

The answer should be the same in this file, but that may not be true of your files. I knew that these options, options1 and options5 existed, and I’ll show you how I found them in the next video.

If you have any questions about how to solve a load flow with Python, get onto https://psspy.org/ and ask a question (it’s free).

Transcript

A look at importing Python into PSSE. Have you ever tried to import psspy into your Python script only to see ImportError: no module named psspy.

That means that Python was not able to find the psspy installation. Even though psspy comes with PSSE and we have Python installed on our computer. Python still can’t find it, and that is because Python is a completely separate program to PSSE.

Telling Python where PSSE is installed

We’re going to need to tell Python where PSSE is installed.

I’m going to import the operating system and system modules. I’m defining a variable PSSE_PATH which is where the PSSE that I have is installed. Yours might be different if you are using PSSE version 33, or if you are running 64 bit Windows.

Python has a sys.path which is a list of directories that it looks in. When you type in import it looks through every single one of these directories for the file that you have said to import. So later on where I’ve typed in import psspy Python will look through all of the directories and finally the one I’ve appended at the end PSSE_PATH will contain the psspy module.

Tell PSSE where PSSE is installed

We could run this program now and Python would find psspy. However we need to do one more thing. We need to tell PSSE where to find itself. PSSE is strange in that we need to set the environment variable PATH. We need to add to the end of that the PSSE installation path.

I’m not sure why PSSE needs to be told where it is, but your programs will crash at some point if you haven’t done that step.

Finally initialise PSSE

Let’s initialise with 100 buses. We can see that PSSE is in fact running.

That is how you can import PSSE into your python scripts. So that you can run the scripts from inside or outside of PSSE. You’ll notice that I didn’t open up the PSSE application at any point.

If you have any questions about how to solve a load flow with Python, get onto https://psspy.org/ and ask a question (it’s free).

Last week, we investigated how a pool pass through contract could save you money. We concluded that when market conditions are favourable, your electricity bill will be lower if you take the pool price. That post raised some discussion about what if hedge contracts had been purchased instead.

In this post we investigate how hedging your electricity contract can insure you against the volatility of the market.

Large consumers buy hedge contracts

We’ll analyst the costs for a single large energy consumer. This customer has 40 MW of installed machinery. Example customers of this size (and larger) might be:

• Steelworks;
• An aggregation of supermarkets or shopping centers; and
• Utility water companies

The minimum purchase price for a hedge contract from d-cypha trade is 1 MW. If your peak electricity usage isn’t over 1 MW, purchasing your own hedge contracts wont save you any money. But I’m sure a long talk with your electricity retailer would - we’ve been told that rates for higher using consumers are extremely competitive right now.

MW or MWh, what is the difference [ 1 ] ↓

What is the right amount of peak and base to buy?

Buy hedge contracts to fit your profile [Full Size]

Here is the daily load profile for the industrial consumer. Consumption is low overnight, typically below 14 MW. At 7 am production quickly ramps up to full output (around 38 MW). Then production finishes around 10 PM at night.

Peak contracts cover your usage between 7am to 10pm seven days a week. So if your facility uses more power during these times (like our example large consumer), it will be a good fit for a peak contract. Base contracts cover all hours of the day seven days a week. Base contracts are good for consumption that occurs consistently day and night. Our example consumer’s consumption doesn’t drop below 14 MW.

Buying a 14 MW base contract

The sum of our two contracts cover the peak [Full Size]

The consumption doesn’t drop below 14 MW. Imagine creating a rectangle 14 MW high on the daily profile chart. We’ve drawn it in green. It represents the base contract. Buying a 14 MW base contract means that we have agreed to pay for 14 MW at $X for every half hour in the day. Even if our machines break- down, or we take a vacation and shut down the plant, we still pay for that 14 MW.

Buying a 22 MW peak contract

The consumption between 7am and 10pm rises to between 37 MW and 38 MW. We’ve already purchased 14 MW with the base contract, which leaves 23 MW outside of contract. Buying 23 MW of another base contract would be wasteful in the extreme, because we don’t use that additional 23 MW at night. Instead we’ll purchase a peak contract for 22 MW (leaving 1-2 MW to take on spot market risk). We opted for 22 MW because the usage fluctuates above and below 23 MW, and we couldn’t purchase 22.5 MW because dcypha only trade contract units of 1 MW.

Making and losing money. A tail of two quarters.

We’ll investigate the cost of the hedge contracts over two financial quarters in 2012:

• The first from April to June; and
• The second from July to September.

The following analysis assumes the hedge prices [ 2 ] ↓ :

• base contract: $48 / MWh; and
• peak contract: $54 / MWh.

In the first quarter (April to June 2012). The total energy costs (both hedges and spot market rates for the energy outside of contract) were $3.28M. The equivalent if all energy was purchased from the spot market was $2.18M.

The second quarter (July to September 2012). The total energy costs were $3.44M. The equivalent if all energy was purchased from the spot market was $4.18M.

Paying for predictable pricing

A set of hedge contracts that fit your load profile can save money when the market becomes volatile. However when markets produce stable prices it is best to buy directly from the pool.

The trouble is, who knows how to predict whether the next quarter will be volatile or stable? If you can predict the volatility of future quarters, give us a call, we’d love to talk :)

You must purchase contracts that fit the shape of your load (or perhaps change the shape of your load to fit the contracts). Because once you are contracted for 14 MW, you must pay for those 14 MW whether you used them or not. An ill-fitting contract shape can cost you more than the volatility it protects you against.

Finally, if your consumption doesn’t rise above 1 MW for at least the peak hours, then it isn’t worthwhile purchasing a hedge contract yourself. This is what retailers are for. They pool all of their customers together and purchase large hedge contracts.

If you have some ideas for further investigation let us know in the comments

Footnotes

MW or MWh, what is the difference? [ 1 ] ↑

A MW (Mega-watt, 1 million watts) is an amount of energy consumed per second. Often pumps and generators have a nameplate with the MW rating written on it. Light bulbs are rated by their power consumption too. 20W to 100W being common around the family home.

A MWh (Mega-watt hour) is the standard measure of energy in the electricity market. Running a 1 MW pump for 1 whole hour will consume 1 MWh, running for half an hour will consume 0.5 MWh. And running a 100W light bulb for an hour will consume 0.0001 MWh.

I normally talk in terms of MW when giving the size of a facility, because it aligns with the ratings given for large equipment. Your electricity bill is for energy consumed (MWh) not your instantaneous power (MW).

Why $48 / MWh and $54 / MWh? [ 2 ] ↑

We estimated that contracts in Victoria could have been purchased at these values for the 2012 year. To understand how the contract price has been changing we read through some of the ”market wrap” reports written by d-cypha trade.

The second quarter was volatile notably because of the introduced carbon price. But we used the same contract prices assuming that they had been purchased well in advance, you can’t get such low prices on hedge contracts any more.

We investigate the raw cost of electricity. How much profit does a retailer make in selling us electricity?

Not everyone will find this article interesting. Because not everyone has a huge electricity bill or works in the electricity industry. If your electricity usage is greater than 50000 kWh / year (the equivalent usage of ten small families) then this is for you.

For those not reading on: Conclusion, A pool pass through contract could save 24% on your annual electricity bill. But by doing so you take on wholesale market risk instead of your retailer, and we note that in recent months it may have been more profitable to be on a fixed contract due to the large spike in volatility after the carbon price introduction.

The wholesale price for electricity varies every half hour

For a period during the 2000s I worked for the Australian National Electricity Market operator (then called NEMMCO now AEMO) and became fascinated with the wholesale electricity market. In most countries now, there exists a market based system for pricing electricity. In Australia that wholesale price changes every half an hour.

But most retail electricity contracts have a flat rate price per kWh used. Your retailer is buying electricity at the varying wholesale rates, and selling to you at a fixed flat rate. Your retailer takes the risk and they don’t take risks out of the kindness of their heart. The risks are carefully evaluated and priced into the flat rate. Your flat rate covers:

• The raw price of electricity
• The wholesale market risk
• Administration and overhead; and
• Retailer profits

A pool pass through exposes you to the wholesale electricity market

The pool pass through contract is the name of a deal you can make with your retailer where you pay the varying wholesale rates, instead of the guaranteed $0.24 / kWh.

Those wholesale prices vary from negative $1 / kWh, where you get paid to use more electricity - up to $12.50 / kWh. The price normally hovers between $0.04 / kWh and $0.06 / kWh.

These pool pass through deals were not popular for a very long time. Businesses were more interested in a flat predictable electricity bill than one which can be higher in summer and lower in winter. But all of that is changing and businesses are beginning to ask for pool pass through contracts.

Businesses may be moving away from pool pass through contracts

There has been a surge of volatility with the introduction of the recent carbon price in the Australian market (see All you need to analyse the electricity market pt 1) The general trend for the last two years was that businesses were asking for a pool pass through contract. Contrast this with the last few months, some businesses have regretted that decision. The recent volatility made the fixed price plans offered to the highest volume customer cheaper than riding the wholesale market.

Throughout the analysis bear in mind that a very large consumer (one spending hundreds of thousands on electricity per year) would be offered much better rates than $0.24 / kWh. Accepting wholesale market risk might work well in the years with low volatility, but could be an unacceptable burden for your short term cash flow when volatile periods happen.

Meet the worst kind of customer

We’ll run some calculations to see what the raw cost of the underlying electricity is on a pool pass through contract. But to do that we need an example customer. Meet Jill, the worst kind of customer. Jill uses the most electricity during the middle of the day when the prices are at their highest. Can Jill save money on a pool pass through contract? Yes. Read on to find out how much.

Jill’s usage profile on a typical weekday [Larger Size]

Here is Jill’s electricity usage profile. She uses 5000 kWh per year and is a smaller sized business. Her cost per year is $1245 including a generous 10% off discount from Origin energy. Jill’s company is a small design firm, and the electricity is mostly made up of the office computers, lights and cooling.

A saving of 24%

We calculated the base wholesale electricity cost had she been on a pool pass through contract for the last financial year (ending 30 June 2012) and made the following adjustments:

• Added $0.023 / kWh to account for the price of carbon which was introduced since July 2012.
• Added a $0.80 / day electricity supply charge

The raw cost of electricity then is:

$562

This raw electricity price doesn’t account for the costs incurred by your retailer in serving you electricity. The type of costs might be market fees, administration and business overhead. We weren’t sure how to calculate these costs. Instead we assumed that they could be around 50% of the total cost. Adding in that 50% brings the total to:

$943

This is a reduction of around 24% on the already discounted retail price of $1245.

What are the risks?

A pool pass through contract comes with very serious risks. In fact, the saved money is due to you taking on risk that your retailer previously faced. Here are the risks that our example business will face:

• The electricity price could reach $12.5 / kWh. Our example businesses would lose $3.60 for every hour that they ran under such high priced conditions.
• A disaster could cause the wholesale price could unexpectedly remain high for a long time.
• You could end up paying more in total for your electricity than if you were on a flat rate

Ride the wholesale ups and downs

By riding the market ups and downs you (on average) will save money across the year. And that saving will be equal to the price of risk that retailers charge to you (and profited from).

If you are a large business paying $200,000 / year, your savings would be less than the figures quoted here. The competition for very large businesses is fierce right now. Retailer margins are extremely low, and they look to make up their profit with volume.

Have a talk to your retailer and ask about their pool pass through plans. Here are some questions to ask them:

• How much energy must I consume to be eligible for a pool pass through contract;
• What are your fixed charges (administration fees); and
• Does any portion of your costs vary with consumption (e.g. market fees)?

Do you know something we don’t about pool pass through contracts? Have you tried them in the past, how did it work out for you?

If you’d like a copy of the data we used or want to comment about pool pass through contracts call us on (03) 8644 8163.

If you want to see more tutorial posts like this: Tell us what topic you’d like covered using this form

Transcript

Hello Power systems engineers. Have you ever wanted to control Microsoft Excel from Python? What about exporting data from PSSE into Microsoft Excel? Well today you’ll learn how to do both of these things: Control Excel from Python and export from PSSE. Lets get started.

I’ve opened up my Python shell here. And I’ve pointed it at the PSSE installation that I’m running which is PSSE 32. I’ve added to the PYTHONPATH so sys.path.append which tells Python where to find the PSSE libraries. And the os.environ PATH addition tells PSSE where its own libraries are added. You will need both of these before you can start importing the next two steps.

First connection to Excel

So I’ve imported the excelpy module. This is distributed with PSSE version 32 and was written by the PSSE team. And it makes it easy to write data to Excel.

I’ll create an excelpy workbook. Now a workbook is just an Excel spreadsheet. And I’ll store a reference to that spreadsheet in a variable I am going to call xl. And I can show that new Excel spreadsheet and it appears on the right as book4.

Write a heading to Excel

I can write headings. So for instance to write a heading “bus numbers” to cell A1 I use the set_cell method. I can also set a heading in the second column to “voltage (p.u)”.

Export data from PSSE to Excel

Now lets export some data from PSSE and put here into Excel. I’m going to export two sets of data: - First all of the bus numbers in the demonstration saved case; - Followed by all of the per unit voltages for those bus numbers.

So first let’s get a list of all the bus numbers. First we’ll import psspy and import redirect.

We’ll use redirect.psse2py to change any text from appearing in pop up dialog boxes to being in the actual Python shell here. (Jervis notes If you forget to run redirect.psse2py you will see a series of incredibly annoying pop up boxes created by PSSE)

Now I’ve initialised PSSE and you can see the output has been put here into this PSSE shell rather than in any pop up boxes.

Now with PSSE initialised let’s load a case. And then read out a list of all the bus numbers. The bus numbers have been stored in a variable called buses.

I’ve used a function called abusint which is one of the subsystem data retrieval functions. The subsystem data retrieval functions let you access a whole host of data for all of the buses at once rather than one at a time. (Jervis notes the subsystem data retrieval functions are incredibly powerful - but can be difficult to use and understand because of the way PSSE wrote them. See our post on Designing an easier subsystem data retrieval API for an alternative that you can use yourself)

You can do the same to get the voltages. And here I’ve collected the voltages in per unit.

Now to write this column of buses and column of voltages into Excel. So I’m going to set the range from row 2 in column A to be the buses. And, the voltages in column B. Now I’ve used the zip function to transpose what was given to me by PSSE as a row into a column.

Now we can save our workbook.

So that’s it for now. If you have any questions about exporting data into Excel. Feel free to hop onto http://psspy.org and ask your question. I’d be happy to answer.

Background - Order 755

CAISO sceptical that markets can be gamed [Larger Size]

The CAISO Market Surveillance Committee believes that there are market inefficiencies in the Californian Electricity Market.

This news follows the Federal Energy Regulatory Commission’s passing Order 755 which relates to payments made for controlling the security of the electricity market.

What are regulation payments?

Every second of the day the electricity system must be in perfect balance. For every house using a kettle, there must be an equivalent amount supplied from a wind turbine, or a power plant somewhere.

If there is not enough power supply, the whole power system begins to slow down. Just like if you were riding your bike, and encountered a hill but didn’t pedal any harder. Eventually the system is too slow and will fall over (like you would come off your bike).

CAISO already knows this. So they have generators everywhere on standby to produce little bits more energy to constantly balance things out. This system of balancing is called regulation.

Regulation payments are the system CAISO uses to pay generators for providing this balancing service.

What is Order 755?

FERC has given Independent System Operators like CAISO a short period of time to comply with their new policy: To pay generators for two things relating to regulation:

1. Just for being available, a capacity payment
2. For doing their job and actually balancing the power, a mileage payment

Generators all bid against each other to say: “I’m available for $X per hour and you need to pay me $Y per MW when I actually change my output to balance the power.”

Can the new payments system be gamed?

The Market Surveillance Committee says yes it can. Here is how:

Offer a low capacity price

The CAISO currently evaluates regulation offers by the lowest capacity price (not the new mileage price). So step one is to offer a very low capacity price so that you are selected to provide regulation services.

Quickly take all of the mileage payments.

The mileage payments are the amount of work you do to balance. Say there was a deficit of 100 MW and you were to increase output by 20 MW. You would get paid the mileage rate for that 20 MW. Everyone else would share in the remaining 80 MW. But what if you were quick, and could provide the entire 100 MW? Then that’s great; you get all of the mileage payments. And because you bid those payments yourself you can set whatever price you want.

The Market Management Council’s response

The opinion of the MMC (PDF) is that the divergence of capacity and mileage payments creates market inefficiencies.

However, the gaming issues identified are unlikely to cause a problem in the short term. Strong competition should keep prices in check. There are few, if any fast enough resources capable of providing regulation. The available fast ramping resources are hydro power plants and all of the hydro plants are owned by regulated load serving entities. So they don’t think any market participants will actually game the system.

Can the system be gamed?

We aren’t sure, the money changing hands in the regulation market is much lower than in the traditional wholesale electricity market, so there might be less incentive. The MMC’s reply to the possibility for market gaming didn’t seem alarmed. They have a market monitoring procedure in place that should pick up on irregularities.

The MMC asserts that a regulated load serving entity wouldn’t take advantage of the new market design to increase their profit. We find that to be interesting because we’ve seen evidence in other markets that even Government- owned businesses still profit from market power when they can.

Is there currently a way to game the Californian electricity market to increase profit? Are energy companies now silently boosting their profits by exerting their market power? What do you think?

[Larger Size]

We found in a recent survey that peak solar power production in Amprion’s network has more than doubled since 2010.

An attractive investment

Sizeable increases have been attributed to the attractive feed-in tariff scheme set up by the German government.

The German solar industry is supported by feed in tariffs paid to solar energy producers. Solar farm owners of less than 10MW in size are paid handsomely for every kWh of energy they produce.

Amprion’s transmission region [Larger Size]

According to http://www.germanenergyblog.de/?p=9756 owners are paid between 13.1 and 19.92 European cents per kWh produced.

These payments are guaranteed for the next 20 years. The investment situation is very stable and banks love it.

Abandoning Nuclear power

The German government plans to abandon nuclear power by 2022. Nuclear plants produced about a third of Germany’s power, this leaves a large gap that renewable energy can fill. We think that the solar power production in Amprion’s transmission region will rise still higher in the future.

Do you think solar power output will have doubled again in two years time?

How did you get the solar production data?

The main focus of this blog is to make you a better Python programmer in the energy industry. Do you want to know where we found that solar power data?

Sign up for updates to our blog, we’ll cover it in the next post! Don’t miss it.

[..] I have a fancy now: to have applications like this one […] available for members interested

the 44th CIGRE Paris has 444 paper presentations

At the 44th CIGRE Paris this year, there are over 6000 engineers attending to see 444 paper presentations held in 4 rooms concurrently.

But CIGRE isn’t just about the presentations, its a time to duck out mid- session and take your loved one for a coffee, a time to meet with someone just like you that works in the same job in another country.

How do you fit it all in, and not miss those 10-30 “must see” papers?

Now there is the CIGRE paper scheduler A place for you to search and find the papers that interest you, and see when and where you need to be to watch the presentations. It has a calendar that updates live as you select papers, showing you when your free times are so you can schedule your own meeting on the Seine.

See you in Paris!

Used the CIGRE scheduler? We’re still improving it. What is one thing that you would change?

This is the final part of a four part series. By the end we’ll have written a Python script to display a chart of electricity market prices.

• All you need to analyse the electricity market pt 1
• All you need to analyse the electricity market pt 2
• All you need to analyse the electricity market pt 3
• All you need to analyse the electricity market final

Australian electricity prices are high - let’s analyse

Very high electricity prices [Larger Size]

Previously I mentioned that the Australian electricity prices have gone through the roof (more than doubling) since the introduction of the carbon tax.

This series of posts is exploring how to analyse market data accessible from the internet. The methods described can be adapted to your country’s data or any sort of data available on the internet.

We began the series with a post detailing how to obtain a CSV file that contains the latest electricity market prices.

Then we unzipped the price data CSV file that was downloaded in Part 1 and had a brief look at its contents.

Then we pulled that CSV file apart using Python.

Now you’ll plot the prices and system demand using matplotlib.

The code we have developed over this series so far:

1. Downloads a zipped file;
2. Unzips it;
3. Reads the file contents as a CSV; and
4. Extracts the half hourly demand and price values.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

from __future__ import with_statement
import csv
import datetime
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

def is_halfhourly_data(row):
    """Returns True if the given row starts with 'D', 'TREGION', '', '1'"""
    return row[:4] == ["D", "TREGION", "", "1"]

halfhourly_data = filter(is_halfhourly_data, prices_csv_reader)

def get_date_region_and_rrp(row):
    """
    Returns the SETTLEMENTDATE, REGION and RRP from the given
    PUBLIC_PRICES CSV data row.

    SETTLEMENTDATE is converted to a Python date (the time is discarded);
    REGION is left as a string; and
    RRP is converted to a floating point.
    """
    return (datetime.datetime.strptime(row[4], '%Y/%m/%d %H:%M:%S').date(),
            row[6],
            float(row[7]))

date_region_price = map(get_date_region_and_rrp, halfhourly_data)

The completed example

Here is the complete code to download and plot the electricity prices with Python. We’ll step through the most important parts and show you two of Python’s advanced features defaultdict and yield.

If you aren’t interested in the advanced Python code, you can skip to the end. The matplotlib code that creates the chart is very short and easy to follow.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103

from __future__ import with_statement
from collections import defaultdict
import csv
import datetime
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

import matplotlib.pyplot as plt

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'
REGIONS = ("QLD1", "NSW1", "VIC1", "SA1", "TAS1")

# zippedfile is now one long string.
try:
    zippedfile = open(ZIP_URL.replace('/', '')).read()
except IOError:
    zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()
    f = open(ZIP_URL.replace('/', ''), 'wb')
    f.write(zippedfile)

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

def is_halfhourly_data(row):
    """Returns True if the given row starts with 'D', 'TREGION', '', '1'"""
    return row[:4] == ["D", "TREGION", "", "1"]

halfhourly_data = filter(is_halfhourly_data, prices_csv_reader)

def get_date_region_and_rrp(row):
    """
    Returns the SETTLEMENTDATE, REGION and RRP from the given
    PUBLIC_PRICES CSV data row.

    SETTLEMENTDATE is converted to a Python date (the time is discarded);
    REGION is left as a string; and
    RRP is converted to a floating point.
    """
    return (datetime.datetime.strptime(row[4], '%Y/%m/%d %H:%M:%S'),
            row[6],
            float(row[7]))

prices = map(get_date_region_and_rrp, halfhourly_data)

def get_region_price(date_region_prices, regions):
    """
    returns the dates and prices in two columns grouped by region,
    suitable for plotting with matplotlib.

    the order of returned prices is the same order as the `regions`
    argument.

    Args:
      date_region_prices: a list of (date, region, price) tuples
       [(datetime(2012, 08, 09), "QLD1", 45.6),
        (datetime(2012, 08, 09, 1), "NSW1", 46.0)
         ...
       ]
      regions: A list of the regions to return.
    >>> get_region_price([(datetime(2012, 09, 09), "QLD1", 43.2),
                          (datetime(2012, 09, 09), "NSW1", 45.5),
                          (datetime(2012, 09, 10), "NSW1", 44.2),
                          ...],
                          ("NSW1", "QLD1"))

    [(datetime(2012, 09, 09), datetime(2012, 09, 10))
     (45.5, 44.2)],

    [(datetime(2012, 09, 09),)
     (43.2,)]

    ...

    """
    region_prices = defaultdict(list)
    for date, region, price in date_region_prices:
        region_prices[region  + 'd'].append(date)
        region_prices[region  + 'p'].append(price)

    for region in regions:
        yield region_prices[region+'d'], region_prices[region+'p']

figure = plt.figure()

for dates, prices in get_region_price(prices, REGIONS):
    plt.plot(dates, prices, '-')

plt.legend(REGIONS)
plt.grid()
plt.xlabel("Time of day")
plt.ylabel("Electricity Price A$/MWh")
figure.autofmt_xdate()

plt.show()

Grouping the regions together

I want to plot each of the five Australian regions’s prices as a separate series. But I don’t have the data organised into separate x axis and y axis values. Instead there is one long Python list that has all regions.

1
2
3
4
5
6
7
8
9

[
('2012-01-01 00:00', 'NSW1', 34.5),
('2012-01-01 00:00', 'VIC1', 33.2),
('2012-01-01 00:00', 'SA1',  36.1),
('2012-01-01 00:00', 'TAS1', 38.2),
('2012-01-01 00:00', 'QLD1', 37.4),
('2012-01-01 00:30', 'NSW1', 34.6),
...
]

Here is a way to use defaultdict to collect the date and price per region. For example the 'NSW1' and 'VIC1' regions. The defaultdict (official docs) is just like a normal dictionary, except that it has one additional powerful feature:

`defaultdict` will auto-initialise a new value if you attempt to access a `key` that is missing.

Confused? Here is a concrete example. Grouping power station names by generator category (Nuclear, Wind,..) using a normal Python dict:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

gens = {}
gens['NUCLEAR'] = ['NUCLEAR-1', 'HILLVIEW-2', 'NUCLEAR-2']

# we can add a new nuclear generator name.
gens['NUCLEAR'].append('HILLVIEW-1')

# we can't add a new wind farm name - yet.
gens['WINDFARM'].append('WINDY-HILL-1')
# KeyError 'WINDFARM'!

# first must make a new empty wind farm list
gens['WINDFARM'] = []
# now this works.
gens['WINDFARM'].append('WINDY-HILL-1')

A KeyError exception will be raised on line 8, because 'WINDFARM' is a key that doesn’t exist in the gens dictionary yet. It isn’t until line 12 that the 'WINDFARM' key is entered into the dictionary and the first wind farm can be appended.

Here is the same code using defaultdict to initialise an empty list when there is a missing key. Notice that there is no need to create a key with an empty list before appending.

1
2
3
4
5
6
7
8
9
10

from collections import defaultdict

gens = defaultdict(list)

# empty list created for 'NUCLEAR' and straight away we extend it.
gens['NUCLEAR'].extend(['NUCLEAR-1', 'HILLVIEW-2', 'NUCLEAR-2'])
gens['NUCLEAR'].append('HILLVIEW-1')

# empty list created for 'WINDFARM' and straight away we append.
gens['WINDFARM'].append('WINDY-HILL-1')

Having seen defaultdict take another look at this code section from final.py:

1
2
3
4

region_prices = defaultdict(list)
for date, region, price in date_region_prices:
    region_prices[region  + 'd'].append(date)
    region_prices[region  + 'p'].append(price)

It makes two lists for every region. The key to the first list region + 'd' would look like NSW1d or VIC1d. The key to the second list is region + 'p' and looks like NSW1p or VIC1p.

The d stands for date, our x axis and the p stands for price, our y axis. Its time to plot those x and y values.

Making your own iterator

Use the yield keyword in a function to turn that function into something that can be used in a forloop (an iterable).

1
2

    for region in regions:
        yield region_prices[region+'d'], region_prices[region+'p']

I used the yield keyword in the get_region_price function to return the date and price (x and y axis) pairs that were grouped using defaultdict. They are returned one region at a time in the forloop.

1
2

for dates, prices in get_region_price(prices, REGIONS):
    plt.plot(dates, prices, '-')

yield will take some getting used to if you’ve never seen it before. Try working with this script on your computer so you can see what is happening:

1
2
3
4
5
6
7
8
9
10
11
12
13

names_age = [("NUCLEAR1", 10), ("NUCLEAR2", 11), ("WINDYPEAK", 2)]

def generator_names(names_age):

  print '[generator_names] started the generator_names generator'

  for name, age in names_age:
    print '[generator_names] about to yield', name
    yield name

for name in generator_names(names_age):
  print 'I got name: ', name

1
2
3
4
5
6
7

[generator_names] started the generator_names generator
[generator_names] about to yield NUCLEAR1
I got name:  NUCLEAR1
[generator_names] about to yield NUCLEAR2
I got name:  NUCLEAR2
[generator_names] about to yield WINDYPEAK
I got name:  WINDYPEAK

Plotting professionally in only 8 lines of code

Plotting the dates and prices is very easy once you have them in two lists, x axis and y axis.

The plot commands are similar to Matlab plotting routines:

1
2
3
4
5
6
7
8
9
10
11
12

figure = plt.figure()

for dates, prices in get_region_price(prices, REGIONS):
    plt.plot(dates, prices, '-')

plt.legend(REGIONS)
plt.grid()
plt.xlabel("Time of day")
plt.ylabel("Electricity Price A$/MWh")
figure.autofmt_xdate()

plt.show()

Here is the list of steps in the code above.

1. Create an empty figure;
2. Plot each region’s prices;
3. Display a legend;
4. Enable grid lines;
5. Set the x-axis label;
6. Set the y-axis label;
7. Auto-rotate the x axis date labels; and
8. Show the plot.

Conclusion

The final program is quite short, just 100 lines of code. But it covers such a wide range of tasks:

• Downloading files from the internet;
• Unzipping files;
• Reading CSV files;
• Sorting, transposing and filtering data; and
• Displaying data on a chart.

The post may not be clear in certain areas, or you may want us to write about something in more detail, so tell us using the form below.

(This is the third part of a four part series. By the end we’ll have written a Python script to display a chart of electricity market prices. Enter your email → on the right so that you don’t miss the final post.)

• All you need to analyse the electricity market pt 1
• All you need to analyse the electricity market pt 2
• All you need to analyse the electricity market pt 3
• All you need to analyse the electricity market final

Australian electricity prices are high - let’s analyse

Very high electricity prices [Larger Size]

Previously I mentioned that the Australian electricity prices have gone through the roof (more than doubling) since the introduction of the carbon tax.

This series of posts is exploring how to analyse market data accessible from the internet. The methods described can be adapted to your country’s data or any sort of data available on the internet.

We began the series with a post detailing how to obtain a CSV file that contains the latest electricity market prices.

Then we unzipped the price data CSV file that was downloaded in Part 1 and had a brief look at its contents.

Now we will teach you how to pull that CSV file apart using Python. You will master the ability to highlight the columns and rows that you are interested in. Just like top Japanese chefs are qualified to cut the good meat from the fugu fish, you too will learn to slice the good data from the CSV file.

The code we have developed over this series so far:

1. downloads a zipped file;
2. unzips it; and
3. reads the file contents as a CSV.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

from __future__ import with_statement
import csv
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

The example CSV file downloaded earlier had over 30 columns of information and many thousands of rows. We’ll use Python to get exactly the columns and rows that we want.

Which rows are important? Knowing what is in the CSV file is paramount at this stage. To this end, the site that provides the data may also provide a specification of the file structure. Failing that, you may have to get intimate with the data and spend a bit of time working out the format for yourself.

For the data provided by the Australian electricity market operator, the first CSV column is a label. Each label describes the purpose of that row. There are three values, C, I or D. Shown below is an example of the data stored in the first column,

1
2
3
4
5
6
7
8
9
10
11

C,
I,
D,
D,
D,
D,
I,
D,
D,
I,
...

Rows marked with a C are comment rows, they give further information about the file itself but aren’t necessary for us to worry about.

Rows marked with an I are header rows. The header row is just like a header row you use in a normal Microsoft Excel spreadsheet, it indicates what data is stored in that column. For our goal of finding the price of electricity in different regions of Australia over time, the columns that we are looking for are SETTLEMENTDATE (date and time), REGIONID (price region) and RRP (electricity price $/MWh).

Rows marked with a D are the data rows. We’ll take these rows for the SETTLEMENTDATE, REGIONID and RRP then plot them on a chart.

Multiple header rows in a CSV file?

Immediately though, we run into a problem. Notice in the CSV file structure figure shown above that there are multiple I header rows? There are no less than four in the CSV file we downloaded. You can think of it as four CSV files crammed into a single file.

We are only interested in one of these four sections, the section with the columns we mentioned before, SETTLEMENTDATE, REGIONID and RRP. Through analysis of the file structure, we know that all the data rows we are interested in all begin with:

1

D,TREGION,,1

Master technique one: filter

Here is how to update the program to print the rows beginning with D,TREGION,,1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

from __future__ import with_statement
import csv
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

def is_halfhourly_data(row):
    """Returns True if the given row starts with 'D', 'TREGION', '', '1'"""
    return row[:4] == ["D", "TREGION", "", "1"]

print filter(is_halfhourly_data, prices_csv_reader)

The filter function will only return the rows for which the function is_halfhourly_data returns True.

Master technique 2: map

Having correctly isolated the rows that interest us, slice up those columns and get the SETTLEMENTDATE, REGIONID and RRP columns (column numbers 4, 6 and 7).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

from __future__ import with_statement
import csv
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

def is_halfhourly_data(row):
    """Returns True if the given row starts with 'D', 'TREGION', '', '1'"""
    return row[:4] == ["D", "TREGION", "", "1"]

halfhourly_data = filter(is_halfhourly_data, prices_csv_reader)

def get_date_region_and_rrp(row):
    return (row[4], row[6], row[7])

date_region_price = map(get_date_region_and_rrp, halfhourly_data)

The map function will return a list of the result of get_date_region_and_rrp called with each row in halfhourly_data as an argument.

1
2
3
4

def get_date_region_and_rrp(row):
    return (row[4], row[6], row[7])

date_region_price = map(get_date_region_and_rrp, halfhourly_data)

The above section of code using map is the equivalent of this code using for loop.

1
2
3
4
5
6
7
8

def get_date_region_and_rrp(row):
    return (row[4], row[6], row[7])

date_region_price = []

for row in halfhourly_data:
    new_row = get_date_region_and_rrp(row)
    date_region_price.append(new_row)

map is an extremely versatile function. Those four lines of for loop code are replaced with one map line of code.

And now only the good data remains

The date_region_price variable will have these contents:

1
2
3
4
5

[["2012/04/02 10:30:00", "NSW1", "34.40"],
 ["2012/04/02 10:30:00", "QLD1", "34.67"],
 ["2012/04/02 10:30:00", "VIC1", "35.83"],
 ...
]

Only the columns that we are interested in. Nice work!

However there is still a small problem. All of the data values are still text. Update the get_date_region_and_rrp to convert the first column to a date, keep the second as a string and the third to a floating point value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

from __future__ import with_statement
import csv
import datetime
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

def is_halfhourly_data(row):
    """Returns True if the given row starts with 'D', 'TREGION', '', '1'"""
    return row[:4] == ["D", "TREGION", "", "1"]

halfhourly_data = filter(is_halfhourly_data, prices_csv_reader)

def get_date_region_and_rrp(row):
    """
    Returns the SETTLEMENTDATE, REGION and RRP from the given
    PUBLIC_PRICES CSV data row.

    SETTLEMENTDATE is converted to a Python date (the time is discarded);
    REGION is left as a string; and
    RRP is converted to a floating point.
    """
    return (datetime.datetime.strptime(row[4], '%Y/%m/%d %H:%M:%S').date(),
            row[6],
            float(row[7]))

date_region_price = map(get_date_region_and_rrp, halfhourly_data)

Great, now our data is in a format that Python can understand and plot. This is what is contained in the date_region_price value now:

1
2
3
4
5

[[datetime.date(2012, 4, 2), "NSW1", 34.40],
 [datetime.date(2012, 4, 2), "QLD1", 34.67],
 [datetime.date(2012, 4, 2), "VIC1", 35.83],
 ...
]

Perfect. We now have all the data formatted exactly the way we want it.

Conclusion

We’ve used filter and map to quickly and efficiently sort and slice the CSV data. Just like a master Japanese chef, I’m sure that you will not poison your patrons with bad slices of data. filter and map are advanced level functions that are often used to replace for loops. Please practice using map and filter. Experiment with them so that you understand how they work!

In the next, and final, blog post of this series we’ll show you how to plot the results using matplotlib.

The AESO manages the system with multiple generators unavailable [Larger Size]

Alberta, Canada was hit by rolling blackouts yesterday (9th July 2012) as the state sweltered through a heatwave that sent temperatures north of 30 degrees celcius.

The unexpected loss of power had some calling for an investigation into excessive use of market power.

How did the blackouts occur?

The AESO did not have enough available generation to securely meet the demand for electricity, so it switched off the power to some houses and businesses across the state until it could securely meet the demand.

Securely meet the demand

I’ll explain what I mean by securely meet the demand. The AESO is constantly asking the question: What is the worst thing that can happen next? There is always a list of things that could go wrong. And that list is as long as your arm:

• A generator could suddenly shut down without notice;
• A main powerline could fail and be unable to transport power;
• A major circuit breaker (a big power switch) could stop functioning;
• and more..

Securely operating the system means that, yes we are ok now, but we’ll still be ok even if one more thing goes wrong.

One of the tests that the AESO use to determine if they are secure is to ask: How much additional generating capacity do I have up my sleeve?

If the answer is “less than 500 MW” (about the size of one large gas power station) then they are not secure. They are in the danger zone, and if one more bad thing happens while they are in the danger zone, the entire power system could collapse. Restoring from a system collapse can take days.

So what happened yesterday?

Looking at the chart above, we can see that three additional generators went out of service at 1300 hrs. On the temperature chart darker red colours mean more generators simultaneously out of service.

It was then that the AESO declared Energy Emergency Alert 1 (shown on the demand chart. Darker colours mean a more severe alert level). This means, I still have about 500 MW, I hope nothing else bad happens, because we could go under 500 MW.

Soon after Alert 1 was sounded, one generator came back online, but then another two went offline. One step forward and two steps back.

This was enough to sound Energy Emergency Alert 2 and Alert 3 at almost the same time.

At Alert 2, customers that have previously agreed to help out by reducing their demand do so. And at Alert 3 there is no choice but to shut down entire suburbs at a time. Desparately clawing for a 500 MW buffer.

So AESO had no choice but to switch off load

But isn’t it strange that so many generators went offline at once? The price for electricity increased by 5000% (50 times higher). Those generator owners were earning a lot of money. C$ 57 million - for 6 hours work.

While the natural response is to call these generators out for being so greedy, take a look at the temperatures (for Edmonton). They are only a few degrees below the highest ever recorded temperatures. Most power equipment is only designed to withstand a certain temperature range. Operating outside of that range can damage the equipment.

A similar situation happened in Tasmania in 2009. The high voltage line connecting it and the mainland was not designed to cope with temperatures above 35 degrees. The shutdown of that cable, and other coincident shutdowns meant widespread rolling blackouts and 20,000 customers without power.

We’ll watch this case carefully, the AESO have already committed to a review but not to releasing the results (unlike the Australian Energy Market Operator that publicly releases investigations). Let’s hope that any lessons are learned and quickly implemented so that we do not see a repeat this summer.

Are you an engineer or market analyst? We’re running a blog series that teaches you everything you need to know about analysing the electricity market read it here. Register your email on the right → so you don’t miss any blog updates.

(This is the second part of a series. By the end we’ll have written a Python script to display a chart of electricity market prices.

• All you need to analyse the electricity market pt 1
• All you need to analyse the electricity market pt 2
• All you need to analyse the electricity market pt 3
• All you need to analyse the electricity market final

Australian electricity prices are high - let’s analyse

Previously I mentioned that the electricity prices have gone through the roof (more than doubled) post introduction of the carbon tax in Australia.

Very high electricity prices [Larger Size]

We’re using these high prices as an excuse to learn how to download and display price data from the internet. This code will work wherever you are in the world.

Using Python to extract a zipped file

Last time, we downloaded a zipped file filled with electricity prices from the energy market operator’s website (read about it here if you want to catch up). Here is the final code from that post:

1
2
3
4
5
6
7
8
9
10

from __future__ import with_statement
from urllib2 import urlopen

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL)

with open('PUBLIC_PRICES.ZIP', 'wb') as pricesfile:
  pricesfile.write(zippedfile.read())

In this post we are going to open a zip file containing CSV files and read the data contained within. To begin, we open the zipped file and get a list of the file names inside it. We’ve used Python’s zipfile module and its ZipFile class to turn the zip file we downloaded into something that we can use.

A listing of all the filenames in the zip file can be produced using the namelist() method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

from __future__ import with_statement
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))
filenames = opened_zipfile.namelist()

print filenames

1
2

# just one CSV file inside this zipped file.
['PUBLIC_PRICES_201207040000_20120705040607.CSV']

There was just one CSV file inside the zipped file we just opened. We will now add some code to open it and convert it to Python lists using the csv module included with Python. Here is a quick refresh on the CSV module.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

from __future__ import with_statement
import csv
from urllib2 import urlopen
from StringIO import StringIO
from zipfile import ZipFile

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

# zippedfile is now one long string.
zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL).read()

# StringIO turns the string into a real file-like object.
opened_zipfile = ZipFile(StringIO(zippedfile))

# assuming there is only one CSV in the zipped file.
csv_filename = opened_zipfile.namelist()[0]

prices_csv_file = opened_zipfile.open(csv_filename)

prices_csv_reader = csv.reader(prices_csv_file)

csv.reader creates an object that has special functions to effortlessly return the CSV price data as Python lists. Printing the object, print prices_csv_reader, shows how the csv module has read in the file:

1
2
3
4

[['C', 'NEMP.WORLD', 'PRICES', 'AEMO',  ...]
 ['I', 'DREGION', '', '2', 'SETTLEMENTDATE', 'RUNNO', 'REGIONID', ...]
  ...
]

Look carefully at that output, notice how csv.reader has detected all the commas and correctly separated all of the columns and rows. All the data is stored in a list of rows. Now that we have the data in this form, extracting useful data from this structure can be achieved using the usual Python techniques.

In the first two posts of this series, we’ve:

1. downloaded a zipped file using urllib2;
2. unzipped the file to get the CSV using zipfile;
3. got ready to read the csv file contents using csv.

In the next blog post we’ll take a closer look at the CSV file using Python. In particular, we will manipulate the CSV file to extract only the data that is of interest to us.

If you want to be emailed when the next blog is ready, just enter your email up the top →. We’ll only email you when a new blog is ready and with tips on becoming an advanced Python using power systems engineer.

(This is the first part of a series. By the end we’ll have written a Python script to display a chart of electricity market prices.

• All you need to analyse the electricity market pt 1
• All you need to analyse the electricity market pt 2
• All you need to analyse the electricity market pt 3
• All you need to analyse the electricity market final

Electricity prices in Australia have gone bananas

Since the carbon tax was introduced in Australia, the spot price for electricity has been very high. From $22 / MWh last month to over $50 / MWh so far (see the chart below). Whether this is a long term change, or just a temporary reaction, we don’t know. Instead of considering the complexities of market forces, we’re going to show you step by step exactly how to build your own Python program to display electricity prices, using Australia as an example.

Very high electricity prices [Larger Size]

Feel free to take the code, and configure it for your own country’s system.

Downloading a zip file.

Australia’s electricity market operator (AEMO) keeps all of the market data online at http://www.nemweb.com.au/Reports/. The webpages are written in simple to understand HTML and link to zipped CSV files.

We’ll use Python to write a simple script that downloads a zipped file.

1
2
3
4
5
6
7
8
9
10

from __future__ import with_statement
from urllib2 import urlopen

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL)

with open('PUBLIC_PRICES.ZIP', 'wb') as pricesfile:
  pricesfile.write(zippedfile.read())

You may find that the link is expired, so you’ll get an error like this:

1
2
3
4
5
6
7
8
9
10
11
12
13

  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found

Got the error above? Change ZIP_URL to match the url of a zip you can find on this http://www.nemweb.com.au/Reports/CURRENT/Public_Prices page.

Unzip the downloaded file (the file is named PUBLIC_PRICES.ZIP) and look at the CSV file inside. You will find something like this:

1
2
3
4
5
6
7

C,NEMP.WORLD,PRICES,AEMO,PUBLIC,2012/04/02,04:06:14,0000000235749444,,0000000235749438
I,DREGION,,2,SETTLEMENTDATE,RUNNO,REGIONID,INTERVENTION,RRP,EEP,ROP,APCFLAG,MARKETSUSPENDEDFLAG,TOTALDEMAND,DEMANDFORECAST,DISPATCHABLEGENERATION,DISPATCHABLELOAD,NETINTERCHANGE,EXCESSGENERATION,LOWER5MINDISPATCH,LOWER5MINIMPORT,LOWER5MINLOCALDISPATCH,LOWER5MINLOCALPRICE,LOWER5MINLOCALREQ,LOWER5MINPRICE,LOWER5MINREQ,LOWER5MINSUPPLYPRICE,LOWER60SECDISPATCH,LOWER60SECIMPORT,LOWER60SECLOCALDISPATCH,LOWER60SECLOCALPRICE,LOWER60SECLOCALREQ,LOWER60SECPRICE,LOWER60SECREQ,LOWER60SECSUPPLYPRICE,LOWER6SECDISPATCH,LOWER6SECIMPORT,LOWER6SECLOCALDISPATCH,LOWER6SECLOCALPRICE,LOWER6SECLOCALREQ,LOWER6SECPRICE,LOWER6SECREQ,LOWER6SECSUPPLYPRICE,RAISE5MINDISPATCH,RAISE5MINIMPORT,RAISE5MINLOCALDISPATCH,RAISE5MINLOCALPRICE,RAISE5MINLOCALREQ,RAISE5MINPRICE,RAISE5MINREQ,RAISE5MINSUPPLYPRICE,RAISE60SECDISPATCH,RAISE60SECIMPORT,RAISE60SECLOCALDISPATCH,RAISE60SECLOCALPRICE,RAISE60SECLOCALREQ,RAISE60SECPRICE,RAISE60SECREQ,RAISE60SECSUPPLYPRICE,RAISE6SECDISPATCH,RAISE6SECIMPORT,RAISE6SECLOCALDISPATCH,RAISE6SECLOCALPRICE,RAISE6SECLOCALREQ,RAISE6SECPRICE,RAISE6SECREQ,RAISE6SECSUPPLYPRICE,AGGREGATEDISPATCHERROR,AVAILABLEGENERATION,AVAILABLELOAD,INITIALSUPPLY,CLEAREDSUPPLY,LOWERREGIMPORT,LOWERREGLOCALDISPATCH,LOWERREGLOCALREQ,LOWERREGREQ,RAISEREGIMPORT,RAISEREGLOCALDISPATCH,RAISEREGLOCALREQ,RAISEREGREQ,RAISE5MINLOCALVIOLATION,RAISEREGLOCALVIOLATION,RAISE60SECLOCALVIOLATION,RAISE6SECLOCALVIOLATION,LOWER5MINLOCALVIOLATION,LOWERREGLOCALVIOLATION,LOWER60SECLOCALVIOLATION,LOWER6SECLOCALVIOLATION,RAISE5MINVIOLATION,RAISEREGVIOLATION,RAISE60SECVIOLATION,RAISE6SECVIOLATION,LOWER5MINVIOLATION,LOWERREGVIOLATION,LOWER60SECVIOLATION,LOWER6SECVIOLATION,RAISE6SECRRP,RAISE6SECROP,RAISE6SECAPCFLAG,RAISE60SECRRP,RAISE60SECROP,RAISE60SECAPCFLAG,RAISE5MINRRP,RAISE5MINROP,RAISE5MINAPCFLAG,RAISEREGRRP,RAISEREGROP,RAISEREGAPCFLAG,LOWER6SECRRP,LOWER6SECROP,LOWER6SECAPCFLAG,LOWER60SECRRP,LOWER60SECROP,LOWER60SECAPCFLAG,LOWER5MINRRP,LOWER5MINROP,LOWER5MINAPCFLAG,LOWERREGRRP,LOWERREGROP,LOWERREGAPCFLAG
D,DREGION,,2,"2012/04/01 04:05:00",1,NSW1,0,24.01213,0,24.01213,0,,5880.16,1.20264,4358.97,0,-1521.19,0,,,100.55,,,,,,,,109.7,,,,,,,,34.35,,,,,,,,206,,,,,,,,94.27,,,,,,,,116.69,,,,,,0,10832.745,0,5907.18945,5912.38,,50,,,,38,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0
D,DREGION,,2,"2012/04/01 04:05:00",1,QLD1,0,21.55498,0,21.55498,0,,4423.12,-0.58301,5327.25,0,904.13,0,,,13,,,,,,,,42.65,,,,,,,,40,,,,,,,,83,,,,,,,,82,,,,,,,,62,,,,,,0,10258,0,4442.88184,4441.41,,34,,,,76.44,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0
D,DREGION,,2,"2012/04/01 04:05:00",1,SA1,0,22.2124,0,22.2124,0,,1144.89,-1.98004,1085.43,0,-59.46,0,,,0,,,,,,,,2,,,,,,,,2,,,,,,,,0,,,,,,,,30,,,,,,,,40.4,,,,,,4.93256,2768.82735,0,1141.90503,1144.93,,16,,,,0,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0
D,DREGION,,2,"2012/04/01 04:05:00",1,TAS1,0,30.8637,0,30.8637,0,,896.9,3.52316,469.4,0,-427.5,0,,,47.74,,,,,,,,0,,,,,,,,13.59,,,,,,,,80.29,,,,,,,,73.59,,,,,,,,48.77,,,,,,-0.61674,2000,0,893.98877,896.9,,0,,,,6.53,,,,,,,,,,,,,,,,,,,1.1,1.1,0,0.9,0.9,0,2.3,2.3,0,2.8,2.8,0,0.1,0.1,0,0,0,0,0.1,0.1,0,2.46391,2.46391,0
D,DREGION,,2,"2012/04/01 04:05:00",1,VIC1,0,21.7,0,21.7,0,,4094.7,-22.9043,5268.16,0,1173.46,0,,,46,,,,,,,,40,,,,,,,,40,,,,,,,,11,,,,,,,,33,,,,,,,,45,,,,,,0,8780,0,4134.81494,4113.61,,20,,,,9.03,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0

The file is very long. In fact there are four separate CSV files crammed into this one long CSV file. In the next blog post we’ll examine exactly how to unzip the file you just downloaded with Python and then use one of these CSV files.

If you want to be emailed when the next blog is ready, just enter your email up the top →. We’ll only email you when a new blog is ready and with tips on becoming an advanced Python using power systems engineer.

Exporting plots from the PSSE graphical interface can be a laborious task, with little flexibility in the final result. If you are running PSSE from Python, you have the option of looking beyond PSSE for tools to render your data.

matplotlib is a Python plotting tool which, given a little bit of effort, can make your life easier by automating figure generation; spend the time to get it right once, then generate the monthly figures with the press of a button.

matplotlib’s syntax is closely aligned with Matlab’s, which is fortunate if you are familiar with Matlab. If you are not familiar with them, don’t worry, the matplotlib community has provide an excellent tutorial and gallery displaying the immense possibilities of matplotlib (all examples come with code included for you to use as a starting point).

I’ll use matplotlib to create this chart:

It is the QV curve at bus 20001 (my favourite bus as it happens) and was plotted from data kept in a CSV file (qv.csv).

Here is the code we used to create that chart.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

from __future__ import with_statement
import csv

from pylab import plot, title, show, grid, xlabel, ylabel

# CSV file in format: V, Q
QV_CSVFILENAME = 'qv.csv'

def main():
  with open(QV_CSVFILENAME) as qvcsvfile:
      reader = csv.reader(qvcsvfile)
      # skip header row.
      header = reader.next()
      voltage_and_reactive = list(reader)

  # now convert the voltages and power values to floats.
  voltages, reactive = [], []
  for voltage, q in voltage_and_reactive:
      voltages.append(float(voltage))
      reactive.append(float(q))

  # now make the chart.
  plot(voltages, reactive, '-o')
  title("QV curve for bus number 20001")
  ylabel("Reactive Power [MVAr]")
  xlabel("Voltage [pu]")
  grid()
  show()

if __name__ == '__main__':
  main()

I’m using the pylab module of matplotlib. The pylab module most closely resembles Matlab code so its a really familiar place to start learning how to use matplotlib.

The first two sections open up the CSV file and convert it into a list of voltage and reactive floating point values.

Now, I’ll pull a section of that code out so that we can look closely:

1
2
3
4
5
6
7

   # now make the chart.
  plot(voltages, reactive, '-o')
  title("QV curve for bus number 20001")
  ylabel("Reactive Power [MVAr]")
  xlabel("Voltage [pu]")
  grid()
  show()

Thats the code that actually turns the lists of voltages and reactive values into a chart. The functions describe themselves pretty well. At the end of the routine show will pop up an interactive graph.

With the interactive graph you can:

• zoom in and out;
• pan around; and
• export the chart to an image file.

If you haven’t ever used matplotlib go ahead and try this example out. Before you do run the Python code here, just make sure you have both matplotlib and numpy installed. Install numpy first because matplotlib definitely needs it and will not install without it.

You’ve seen how easy it can be to start making your own charts with matplotlib. The quality of the resulting images is really exceptional. We’ve used them throughout many industry published documents. If you have any questions about making your own charts, jump onto our forum for power systems engineers we’ll help you out.

If you made it this far, you must be pretty keen on Python. We are too! Let us update you when we have something new to say about Python and Power Systems (semi regularly). Get Python Tips (subscribe on the right) →

Sri Lanka’s peak electricity demand of 2000 MW is growing at a rapid 5%-8% every year.

To keep pace with the increase, the Ceylon Electricity Board (CEB) have developed a clever plan. Over the next 10 years, over 400 MW of mini hydro and wind farms will be installed. It is anticipated the additional generation will meet the increased demand levels and reduce Sri Lanka’s dependance on foreign fossil fuel sources, such as coal, natural gas and petroleum.

Sri Lanka currently has over 200 MW of mini hydro power stations installed, which are not dispatched centrally, but shown as a reduction in demand. Additionally, there is 30 MW of wind generation installed and another 60 MW soon to be connected.

We met with Mr. Shihan Diddeniya, CEO of renewable energy at CEB, to learn more about Sri Lanka’s plan to increase the capacity of intermittent renewable generation.

Rapid growth in demand

Mr. Diddeniya explained that Sri Lanka currently has about 88% electrification. That is, 88% of households in Sri Lanka have direct access to the electricity grid. The proportion of electrified homes is growing quickly with complete electrification likely to occur over the next few years. The demand growth rate is a staggering 8% and once total electrification is complete, growth will remain at a high 5%.

A surplus of generation at low demand periods

Shihan told of how the challenge for Sri Lanka is not with fault level contributions from embedded mini hydros or building lines to remote wind farms. He emphasized Sri Lanka’s challenge is with excess generation capacity during low demand periods where electricity usage drops dramatically below the 2000 MW peak to 860 MW.

Mini hydro and wind power plant output cannot be reduced because their power is given first preference. As an example, if run of the river mini hydro and wind power plants were to be producing 400 MW, then only 460 MW remains to be supplied by thermal coal stations. The surplus generation could be resolved by switching off coal fired stations. But coal power stations are limited in their ability to switch on and off quickly, it may take days for a coal power station to run at full capacity from a cold start.

Sri Lanka will soon have 100% electrification

A new link to India

A transmission interconnect between Sri Lanka and India, which has been contemplated since the 70’s, is expected to be started after 2013. The 285km, high voltage DC link will have a total capacity of 1000MW, to be installed in two 500MW stages.

The extra capacity made available by this link will go a long way to relieve the immediate pressures the grid is currently experiencing.

Mr. Diddeniya will raise funds to complete the new renewable energy projects from local and international banks and investors during the next few years.

These old style meters only report your electricity once in three months. Smart meters record monitor your electricity every half hour.

Do you have a smart meter?

Families commonly ask “How do I know if we have a smart meter?” The answer is that you are notified well in advance by your retailer that the installation will take place, and information will be left for you in your mailbox following the installation. Our story begins with the installation of a smart meter at the little guy’s family house.

Why is the data valuable?

Having worked in the industry, the little guy knew that the smart meter data was extremely valuable. With it he could shop around and get a better deal on his electricity - saving up to $100 / year on his bill. He can also see when he uses the most electricity, and whether or not it is during the most expensive part of the day, where customers are slugged double the price [1].

Here is how he got an Excel Spreadsheet of the valuable smart meter output emailed to him directly every quarter.

Contact his electricity company by email

Let them know you have a smart meter installed, you are a happy customer and would like to have regular access to your energy usage so you can save on your energy bill.

Here are the contact details of some of the retailers:

• Red Energy: enquiries@redenergy.com.au
• Origin Energy: https://www.originenergy.com.au/3244/Interval-meter-data-request
• Tru Energy: https://secure.truenergy.com.au/customerforms/LodgeEnquiry.aspx
• AGL: https://online.agl.com.au/enquiry_form.do

Bear in mind many retailers haven’t had a great deal of experience with smart meters yet, and may not have the systems in place to send you data in a timely manner.

Gently remind them of their obligations under the Retail Code

The little guy got pushback from his retailer who claimed there was no requirement to send data more than once per year. That was the old rule, now there is a new rule that says they must provide whenever you request it and no mention of payments being necessary. Be brave, ask for them to send your data every day. For them, sending it once a month (by comparison) is a good compromise.

What can you do with your data?

Do you use electricity during the most expensive part of the day? [full size →]

Run a chart like the one here showing the ‘average value’ which is just your daily electricity use and your retailer’s time of use pricing plans in red. The huge jump in the red line is where you’ll pay the most to your electricity company.

Shop your data around, get together with 30 or so other little guys and form a cartel. You can shop the business of 30 people better than you can individually. This is what the big players like 7/11 and McDonalds do to save a fortune on their electricity costs.

Do you have any success stories about getting your data? Do you want to see your family or small business energy use on a chart like the one above? Leave a comment or email us directly at hello@whit.com.au.

[1] time of use pricing. Most of you won’t be on a time of use pricing plan yet because many retailers have been slow to introduce these plans. Your current plan is likely to be one where you pay a flat rate (say 21 cents / kWh) all day and all year round - even during winter when the market prices for electricity is low and your retailer makes a fortune from you.

With the time of use plans you pay 10 cents / kWh over night, 18 cents during most of the day and 40 cents in the afternoon. “Complicated?”, yes it is complicated, but you could save a lot of money every year if you avoid using electricity during the afternoon. The red line in the chart above shows the price difference throughout a day. I have heard that these plans are a year or two away and seasonal pricing may take even longer to introduce.

Silence!

PSSE prints a voluminous amount of information to the command line when you are running a program. Do you remember seeing this when you start PSSE up?

1
2
3
4
5
6

PSS«E Version 32
Copyright (c) 1976-2012
Siemens Energy, Inc.,
Power Technologies International (PTI)
This program is a confidential unpublished work created and first
licensed in 1976. It is a trade secret which is the property of PTI

We will show you a little function that you can use to selectively turn off this output (and other PSSE output) or redirect it to a log file.

This blog post is in response to a question by chip on the python for power systems engineers forum. Chip wanted to ignore some of the output that PSSE was printing without ignoring all of the program’s output.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

from __future__ import with_statement
from contextlib import contextmanager
import sys
import os

@contextmanager
def silence(file_object=None):
    """
    Discard stdout (i.e. write to null device) or
    optionally write to given file-like object.
    """
    if file_object is None:
        file_object = open(os.devnull, 'w')

    old_stdout = sys.stdout
    try:
        sys.stdout = file_object
        yield
    finally:
        sys.stdout = old_stdout

The function silence can be used to selectively turn off PSSE output printing in your program. Use it like this:

1
2

with silence():
  psspy.psseinit(10000)

Printing from within the context block (the with statement) will be ignored. (Note that we have initialised PSSE with buses set to ten thousand - based on our research of the optimal bus number).

What if you must redirect the PSSE output to a file? Got you covered. Use the silence function like this:

1
2
3

psse_log = open('psse_logfile.log', 'w')
with silence(psse_log):
  psspy.psseinit(10000)

There are more juicy details about how the context manager works on the forum: Silencing PSSE Stdout

edit: You must also redirect PSSE output to Python using the redirect module included with every PSSE installation:

1
2
3
4
5
6

import redirect
redirect.psse2py()

psse_log = open('psse_logfile.log', 'w')
with silence(psse_log):
  psspy.psseinit(10000)

..when you have eliminated the impossible, whatever remains, however improbable, must be the truth

This is a story about psspy.psseinit and how its buses argument is not so innocent and will one day commit a terrible crime on your study.

Have you ever sat and wondered why PSSE insists on asking for a buses argument when initialising? Did you notice that it isn’t optional? I’ve sat at my computer many times thinking: “Will I need 300, 1000 or 10,000 buses?”

What is the buses argument?

According to the API guide in section 12.13: The buses argument is the requested bus size. Not especially helpful, so in plain english we say the buses argument is:

buses - The maximum number of buses allowed in your study

Why have a maximum number?

PSSE was written in a time when creating a busmatrix in the computer’s memory was a difficult task. In those days, once you had picked a matrix size, thats was it, there was no second chance to resize it. So picking the maximum number of buses was very important. Choose a huge matrix and your poor computer’s memory would fill up, choose one too small then the matrix wouldn’t fit all of your buses.

PSSE uses more memory when you initialise more buses

We decided to put this to the test. The PSSE documentation states that the maximum number of buses it can handle is 150,000. So we initialised PSSE over and over and over with a maximum of 10 buses up to 1 million and measured how much memory it used.

The results are clear, allocating more maximum buses uses more memory until 150,000 when allocating more buses has no effect - because the additional buses beyond 150,000 are ignored.

The terrible crime and how to avoid it

Because of the history when PSSE was written, you must pick the correct size. If you pick a maximum number of 100 buses, and your raw file has 120 buses then 20 of those buses will not be loaded. Unlike for raw files, our experiments have shown that loading a saved case (.sav file) resets the maximum bus size to the one used in the sav case.

So some advice: be sure to set a maximum bus size large enough to handle your system size, but not too large otherwise PSSE will gobble up all of your memory. We suggest that 10,000 buses is enough, if you regularly use more than 10,000 buses let us know in the comments.

The following lines of code are the black magic which we aim to address:

1
2
3
4

import os, sys
sys.path.append(r"C:\Program Files\PTI\PSSE32\PSSBIN")
os.environ['PATH'] = (r"C:\Program Files\PTI\PSSE32\PSSBIN;"
                        + os.environ['PATH'])

The above code only works on systems where the PSSBIN folder is located at "C:\Program Files\PTI\PSSE32\PSSBIN". If you were to share this script with a colleague who was running on 64 bit install of windows ("C:\Program Files x86\PTI\PSSE32\PSSBIN") or a different version of PSSE ("C:\Program Files\PTI\PSSE33\PSSBIN"), then the script would fail to run.

pssepath solves this problem by automatically configuring these settings with a single line of code. The project is hosted on github and based on an idea proposed by chip in our Python for power systems engineers Q&A site (the answer that started it all). The goal is to be able to setup and import psspy on any computer with three simple lines of code:

1

import pssepath

1
2

pssepath.add_pssepath()
import psspy

Thats it! Now you can share your code with anyone and get back to doing productive work!

To use pssepath, download the zip from the pssepath download, extract the pssepath.py file and place it in the same folder as the script you want to run. We are working to make installation global (so you won’t have to place it in every folder) and much easier, so check back soon for developments.

The code is licensed to be compatible for use within a commercial environment. If you have any problems using pssepath, let us know about your difficulty and we will work with you to get it fixed for everyone. Alternatively, as the project is open source, you are free to take our code and do whatever you want with it (but if you do something really cool, please share it with the rest of us :)