Python for Power Systems

A blog for power systems engineers to learn Python.

All You Need to Analyse the Electricity Market

If you are an electrical engineer, and want to know how to use Python to get data from the internet and display it, this post is for you.

(This is the first part of a series. By the end we’ll have written a Python script to display a chart of electricity market prices.

Electricity prices in Australia have gone bananas

Since the carbon tax was introduced in Australia, the spot price for electricity has been very high. From $22 / MWh last month to over $50 / MWh so far (see the chart below). Whether this is a long term change, or just a temporary reaction, we don’t know. Instead of considering the complexities of market forces, we’re going to show you step by step exactly how to build your own Python program to display electricity prices, using Australia as an example.

Very high electricity prices [Larger Size]

Feel free to take the code, and configure it for your own country’s system.

Downloading a zip file.

Australia’s electricity market operator (AEMO) keeps all of the market data online at http://www.nemweb.com.au/Reports/. The webpages are written in simple to understand HTML and link to zipped CSV files.

We’ll use Python to write a simple script that downloads a zipped file.

downloadzip.py
1
2
3
4
5
6
7
8
9
10

from __future__ import with_statement
from urllib2 import urlopen

PRICE_REPORTS_URL = 'http://www.nemweb.com.au/Reports/CURRENT/Public_Prices'
ZIP_URL = '/PUBLIC_PRICES_201207040000_20120705040607.ZIP'

zippedfile = urlopen(PRICE_REPORTS_URL + ZIP_URL)

with open('PUBLIC_PRICES.ZIP', 'wb') as pricesfile:
  pricesfile.write(zippedfile.read())

You may find that the link is expired, so you’ll get an error like this:

error
1
2
3
4
5
6
7
8
9
10
11
12
13

  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found

Got the error above? Change ZIP_URL to match the url of a zip you can find on this http://www.nemweb.com.au/Reports/CURRENT/Public_Prices page.

Unzip the downloaded file (the file is named PUBLIC_PRICES.ZIP) and look at the CSV file inside. You will find something like this:

PUBLIC_PRICES.ZIP
1
2
3
4
5
6
7

C,NEMP.WORLD,PRICES,AEMO,PUBLIC,2012/04/02,04:06:14,0000000235749444,,0000000235749438
I,DREGION,,2,SETTLEMENTDATE,RUNNO,REGIONID,INTERVENTION,RRP,EEP,ROP,APCFLAG,MARKETSUSPENDEDFLAG,TOTALDEMAND,DEMANDFORECAST,DISPATCHABLEGENERATION,DISPATCHABLELOAD,NETINTERCHANGE,EXCESSGENERATION,LOWER5MINDISPATCH,LOWER5MINIMPORT,LOWER5MINLOCALDISPATCH,LOWER5MINLOCALPRICE,LOWER5MINLOCALREQ,LOWER5MINPRICE,LOWER5MINREQ,LOWER5MINSUPPLYPRICE,LOWER60SECDISPATCH,LOWER60SECIMPORT,LOWER60SECLOCALDISPATCH,LOWER60SECLOCALPRICE,LOWER60SECLOCALREQ,LOWER60SECPRICE,LOWER60SECREQ,LOWER60SECSUPPLYPRICE,LOWER6SECDISPATCH,LOWER6SECIMPORT,LOWER6SECLOCALDISPATCH,LOWER6SECLOCALPRICE,LOWER6SECLOCALREQ,LOWER6SECPRICE,LOWER6SECREQ,LOWER6SECSUPPLYPRICE,RAISE5MINDISPATCH,RAISE5MINIMPORT,RAISE5MINLOCALDISPATCH,RAISE5MINLOCALPRICE,RAISE5MINLOCALREQ,RAISE5MINPRICE,RAISE5MINREQ,RAISE5MINSUPPLYPRICE,RAISE60SECDISPATCH,RAISE60SECIMPORT,RAISE60SECLOCALDISPATCH,RAISE60SECLOCALPRICE,RAISE60SECLOCALREQ,RAISE60SECPRICE,RAISE60SECREQ,RAISE60SECSUPPLYPRICE,RAISE6SECDISPATCH,RAISE6SECIMPORT,RAISE6SECLOCALDISPATCH,RAISE6SECLOCALPRICE,RAISE6SECLOCALREQ,RAISE6SECPRICE,RAISE6SECREQ,RAISE6SECSUPPLYPRICE,AGGREGATEDISPATCHERROR,AVAILABLEGENERATION,AVAILABLELOAD,INITIALSUPPLY,CLEAREDSUPPLY,LOWERREGIMPORT,LOWERREGLOCALDISPATCH,LOWERREGLOCALREQ,LOWERREGREQ,RAISEREGIMPORT,RAISEREGLOCALDISPATCH,RAISEREGLOCALREQ,RAISEREGREQ,RAISE5MINLOCALVIOLATION,RAISEREGLOCALVIOLATION,RAISE60SECLOCALVIOLATION,RAISE6SECLOCALVIOLATION,LOWER5MINLOCALVIOLATION,LOWERREGLOCALVIOLATION,LOWER60SECLOCALVIOLATION,LOWER6SECLOCALVIOLATION,RAISE5MINVIOLATION,RAISEREGVIOLATION,RAISE60SECVIOLATION,RAISE6SECVIOLATION,LOWER5MINVIOLATION,LOWERREGVIOLATION,LOWER60SECVIOLATION,LOWER6SECVIOLATION,RAISE6SECRRP,RAISE6SECROP,RAISE6SECAPCFLAG,RAISE60SECRRP,RAISE60SECROP,RAISE60SECAPCFLAG,RAISE5MINRRP,RAISE5MINROP,RAISE5MINAPCFLAG,RAISEREGRRP,RAISEREGROP,RAISEREGAPCFLAG,LOWER6SECRRP,LOWER6SECROP,LOWER6SECAPCFLAG,LOWER60SECRRP,LOWER60SECROP,LOWER60SECAPCFLAG,LOWER5MINRRP,LOWER5MINROP,LOWER5MINAPCFLAG,LOWERREGRRP,LOWERREGROP,LOWERREGAPCFLAG
D,DREGION,,2,"2012/04/01 04:05:00",1,NSW1,0,24.01213,0,24.01213,0,,5880.16,1.20264,4358.97,0,-1521.19,0,,,100.55,,,,,,,,109.7,,,,,,,,34.35,,,,,,,,206,,,,,,,,94.27,,,,,,,,116.69,,,,,,0,10832.745,0,5907.18945,5912.38,,50,,,,38,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0
D,DREGION,,2,"2012/04/01 04:05:00",1,QLD1,0,21.55498,0,21.55498,0,,4423.12,-0.58301,5327.25,0,904.13,0,,,13,,,,,,,,42.65,,,,,,,,40,,,,,,,,83,,,,,,,,82,,,,,,,,62,,,,,,0,10258,0,4442.88184,4441.41,,34,,,,76.44,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0
D,DREGION,,2,"2012/04/01 04:05:00",1,SA1,0,22.2124,0,22.2124,0,,1144.89,-1.98004,1085.43,0,-59.46,0,,,0,,,,,,,,2,,,,,,,,2,,,,,,,,0,,,,,,,,30,,,,,,,,40.4,,,,,,4.93256,2768.82735,0,1141.90503,1144.93,,16,,,,0,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0
D,DREGION,,2,"2012/04/01 04:05:00",1,TAS1,0,30.8637,0,30.8637,0,,896.9,3.52316,469.4,0,-427.5,0,,,47.74,,,,,,,,0,,,,,,,,13.59,,,,,,,,80.29,,,,,,,,73.59,,,,,,,,48.77,,,,,,-0.61674,2000,0,893.98877,896.9,,0,,,,6.53,,,,,,,,,,,,,,,,,,,1.1,1.1,0,0.9,0.9,0,2.3,2.3,0,2.8,2.8,0,0.1,0.1,0,0,0,0,0.1,0.1,0,2.46391,2.46391,0
D,DREGION,,2,"2012/04/01 04:05:00",1,VIC1,0,21.7,0,21.7,0,,4094.7,-22.9043,5268.16,0,1173.46,0,,,46,,,,,,,,40,,,,,,,,40,,,,,,,,11,,,,,,,,33,,,,,,,,45,,,,,,0,8780,0,4134.81494,4113.61,,20,,,,9.03,,,,,,,,,,,,,,,,,,,0.96,0.96,0,0.39,0.39,0,0.85,0.85,0,1.35,1.35,0,0.5,0.5,0,1.5,1.5,0,3.85809,3.85809,0,6.222,6.222,0

The file is very long. In fact there are four separate CSV files crammed into this one long CSV file. In the next blog post we’ll examine exactly how to unzip the file you just downloaded with Python and then use one of these CSV files.

If you want to be emailed when the next blog is ready, just enter your email up the top →. We’ll only email you when a new blog is ready and with tips on becoming an advanced Python using power systems engineer.