Sunday, 20 April 2014

How to upload data from Growatt inverter to pvoutput.org using a Raspberry Pi.

Uploading data from your solar inverter to a site like pvoutput.org can have have the benefit of letting you monitor your power output from anywhere, comparing data with other systems around the world from various other vendors or just simply to keep track.


I have prepared a walk-through on how to use a Raspberry Pi to upload your Growatt inverter data together with weather data to pvoutput.org. More technical skilled people might find this guide too detailed - they are welcome to grab the goodies at the end or ignore the whole thing :-)

Background


I have 2 Growatt 3000 inverters (this setup probably also work with any other Growatt or Sungold inverter) and 6000kW panels.

I did initially setup an old Windows XP PC with 2 COM ports - running PVBeancounter which ran fine for a couple of months - it nagged me having a full PC running a XP license stuffed with a .Net framework - only to get my inverter data uploaded. So having a Raspberry Pi laying around I did plan to use that instead at a point in time - when I got the time to look into it. I was being stupid enough to promise to help someone else with a similar setup (to keep the pressure on myself) - a very patient guy indeed :-)

I did make a small Object Pascal console program to get the data from the inverter at that time - not using the mod-bus protocol - but then got busy elsewhere.

When I finally came around to look at the setup again - I did a quick search, and found out that once again if you wait long enough - someone else has found a solution to help you scratch your itch. Credit should go to Andrew Elwell for his initial explorations and Kuhlmeier|Sisand - who has a walk-through of his setup and also has some other interesting projects.

Since our setups are not identical and I wanted things a bit different - I have done my own setup guide trying to target people who don't really have that much Raspberry Pi or Linux experience. But they should at least have a PC with Windows and be able to use it.

Ingredients needed

  • 1 Raspberry Pi Model B
  • 1 SD card (4 GB or more) - preferable Class 10
  • 1 microUSB power adapter
  • 1 USB-RS232 adapter with the number of RS232 ports that equals your number of inverters. If you have more than 1 inverter you might additional need an extension RS232 cable. Do not buy these too cheap - cables with PL2303 chips seems ok.
  • 1 USB WiFi adapter or Ethernet cable
  • 1 USB keyboard and mouse
  • 1 USB hub (if you easily need to setup the WiFi adapter up on an encrypted network)
  • 1 SD card reader (you might have one in your laptop)
  • Monitor/TV with hdmi or composite video, along with cable.
The ingredients list might look a bit too "detailed" - but some people only have laptops or tablets. If you do have the possibility of cabled networking at your inverter(s) and do not want/need WiFi - leave out the USB hub and mouse.

Setup steps

Part 1 - In front of the telly.
  1. Download the Operating System image file for your Raspberry Pi here: http://downloads.raspberrypi.org/raspbian_latest
  2. Follow the guide to write the image to the SD card shown here.
  3. Register your solar system at pvoutput.org and get your API key located at the bottom of the settings page, also remember to enabled the API access just above the key. Write down your systems id.
  4. If you want weather and temperature data along with your solar data, register at openweathermap.org and get your API key. Use Google map or something else to find your longitude and latitude, if you want near correct weather data :-)
  5. While the SD card is still in your computer, create a new file named pvoutput.txt using notepad on the SD card with the following content (replace the content between the < > with your values:
    1. # Register at pvoutput.org to get your SYSTEMID and APIKEY
        SYSTEMID=<Your 5-digit pvoutput.org systems id>
          APIKEY=<Your long API key from pvoutput.org>
            # Numbers of inverters
              Inverters=<The number of inverters you have, 1 or 2>
                # Register at openweather.org to get your APIKEY
                  # If OWMKEY and Longitude and Latitude is not supplied
                    # no weather will be read, and tried uploaded to 
                      # pvoutput.org - together with your energy data.
                        OWMKEY=<Your openweather.org API key>
                          Latitude=<Latitude of your panels location>
                            Longitude=<Longitude of your panels location>
                          1. Also copy the Python code to the SD card - you can download it from here.
                          2. Insert the SD card, keyboard/mouse/hub, network cable/adapter, monitor into your Raspberry Pi and add power  to it on.
                          3. At the "login as" prompt, log in as pi and password is raspberry - you might want to change that later.
                          4. Run sudo raspi-config to enable SSH (option 8-A4) - you might also want to setup timezone and keyboard (option 4), and use the whole SD card (option 1) - then finish.
                          5. If you need a WiFi adapter to be setup - a easy way is to use the graphical wifi-manager - so run startx and run the wifi-manager, entering the password for your network. Logout afterwards.
                          6. Write down the IP-address of your Raspberry Pi - run ifconfig and look for the inet addr for eth0 (or wlan0 if WiFi).
                          7. Back at the prompt power off the Raspberry Pi by entering the command: sudo poweroff - you should never just cut the power - it might corrupt the data on your SD card.
                          Part 2 - Finishing the setup remotely.
                          1. Now remove the keyboard/mouse/hub from the Raspberry Pi, and insert the USB-RS232 cable.
                          2. Power on the Raspberry Pi again - and download a terminal program like PuTTY, so that you can access your Raspberry Pi remotely.
                          3. Start PuTTY and enter the IP-address of the Raspberry Pi, select SSH and Open - and accept the SSH host key message. Log in as previously.
                          4. Now check if the USB-RS232 is detected by the Raspberry Pi - run lsusb - and you should something similar to below - the key here is that you have Serial Port(s) listed:
                          5. Now we want to update the OS and install the needed packages - by running the following commands:
                            1. sudo apt-get update
                              sudo apt-get upgrade
                              sudo apt-get install build-essential
                              sudo apt-get install python-dev
                              sudo apt-get install python-setuptools
                              sudo easy_install pip
                              sudo pip install pyowm
                              sudo pip install configobj
                              sudo pip install pymodbus
                            2. Copy the Python code to the home of the pi user - run sudo cp /boot/pvout_upload.py /home/pi/pvout_upload.py.
                            3. Provided that you have the Raspberry Pi connected to the inverter(s) and the sun is shining :-), you can test if everything is setup okay by running the command: python pvout_upload.py - you should then be able to see data on pvoutput.org - if all parameters where set correctly in the pvoutput.txt file.
                            4. Now you only need to setup that the program is run every 10 minutes - run crontab -e and add the following line at the end - and save and close:
                            5. */10 * * * * cd /home/pi/; python pvout_upload.py;
                            6. Done - enjoy the sun :-).

                            Additional info

                            Look through the Python code - you might want to add your own improvements, change the weather info provider or send an email with the total production of the day to yourself.

                            Also I did only used the input register values from the inverter that is needed for pvoutput.org - but for the rest of the Growatt mod-bus registers you can take a look at table 4.2 in this document.

                            I did not use the "high" values in the register pairs - since I will never generate that much power, and it just makes the code harder to read. Also notice that the "work time" are 2 ticks per second.

                            For users with inverters running firmware prior to 1.8, you need to use the serial protocol described here - since I do not think the mod-bus was available.

                            The German zentralpower inverters should be a clone of the Growatt/Sungold inverters - but I do not know if they also run the same firmware.

                            29 comments:

                            1. At step 5 there is a typo:
                              sudo easy-install pip

                              should be

                              sudo easy_install pip

                              ReplyDelete
                            2. This tutorial helped me a lot and in about half an hour I was uploading to pvoutput.org.
                              Thank you very much Steve.

                              ReplyDelete
                              Replies
                              1. Thanks - credit should also go to some of the people mentioned :-)
                                I might update, or do a part two at a point in time - since I want it to do more - I do have some minor glitches in my uploads - probably due to network issues.
                                Happy you got it working.

                                Delete
                            3. Thanks for sharing. I could get my inverter sending data in a few hours. And this is the first time I am seeing/using raspberrypi and linux. Have you experienced something odd with the weather update? I was getting 37C, but it was more like 25-27C yesterday. I checked the coordinates and those are correct.

                              ReplyDelete
                            4. Glad to hear. No my weather data seems to be fine - just doubled checked again - comparing the live data I had sent to pvoutput.org and the ones on openweathermap.org. I did initially switch the longitude and latitude parameters incorrectly because that made more sense to me :-). And I guess you didn't modify my scripts to take the inverters temp. data - because they reflect how hot the inverter is, and is not "weather" data :-D. My first guess would be coordinates, but you checked those. You could run the part of the code that receives the data from OWM - and you also did setup the °C/°F correct? - otherwise your figures would be higher. You could write to the OWM support/forum i guess after you have check what their web service returns.

                              ReplyDelete
                            5. Sorry, it was the coordinates after all. I specified them as follows: Latitude=47.443896, Longitude=19.02416 which are the correct coordinates for Budapest Hungary. But the weather I am getting is more for Lat=19 and Long=47 which is in the middle of the Saudi desert. That explains why I was getting 43C when it is only 21C. So I swapped the value around. Therefore the code is feeding the coordinates into the OWM service in the wrong order. I hope I got it right and not made a bigger loser of myself.

                              ReplyDelete
                            6. Super, as I said I did that myself - because I didn't find the order of the parameter logical :-D

                              ReplyDelete
                            7. Sorry, but it is me again. My setup works just great for the last 6 month, I am really happy for that.
                              I was trying to do the same for my father-in-law. Same inverter, same code. The only thing I changed is the SID. I program is set up to run every 10 minutes. But I am only getting values about every 30 min or once every hours. I don't think it is the network, because I can always SSH in and the response is very quick. Is it possible that Growatt does not send the values all the time? I was trying to add more print to the python code, but I messed it up all the time and got errors. What do you suggest the best way of debugging this? I even had to comment out the OWM section, because it did not work. And again the code is the same.
                              Unfortunately I have no remote connection to the PI, so I can only fix it when I pay a visit for Xmas. I would like to get prepared with options before I get there.
                              Thanks.

                              ReplyDelete
                            8. I would put my money on the usb-rs232 cable to be the culprit - but get your father-in-law to sent pi, power adapter and cable back to you, then you have a better chance to debug, and not rulning the xmas holidays :)

                              ReplyDelete
                            9. This mean I might have been very lucky that the USB-RS232 cable I had at home laying around has worked for me. Is there a particular chipset I should go for? Is this an issue with the Linux driver or Growatt itself?

                              ReplyDelete
                              Replies
                              1. As mentioned in the blog post, PL2303 based seems to work. But get hold of your farther-in-laws HW, so that you can test the various parts against each other :)
                                I am not sure if it isn't also the pi USB.

                                Delete
                            10. Hi, thank you very much for this tutorial. I am having trouble getting any values coming up in pvoutput.org...... http://www.pvoutput.org/intraday.jsp?id=9161&sid=7461&dt=20141212&gs=0&m=1
                              Its connected to the site as its updating the time...but no data. I set the address of the router to 1 via the windows shinenet software...is there anywhere in the pvoutput.txt where it is? I have a FTDI USB to RS232 cable which comes up as......
                              Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.
                              Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
                              Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
                              Bus 001 Device 004: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC

                              Any help would be great,
                              Simon.

                              ReplyDelete
                            11. Tried a new lead with the PL2303 chipset....still no good!
                              Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.
                              Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
                              Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
                              Bus 001 Device 004: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port
                              pi@raspberrypi ~ $

                              ReplyDelete
                            12. Thanks for the guide.
                              After a few months my pi got hacked (I stupidly forgot to change the password after I set up remote access) so I formatted it and started again. Using the same pi, same code, same instructions and it now won't connect to the inverter. The code runs fine until it try's to read the register and it says it's empty. I remember when I set it up the first time I had to do something extra to get it to work properly but I can't remember what. Any suggestions on how to get it to work

                              ReplyDelete
                              Replies
                              1. Hi I guess it is you AlexP - sorry for now replying earlier - otherwise see reply to AlexP below.

                                Delete
                            13. I'm pulling my hair out :-( help
                              I've started again twice and followed the instructions to the letter. I've even tried with two serial-usb cables (HL340 and PL2303) but every time I read rr.registers it tells me they are empty. Its like modbus isn't installed but it is. Does anyone have any ideas?

                              ReplyDelete
                              Replies
                              1. I would actually check if the communication params were ok on the Inverter and are setup on the Pi with something like MiniCom - but have an extra look at the bottom page of the Sisand link: http://www.sisand.dk/?tag=raspberry-pi-python-growatt-pv-inverter

                                Delete
                            14. Hi Steffen, thanks for getting back to me.
                              I've even installed minimalmodbus to see if that made a difference but unfortunately not.
                              The inverter is set to a comm port and in minicom it looks the same as Sisands.
                              The only thing I've found is that if I run
                              dmesg | grep tty*s*
                              I get a lot of error messages, especially get/set_line_request, set_control_lines and usb_serial_generic_write_bulk_callback - urb stopped: -32

                              [ 0.001665] console [tty1] enabled
                              [ 0.109827] Setting up static identity map for 0x57d9f8 - 0x57da30
                              [ 0.137721] 20201000.uart: ttyAMA0 at MMIO 0x20201000 (irq = 83, base_baud = 0) is a PL011 rev2
                              [ 0.586266] Setting default values for core params
                              [ 0.587841] Finished setting default values for core params
                              [ 138.838285] usb 1-1.2.1: pl2303 converter now attached to ttyUSB0
                              [ 353.407625] pl2303 ttyUSB0: pl2303_get_line_request - failed: -32
                              [ 353.411545] pl2303 ttyUSB0: pl2303_get_line_request - failed: -32
                              [ 353.588774] pl2303 ttyUSB0: usb_serial_generic_read_bulk_callback - urb stopped: -32
                              [ 353.598914] pl2303 ttyUSB0: usb_serial_generic_read_bulk_callback - urb stopped: -32
                              [ 353.710301] pl2303 ttyUSB0: error sending break = -32
                              [ 379.788836] pl2303 ttyUSB0: pl2303_get_line_request - failed: -32
                              [ 379.789085] pl2303 ttyUSB0: pl2303_set_line_request - failed: -32

                              ReplyDelete

                            15. Hi, good guide. But I have a problem: when I run the command "python pvout_upload.py", I get the following error:
                              Traceback (most recent call last):
                              File "pvout_upload.py", line 12, in
                              SYSTEMID = config ['SYSTEMID']
                              File "/usr/local/lib/python2.7/dist-packages/configobj.py", line 554, in __getitem__
                              val = dict .__ getitem __ (self, key)
                              KeyError: 'SYSTEMID'

                              How can I fix? thanks

                              ReplyDelete
                              Replies
                              1. this error I found the solution, but now I have another:
                                Traceback (most recent call last):
                                File "pvout_upload.py", line 43, in
                                if owm.API_online:
                                AttributeError: 'OWM25' object has no attribute 'API_online'

                                Delete
                              2. Unknown just pointed out that there is a change in the OWM module - so line 43 - needs to be something like "if owm.is_API_online()"

                                Delete
                            16. This comment has been removed by the author.

                              ReplyDelete
                            17. Hello

                              When i test it, i get this:

                              pi@raspberrypi ~ $ python pvout_upload.py
                              Traceback (most recent call last):
                              File "pvout_upload.py", line 15, in
                              OWMLon = float(config['Longitude'])
                              ValueError: could not convert string to float: <4.83603>

                              What am i doing wrong?

                              ReplyDelete
                              Replies
                              1. As Unknown spotted - the notation typically means that you should replace the value with your value incl. the <>. Sorry for the late reply.

                                Delete
                            18. turbootje: It looks like you've left the <> around the lat and long values in your config file

                              ReplyDelete
                              Replies
                              1. Thanks for spotting - now that I haven't been good at replying for a while.

                                Delete
                            19. Looks like there is a new version of the OWM module out there which needs owm.is_API_online() at line 43. Other than that it just worked.

                              ReplyDelete
                              Replies
                              1. Thanks for the info - I need to revise my own setup - since it is a bit flaky - whenever my Pi loose power or network - and also time to add stuff.

                                Delete