Using the TonidoPlug² as a Datalogger for Photovoltaic Panels

I managed to setup my TonidoPlug² (TP2) as a data-logger that automatically uploads live statistics of my solar panels to pvoutput.org. This post details the steps I followed to run sma-bluetooth on the TP2 .

After ssh'ing to your TP2, prepare a build environment on the plug pc:

• follow the steps in: http://www.tonido.com/communitywiki/doku.php?id=advanceduses:setupbuildenvironment
• step 4 & 5 are slightly outdated:
  • (4a) I had to install the deb-multimedia-keyring package first:

    apt-get install deb-multimedia-keyring 
    

  • (4b) the url for wget is diffent:

    wget http://www.debian-multimedia.org/pool/main/d/debian-multimedia-keyring/debian-multimedia-keyring_2010.12.26-3_all.deb 
    

  • (5) dpkg -i debian-multimedia-keyring_2010.12.26-3_all.deb

After that you can follow the steps I detailed in my previous post: http://michael-peeters.blogspot.com/2012/04/sb-inverter-bluetooth-connection-on.html. Summarized:

• compile sma-bluetooth:

  wget http://sma-bluetooth.googlecode.com/files/sma-bluetooth-0.17b.tar.gz 
  tar -xzf sma-bluetooth-0.17b.tar.gz 
  cd sma-bluetooth-0.17 
  sudo apt-get install mysql-server mysql-client  libmysqlclient-dev 
    # (don't forget to enter a non-blank password for mysql!) 
   
   gcc -c smatool.c 
   gcc smatool.o -L/usr/lib -lmysqlclient -lbluetooth -lcurl -lm -o smatool 
  

• edit the configuration. This can be tricky! Check the details first.

  # nano ./smatool.conf 
  

• install the mysql tables:

  ./smatool --INSTALL 
  

• run / debug errors:

  ./smatool -d -v 
  

Once smatool runs OK from the command-line, you can consider scheduling the run in cron.

• create a smatool.cron file in your home-directory. Contents:
  

  # /etc/cron.d/smatool: crontab entries for smatool (monitoring of Sunnyboy Grid Interactive Inverter) 
   
  SHELL=/bin/sh 
  PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/smatool/bin 
   
  # The following line runs the smatool every 15 minutes between 0500 and 2300 and logs all output to the /var/log/smatool.log file. 
  */15 5-23 * * *   root cd /root/sma-bluetooth-0.17; ./smatool 2>&1 | logger -t smatool -p local5.info 
  

• activate the new cron entry:

  cp ./smatool.cron /etc/cron.d/smatool 
  chmod 744 /etc/cron.d/smatool 
  

• checkout /var/log/syslog for errors
• each run is logged in /var/log/syslog

The result:



Possible improvements:

• sma-bluetooth requires mysql which is a relatively heavy database for a plugpc. Sqlite would be a better choice. I experimented with the excellent fork http://rcphills-sma-bluetooth-dbabstraction.googlecode.com/ that supports sqlite, but the connection to pvoutput.org didn't work anymore.
• the current setup does too much I/O preventing the SATA HD to sleep. Unfortunately booting from an external USB flash disk is not easy on TP², so I'll have to investigate other alternatives.

Bluetooth Connection of the Sunny Boy Inverter on Ubuntu

My photovoltaic panels are connected to the power grid through a Sunny Boy SB5000TL inverter. Although, this inverter comes out-of-the-box with bluetooth connectivity, this feature is only supported on Windows. Luckily, there exists a free tool that reads out the generation data on Linux: sma-bluetooth. This involves a few manual commands detailed in http://code.google.com/p/sma-bluetooth/wiki/ubuntu_compile_guide. The guide didn't work a 100% for me, so the next section details the steps I followed.

The code can be downloaded from http://code.google.com/p/sma-bluetooth.

In the terminal ( Ctl + Alt + T):

• sudo apt-get install mysql-server mysql-client mysql-query-browser libmysqlclient-dev (the libmysqlclient-dev package is needed for compilation)
• sudo apt-get install bluez libbluetooth-dev curl libcurl3 libcurl4-nss-dev
• create a new Makefile in the sma-bluetooth-0.17 directory:

  gedit Makefile 
  

  Contents (make sure you use a tab, not spaces, in front of the gcc commands):

  smatool: smatool.o 
   
          gcc smatool.o -L/usr/lib -lmysqlclient -lbluetooth -lcurl -lm -o smatool  
   
  smatool.o: smatool.c 
   
          gcc -c smatool.c 
   
  

• make

Now prepare the Bluetooth connection:

• my bluetooth USB dongle is automatically up & read on ubuntu 11.10:

  $ lsusb | grep -i bluetooth 
  Bus 002 Device 002: ID 0a5c:4500 Broadcom Corp. BCM2046B1 USB 2.0 Hub (part of BCM2046 Bluetooth) 
  

• hcitool scan
• workaround if hcitool scan gives no result: (seehttps://help.ubuntu.com/community/BluetoothSetup#Setup_Devices for details)

  sudo hciconfig hci0 inqmode 0 
  

• cp smatool.conf.new smatool.conf
• gedit smatool.conf
  • modify the lines as explained in the comments
  • you can optionally upload your data to the excellent pvoutput.org service - see http://pvoutput.org/account.jsp for details:
    • get the API key
    • configure your unique SID
  • use the bluetooth password for 'normal' usage (by default 0000 in some devices). I had trouble with the different password reserved for maintenance.

If you use mysql:

• install the database / create the tables (this requires a user with admin rights):

  ./smatool --INSTALL 
  

• an empty password is not supported

Running:

• ./smatool
• debug & verbose can help identifying errors:

  ./smatool -d -v 
  

• ./smatool -? lists out all options available

My pv-output: http://pvoutput.org/list.jsp?userid=9300

Next steps:

• schedule the smatool in cron
• replace mysql with sqlite for easier is
• deploy the config on a plug computer

Sunny Beam under Ubuntu: SunnyBeamTool v0.6.0

http://sourceforge.net/projects/sunnybeamtool/

The sbtool Perl script of Stefan Arts provides a nice basic setup for accessing the Sunny Beam under Linux. But the Perl solution has some drawbacks: it is quite complex to set up and the codebase, with low-level hex-manipulations, is rather complex to extend, at least for an occasional Perl-developer like me...

With this in mind I translated the sbtool into C language and put it on sourceforge:

http://sourceforge.net/projects/sunnybeamtool/

The basic functionalities are included, but additional feedback, testing and extensions are welcome. Of course use at your own risk ! :-).

$ sudo ./sunnybeamtool
Dev #6: 1587 - 002D -SMA Technologie AG -
Serial Number: 00024383
pac: 0.00 W
e-today: 12.37 kWh
e-total: 1551.83 kWh

$ sudo ./sunnybeamtool -l
Dev #6: 1587 - 002D -SMA Technologie AG -
Serial Number: 00024383
pac: 0.00 W
e-today: 12.37 kWh
e-total: 1551.83 kWh

today:
...
2009-06-11 13:00:00: 756 W
2009-06-11 13:10:00: 888 W
2009-06-11 13:20:00: 569 W
2009-06-11 13:30:00: 205 W
...

last month:
...
2009-06-07: 11.790 kWh
2009-06-08: 10.060 kWh
2009-06-09: 8.480 kWh
2009-06-10: 12.370 kWh

This commandline is much easier and cleaner than running the full blown complex Sunny Data Control application under windows.

If your Sunny Beam doesn't respond to the USB-communication, try this: unplug the USB cable, switch a few times between daily graph and monthly graph, go to the menu and select 'exit', plug the USB cable. This should 'unfreeze' the USB buffer of the Sunny Beam. I never had to remove the batteries from my Sunny Beam device during all my experiments.

If you're still having trouble, send the output of 'sudo sunnybeamtool -l -v' in your bug-reports on sourceforge.

Other background info: I kept the libusb approach, as this allows the application to run in user mode, outside of the kernel. The development is done wih Netbeans 6.5 and Subversion as development platform.

Using the Sunny Beam under Linux: first steps!

update 13/6/2009: check out my new sunnybeamtool command line application.

In an earlier post, I discussed the missing Linux drivers for the Sunny Beam device. In the 'comments' section of that post, Stefan Arts dug deeper into the Sunny Beam protocol by 'snooping' the USB-communication with SnoopyPro. His findings were put in a very basic perl script on sourceforge. In order to run it I had to add some packages to my ubuntu 9.04 install:

sudo perl -MCPAN -e 'install +YAML'
sudo perl -MCPAN -e 'install Inline::MakeMaker'
sudo apt-get install libusb-dev
sudo perl -MCPAN -e 'install Device::USB'
sudo perl -MCPAN -e 'install Digest::CRC'

running:

sudo ./sbtool

Unfortunately, I didn't get any data back.

Update 1/6/2009:
The messages sent to the Sunny Beam have a device-dependent 'destination' field. Stefan suggested me to replace one line in the sbtool script:

my $CMD_GET_DATA="\x7e\xff\x03\x40\x41\x00\x00\xd4\xf5\x10\x00\x0b\x0f\x09\x00\x69\xda\x7e";

And, I get a successful response now:

$ sudo ./sbtool
Sunny Beam, SMA Technologie AG
Serial number: 00024383

Pac: 270 Watt
E-Today: 16.38 kWh
E-Total: 1427.98 kWh

3/6/2009:
Stefan now added CRC support and 'destination parsing' to his '0.0.4' version, so his sbtool script should work on any sunny beam device.

Cool, isn't it? Next steps:add the 'get daily details' and 'get last 31 days' calls to the script.Update

On a parallel track, I also noted that Alexander Morozov provided some USB-support to wine. This could potentially allow a Windows driver to be emulated in Wine. The fix isn't merged into the git source repositories and I couldn't evaluate it with the Sunny Beam USB driver yet.

Great to see this area moving forward :-).

background info:

• Details of the SMA protocol: http://256.com/solar/scripts/smadat-11-ze2203.pdf
• recap of stefans' findings:
• data sent: 7e ff 03 40 41 00 00 ff 00 10 00 0b 0f 09 00 07 ac 7e
  7e = start flag
  ff = address byte (broadcast)
  03 = a bit coded HDLC control field, 0x03 stands for unnumbered data blocks
  40 41 = protocol used (SMA data telegram)
  00 00 ff 00 10 00 0b 0f 09 00 = data telegram content:
  00 00 = destination address
  00 ff = source address
  10 = control 'STRINGFILTER'
  00 = packet counter
  0b = command 'CMD_GET_DATA'
  0f 09 00 = content:
  09 0f = channel type
  00 = channel Idx
  ac 0f = frame check sequence, 16 bit check sum
  7e = stop flag
  
• data received: 7e ff 03 40 41 ff 00 00 00 50 00 0b 0f 09 00 01 00 f1 cb 09 4a 01 00 00 00 00 00 70 42 29 5c d7 40 ae 47 65 41 be d9 7e
  7e = start flag
  ff = address byte (broadcast)
  03 = a bit coded HDLC control field, 0x03 stands for unnumbered data blocks
  40 41 = protocol used (SMA data telegram)
  ff 00 00 00 50 00 0b 0f 09 00 01 00 f1 cb 09 4a 01 00 00 00 00 00 70 42 29 5c d7 40 ae 47 65 41 = data telegram content
  00 ff = destination address
  00 00 = source address
  50 = control 'ANSWER'
  00 = packet counter
  0b = command 'CMD_GET_DATA'
  0f 09 00 01 00 f1 cb 09 4a 01 00 00 00 00 00 70 42 29 5c d7 40 ae 47 65 41 = content:
  09 0f = channel type
  00 = channel Idx
  00 01 = number of data records: 1
  f1 cb 09 4a = time/date in unix format
  01 00 00 00 = time basis (1 sec)
  00 00 70 42 = Pac (floating point, 60 WATT)
  29 5c d7 40 = E-today (floating point, 6.7 kWh)
  ae 47 65 41 = E-Total (floating point 14.3 Kwh)
  be d9 = frame check sequence, 16 bit check sum
  7e = stop flag
  
  
• it sends the data string:
  7e ff 03 40 41 00 00 ff 00 10 00 0b 0f 09 00 07 ac 7e 01 00
  
• a result was:
  01 60 7e ff 03 40 41 ff 00 00 00 50 00 0b 0f 09 00 01
  01 60 00 64 33 0b 4a 01 00 00 00 00 00 00 00 ec 51 00
  01 60 41 33 33 b2 41 68 5e 7e
  
• where
  
• 64 33 0b 4a = 4a0b3364 HEX = 1242248036 DEC = Wed, 13 May 2009 20:53:56 GMT
• 00 00 00 00 = 0
  ec 51 00 41 = 8.02
  33 33 b2 41 = 22.4
  
  

Monitoring the Energy-production of my Solar Panels

update 13/6/2009: check out my new sunnybeamtool command line application.

I just retrieved and processed the energy-production details of my photovoltaic panels using the SunnyBeam under Windows:


Google docs offers a great way to share charts and detailed data; even the atom feed is not missing.

Using the Sunny Beam under Linux: Not OK - yet?

update 13/6/2009: check out my new sunnybeamtool command line application.

As mentioned in my previous post, I have a "Sunny Beam" appliance that monitors the electricity production of my solar panels. The Beam maintains a wireless connection with the inverter to capture the power data. The clean LCD display offers a nice graph and the module recharges its batteries with solar cells. In addition, the Sunny Beam has a USB connection to send the data to a PC. Unfortunately, the Sunny Beam uses no standard 'USB mass storage' interface, but requires a Windows-only USB-driver. SMA.de responded to my mail stating that "The Sunny Beam can only work on windows operating systems. Windows 2000, XP (32 and 64) and Vista (32)". It is sad to see that succesful companies ignore the fact that other operating systems like MacOS and Linux are becoming increasingly popular. I'm not a Mac-fanboy, but MacOS has around 9% market share according to this site.

Once the USB driver is setup under Windows, you must use the complex Sunny Data Control application to gather the data. It is not a simple program to use, especially for small scaled solar systems. You really need to dive into the manual to get started. SMA informed me that a new, simpler application will be released in the beginning of 2009. According to the sma-america.com site, this new program will probably be called 'Sunny Beam WebConnect'.

I tried to install the Sunny Data Control under the latest version of Wine HQ. By default Ubuntu provides an outdated version of Wine, but upgrading is well documented on the Wine site. The application installs seamingly and starts fine; I even managed to register the USB driver with the following command:

wine rundll32 setupapi.dll,InstallHinfSection DefaultInstall 128 Desktop/sbeamdriver/SBeamUSB.inf

Unfortunatly, I found out that USB kernel drivers are not supported in Wine. Dead end street...

Next, I found out that SMA provides a C library, YASDI -- 'Yet Another SMA Device Interface' -- for generic low-level access to their inverter interfaces. The Yasdi-API has everything to convince me: it's LGPL, linux is supported out-of-the box, Java JNI wrappers exist and it has a nice layered architecture where the technical drivers are decoupled from higher layers like the protocol-handling. Unfortunatly, Yasdi has no USB driver. I contacted the author of Yasdi, to check if the Sunny Beam fits in the standard Yasdi-API framework concept, so it would be easy to add the USB driver to it. Bad luck, Heiko informed me that the protocol used in SunnyBeam is a little bit different from other -- I suppose older -- devices. In order to add the SunnyBeam module to Yasdi, the protocol-layers will have to be adapted. I suppose the YASDI support and development is frozen at SMA.

The suggestion I got from Heiko is to use VMware or VirtualBox to run a Windows-system. Since my system is dual-boot (Windows is the last -- painful -- option) this doesn't add much value to the setup. Other suggestions are welkom.

The device presents itself to Ubuntu as follows:

$ lsusb -v
Bus 001 Device 002: ID 1587:002d SMA Technologie AG
Device Descriptor:
bLength                18
bDescriptorType         1
bcdUSB               1.10
bDeviceClass            0 (Defined at Interface level)
bDeviceSubClass         0
bDeviceProtocol         0
bMaxPacketSize0         8
idVendor           0x1587 SMA Technologie AG
idProduct          0x002d
bcdDevice            2.00
iManufacturer           1 SMA Technologie AG
iProduct                2 Sunny Beam
iSerial                 3 00024383
bNumConfigurations      1
Configuration Descriptor:
bLength                 9
bDescriptorType         2
wTotalLength           32
bNumInterfaces          1
bConfigurationValue     1
iConfiguration          0
bmAttributes         0xa0
(Bus Powered)
Remote Wakeup
MaxPower              200mA
Interface Descriptor:
bLength                 9
bDescriptorType         4
bInterfaceNumber        0
bAlternateSetting       0
bNumEndpoints           2
bInterfaceClass       255 Vendor Specific Class
bInterfaceSubClass    255 Vendor Specific Subclass
bInterfaceProtocol    255 Vendor Specific Protocol
iInterface              2 Sunny Beam
Endpoint Descriptor:
bLength                 7
bDescriptorType         5
bEndpointAddress     0x81  EP 1 IN
bmAttributes            2
  Transfer Type            Bulk
  Synch Type               None
  Usage Type               Data
wMaxPacketSize     0x0040  1x 64 bytes
bInterval               0
Endpoint Descriptor:
bLength                 7
bDescriptorType         5
bEndpointAddress     0x02  EP 2 OUT
bmAttributes            2
  Transfer Type            Bulk
  Synch Type               None
  Usage Type               Data
wMaxPacketSize     0x0040  1x 64 bytes
bInterval               0
Device Status:     0x0000
(Bus Powered)
$ dmesg | tail
[ 3100.544038] usb 1-1: new full speed USB device using uhci_hcd and address 2
[ 3100.781392] usb 1-1: configuration #1 chosen from 1 choice

Photovoltaic Panels on my Roof

At home I have a reef aquarium that consumes a lot of electricity. In order to cut my electric bill and thanks to local subsidies and regulations, I decided to install photo-voltaic panels on the southern part of my roof. The planning of the system was executed by Planet Eco.

Technical details:

• 17 panels from EcoStream ("Atrium");
  single panel specs:
  
• 185Wpeak
• 1628x840x42mm
• 16,2 kg
• polycrystalline cells
• frame: aluminium
• total: 3,145kWp
• minimum 2673 kWh
• The inverter is a SMA "SunnyBoy" 2500, with a wireless 'piggyback' connection to a standalone 'SunnyBeam' unit.
  

the SunnyBoy 2500

the official 'green' current meter for getting subsidies

As an extra, I bought the wireless 'SunnyBeam' module. Once the wireless piggyback is installed on the inverter by a professional, the SunnyBeam offers a clean way to monitor the electricity production of the solar panels. The SunnyBeam itself uses solar energy to charge its batteries. The device maintains a detailed graph of the production for one day and the detailed total production for the last 31 days.

The connection from the inverter to the SunnyBeam is over wifi, although a proprietary protocol is used. There is also a USB connector, but unfortunately, only Windows drivers are available. The "Sunny Data Control" software is needed to retrieve the data over USB, but the software's interface is rather overloaded and non-intuitive, especially for small-scaled needs. You really need to delve into the manual to get started.

update 13/6/2009: check out my new sunnybeamtool command line application.