A better alternative to understand an opaque java application might be to use a profiler. JProfiler is my favorite, other choices are JProbe or the embedded profiler of NetBeans. When running your application against a profiler, you don't hit timeouts and you quickly get a very useful and in-depth view of the application: cpu an heap usage statistics with detailed call tree graphs immediately point you to the relevant 'hot' parts of the application.
Conclusion: profilers can be a welcome addition to debuggers and other analysis tools when inheriting a complex codebase. This is the first time that I look at profilers this way, and I must admit, having done many JProfiling sessions that you quickly get an indepth view in the most obscure (generated) code with these tools.
- testing extremes
- using anonimized production-data
- applying complex validations on inputdata (e.g. realistic names, gender, birth date, addresses,...)
- multiple input/output formats supported: xml, csv, database
- highly customizable
- free (GPL)
This presentation showed an overview of the Glassfish V3 'prelude' features:
- core: Apache Felix (OSGi)
- hot deploy
- embedded API starts <1>
- WebSynergy: Portal/portlet technology
- Grizzly: Async IO (with 'NIO') as an efficient http frontend to glassfish (+AJAX and Comet support)
- Metro 1.4 (WebServices)
- Jersey: JAX-RS REST support
- OpenMQ: JMS provider
- Open ESB
Tuning Java applications for performance covers the following areas:
- technology: specific JVM details, garbage collector ...
- tooling: visualvm, vmstart, verbose gc ...
- methodology: identify goals, measure production data, benchmark ...
Java applications run on a dynamic environment. This is a huge advantage as the HotSpot compiler can adapt and optimize the runtime according to production-time measurements, usage scenarios and available hardware. Static languages like C and C++ don’t have that possibility and can only be optimized at compile time. With this dynamicity in mind, one investigating Java Performance must investigate the complete ‘box’, which consists of these layers:
- Actors: batch jobs, users, peak usage …
- Application: locks (locks are not always bad: they can preserve CPU), static code, libraries …
- JVM / OS: Sun JVM vs IBM, thread implementation …
- Hardware: memory, cpu, optimized instruction sets …
--> look for the dominant element of performance issues (=break down).
Some tools to investigate performance:
- OS: vmstat, mpstat, corestat …
- JVM: -verbosegc (even on production), gchisto, hpjmeter
- specific IBM “monitoring and diagnostic” tools (beta)
- java –xtrace: … for simple ‘entering/leaving method’ style logging, built in the jvm
- profilers (JProfiler is my favorite)
- loadtesting tools: JMeter and Grinder (read on)
Although standard Garbage Collectors run well out of the box, they often require optimizations and tuning once ‘real life’ applications run. When using “Generational collectors”, tuning is typically done with visualvm by modifying the eden, generational and old heap spaces at the JVM command line.
Load-testing Java applications:
- <> stress test: load tests an application using production-like load, while a stress test investigates the breaking point of an application.
- have a production-like setup
- gather relevant expected load figures
- avoid caches
- avoid ‘noise’ like irrelevant network overhead
- use mocks for external systems
- tools: e.g. JMeter
Unfortunately, the native rtl8187 module proves very unstable with the Rtl8187B: although the connection appears fine in the menu bar, the wifi interrupts its connection or is suddenly very slow.
After some investigation I found a workaround: http://ubuntuforums.org/showthread.php?t=792092
gksudo gedit /etc/rc.local
- add at the end, before the
iwconfig wlan0 rate 5.5M
I just retrieved and processed the energy-production details of my photovoltaic panels using the SunnyBeam under Windows:
Google docs offers a great way to share charts and detailed data; even the atom feed is not missing.
As mentioned in my previous post, I have a "Sunny Beam" appliance that monitors the electricity production of my solar panels. The Beam maintains a wireless connection with the inverter to capture the power data. The clean LCD display offers a nice graph and the module recharges its batteries with solar cells. In addition, the Sunny Beam has a USB connection to send the data to a PC. Unfortunately, the Sunny Beam uses no standard 'USB mass storage' interface, but requires a Windows-only USB-driver. SMA.de responded to my mail stating that "The Sunny Beam can only work on windows operating systems. Windows 2000, XP (32 and 64) and Vista (32)". It is sad to see that succesful companies ignore the fact that other operating systems like MacOS and Linux are becoming increasingly popular. I'm not a Mac-fanboy, but MacOS has around 9% market share according to this site.
Once the USB driver is setup under Windows, you must use the complex Sunny Data Control application to gather the data. It is not a simple program to use, especially for small scaled solar systems. You really need to dive into the manual to get started. SMA informed me that a new, simpler application will be released in the beginning of 2009. According to the sma-america.com site, this new program will probably be called 'Sunny Beam WebConnect'.
I tried to install the Sunny Data Control under the latest version of Wine HQ. By default Ubuntu provides an outdated version of Wine, but upgrading is well documented on the Wine site. The application installs seamingly and starts fine; I even managed to register the USB driver with the following command:
wine rundll32 setupapi.dll,
InstallHinfSection DefaultInstall 128 Desktop/sbeamdriver/SBeamUSB.inf
Next, I found out that SMA provides a C library, YASDI -- 'Yet Another SMA Device Interface' -- for generic low-level access to their inverter interfaces. The Yasdi-API has everything to convince me: it's LGPL, linux is supported out-of-the box, Java JNI wrappers exist and it has a nice layered architecture where the technical drivers are decoupled from higher layers like the protocol-handling. Unfortunatly, Yasdi has no USB driver. I contacted the author of Yasdi, to check if the Sunny Beam fits in the standard Yasdi-API framework concept, so it would be easy to add the USB driver to it. Bad luck, Heiko informed me that the protocol used in SunnyBeam is a little bit different from other -- I suppose older -- devices. In order to add the SunnyBeam module to Yasdi, the protocol-layers will have to be adapted. I suppose the YASDI support and development is frozen at SMA.
The suggestion I got from Heiko is to use VMware or VirtualBox to run a Windows-system. Since my system is dual-boot (Windows is the last -- painful -- option) this doesn't add much value to the setup. Other suggestions are welkom.
The device presents itself to Ubuntu as follows:
$ lsusb -v
Bus 001 Device 002: ID 1587:002d SMA Technologie AG
bDeviceClass 0 (Defined at Interface level)
idVendor 0x1587 SMA Technologie AG
iManufacturer 1 SMA Technologie AG
iProduct 2 Sunny Beam
iSerial 3 00024383
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 255 Vendor Specific Subclass
bInterfaceProtocol 255 Vendor Specific Protocol
iInterface 2 Sunny Beam
bEndpointAddress 0x81 EP 1 IN
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bEndpointAddress 0x02 EP 2 OUT
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
Device Status: 0x0000
$ dmesg | tail
[ 3100.544038] usb 1-1: new full speed USB device using uhci_hcd and address 2
[ 3100.781392] usb 1-1: configuration #1 chosen from 1 choice
- 17 panels from EcoStream ("Atrium");
single panel specs:
- 16,2 kg
- polycrystalline cells
- frame: aluminium
- total: 3,145kWp
- minimum 2673 kWh
- The inverter is a SMA "SunnyBoy" 2500, with a wireless 'piggyback' connection to a standalone 'SunnyBeam' unit.
As an extra, I bought the wireless 'SunnyBeam' module. Once the wireless piggyback is installed on the inverter by a professional, the SunnyBeam offers a clean way to monitor the electricity production of the solar panels. The SunnyBeam itself uses solar energy to charge its batteries. The device maintains a detailed graph of the production for one day and the detailed total production for the last 31 days.
The connection from the inverter to the SunnyBeam is over wifi, although a proprietary protocol is used. There is also a USB connector, but unfortunately, only Windows drivers are available. The "Sunny Data Control" software is needed to retrieve the data over USB, but the software's interface is rather overloaded and non-intuitive, especially for small-scaled needs. You really need to delve into the manual to get started.
update 13/6/2009: check out my new sunnybeamtool command line application.
Java FX has a huge potential as being a real Open Source (GPL) RIA platform compared to its more or less closed source competitors. I know, Flash has some parts of it made 'open', but Flash basically remains a commercial platform, requiring commercial Adobe developer tools for the 'real' work (including the now classic video à la youtube). It remains to see if Sun will be able to maintain the full GPL license for Java FX, since video and sound are usually licensed from third party providers using patented technologies (e.g. On2).
Although RIA apps have a tremendous 'whaw' factor (e.g. parleys.com) , I'm quickly bored by all the transitions, fading in/out and other visual magic. It's also very hard developing good RIA applications that are both functionally relevant and appealing to the eye. Developers must combine design skills, good taste and programming talents. Otherwise you end up with disastrous websites like this amusement park. It takes about 15-20s of looking at a shiny progress bar, hearing silly music and looking at stupid animations to finally find the opening hours. Definitely in the top-10 of 'worst-RIA-sites-ever'.
RIA apps fail to embrace the strengths of the classic web:
- bookmarkability & addressability: In typical RIAs, only the main page has an address; forget to send a to-the-point URL to a friend... Bookmarking individual parts of a RIA app requires the oh-so-stupid 'link this' button. This is opposite to the current "REST" trend where everything is a URL, a resource.
- user driven action: RIA apps tend to limit the users' behavior, forcing them to follow a fixed pattern, where classic webshops let the user decide where to click. Have a look at the on-line promotion folder of a classic brick and mortar shop. It takes about 2-3 seconds to change to the next page; at least one or 2 minutes of clicking if you're searching a specific item.
- speed: it looks like every RIA app needs its shiny progressbar, tricking the user to wait much longer than he'd be willing to wait for a classic web-page
- search engine friendliness: by default RIA apps can not be indexed by google, unless tricking google by offering a 'ghost' site to the search robots. This is how parleys.com solves their search ranking problem. Someone said 'hacking'?
- OS-agnostic: only Java FX promises to target Windows, MacOS and Linux. Of course, Flash support under Linux has made huge progress, with e.g. the Flash 64bit version released under Linux first. But all RIA platforms try to address the 'fat client' offline problem: running apps while disconnected from internet, even outside the browser (e.g. Adobe AIR or Java Update 10). Unfortunatly, those 'fat client' standalone versions of RIA apps offer a consequent look on all OSes. This might be a plus, but the current trend is to adapt your UI to the OS. Firefox 3 for example has a different look and feel under Gnome than under Vista.
- hackability: I'm addicted to noscript, adblock and firebug, but those tools are useless in a RIA context. Even a simple 'save as', as Firefox provides, is not straightforward. Perhaps the development of gnash might bring some new hacker-tools to the scene.
We'll see what the RIA-future brings: all Flash (flex/air), Java FX, or a "back to the web" reaction?