Understanding existing 'legacy' java applications is not always simple. Especially if the original developers left the company and you have to fix a bug or apply some enhancements. Dependency injection frameworks (Spring) and other very dynamic factors like Hibernate make it very difficult to quickly grasp the structure of an application and understand where the 'core business' resides. Traditional tools like a debugger do help, but their view is often too fine-grained or they cause timeouts, breaking the normal program flow.
A better alternative to understand an opaque java application might be to use a profiler. JProfiler is my favorite, other choices are JProbe or the embedded profiler of NetBeans. When running your application against a profiler, you don't hit timeouts and you quickly get a very useful and in-depth view of the application: cpu an heap usage statistics with detailed call tree graphs immediately point you to the relevant 'hot' parts of the application.
Conclusion: profilers can be a welcome addition to debuggers and other analysis tools when inheriting a complex codebase. This is the first time that I look at profilers this way, and I must admit, having done many JProfiling sessions that you quickly get an indepth view in the most obscure (generated) code with these tools.
References:
2008/12/31
Devoxx: Creating performance test data with the benerator (Volker Bergmann)
The Benerator tool focuses on a single problem: generating testdata. Although a simple problem to start, getting good testdata is tough:
Resources:
- testing extremes
- using anonimized production-data
- applying complex validations on inputdata (e.g. realistic names, gender, birth date, addresses,...)
- multiple input/output formats supported: xml, csv, database
- highly customizable
- free (GPL)
Resources:
Devoxx: GlassFish in Action: forget the app server for a minute!
Speakers: Linda Schneider, Jean-Francois Arcand, Paul Sandoz, Alexis Moussine-Pouchkine and Bogdan Stefanescu
This presentation showed an overview of the Glassfish V3 'prelude' features:
This presentation showed an overview of the Glassfish V3 'prelude' features:
- core: Apache Felix (OSGi)
- hot deploy
- embedded API starts <1>
- WebSynergy: Portal/portlet technology
- Grizzly: Async IO (with 'NIO') as an efficient http frontend to glassfish (+AJAX and Comet support)
- Metro 1.4 (WebServices)
- Jersey: JAX-RS REST support
- OpenMQ: JMS provider
- Open ESB
Devoxx: Java Performance (Kirk Pepperdine and Holly Cummins)
In this University session of Devoxx, Kirk Pepperdine and Holly Cummins detailed java performance issues and optimizations. Kirk concentrated on the Sun JVM while Holly detailed the IBM JVM.
Tuning Java applications for performance covers the following areas:
Tuning Java applications for performance covers the following areas:
- technology: specific JVM details, garbage collector ...
- tooling: visualvm, vmstart, verbose gc ...
- methodology: identify goals, measure production data, benchmark ...
Java applications run on a dynamic environment. This is a huge advantage as the HotSpot compiler can adapt and optimize the runtime according to production-time measurements, usage scenarios and available hardware. Static languages like C and C++ don’t have that possibility and can only be optimized at compile time. With this dynamicity in mind, one investigating Java Performance must investigate the complete ‘box’, which consists of these layers:
- Actors: batch jobs, users, peak usage …
- Application: locks (locks are not always bad: they can preserve CPU), static code, libraries …
- JVM / OS: Sun JVM vs IBM, thread implementation …
- Hardware: memory, cpu, optimized instruction sets …
--> look for the dominant element of performance issues (=break down).
Some tools to investigate performance:
- OS: vmstat, mpstat, corestat …
- JVM: -verbosegc (even on production), gchisto, hpjmeter
- specific IBM “monitoring and diagnostic” tools (beta)
- java –xtrace: … for simple ‘entering/leaving method’ style logging, built in the jvm
- profilers (JProfiler is my favorite)
- visualvm
- loadtesting tools: JMeter and Grinder (read on)
Although standard Garbage Collectors run well out of the box, they often require optimizations and tuning once ‘real life’ applications run. When using “Generational collectors”, tuning is typically done with visualvm by modifying the eden, generational and old heap spaces at the JVM command line.
Load-testing Java applications:
- <> stress test: load tests an application using production-like load, while a stress test investigates the breaking point of an application.
- have a production-like setup
- gather relevant expected load figures
- avoid caches
- avoid ‘noise’ like irrelevant network overhead
- use mocks for external systems
- tools: e.g. JMeter
resources:
2008/12/19
Getting the Realtek RTL8187b WiFi stable under Ubuntu
The latest Ubuntu 8.10 with its Linux 2.6.27 kernel supports the Realtek Rtl8187b wifi chipset (USB) natively. In prior versions of Ubuntu, I had to use ndiswrapper to get the integrated WiFi going on my Medion MD96290 laptop. This is not needed anymore, so I removed the ndiswrapper call from
Unfortunately, the native rtl8187 module proves very unstable with the Rtl8187B: although the connection appears fine in the menu bar, the wifi interrupts its connection or is suddenly very slow.
After some investigation I found a workaround: http://ubuntuforums.org/showthread.php?t=792092
/etc/modules.Unfortunately, the native rtl8187 module proves very unstable with the Rtl8187B: although the connection appears fine in the menu bar, the wifi interrupts its connection or is suddenly very slow.
After some investigation I found a workaround: http://ubuntuforums.org/showthread.php?t=792092
gksudo gedit /etc/rc.local- add at the end, before the
exit 0command:iwconfig wlan0 rate 5.5M
2008/12/14
Monitoring the Energy-production of my Solar Panels
update 13/6/2009: check out my new sunnybeamtool command line application.
I just retrieved and processed the energy-production details of my photovoltaic panels using the SunnyBeam under Windows:
Google docs offers a great way to share charts and detailed data; even the atom feed is not missing.
I just retrieved and processed the energy-production details of my photovoltaic panels using the SunnyBeam under Windows:
Google docs offers a great way to share charts and detailed data; even the atom feed is not missing.
2008/12/05
Using the Sunny Beam under Linux: Not OK - yet?
update 13/6/2009: check out my new sunnybeamtool command line application.As mentioned in my previous post, I have a "Sunny Beam" appliance that monitors the electricity production of my solar panels. The Beam maintains a wireless connection with the inverter to capture the power data. The clean LCD display offers a nice graph and the module recharges its batteries with solar cells. In addition, the Sunny Beam has a USB connection to send the data to a PC. Unfortunately, the Sunny Beam uses no standard 'USB mass storage' interface, but requires a Windows-only USB-driver. SMA.de responded to my mail stating that "The Sunny Beam can only work on windows operating systems. Windows 2000, XP (32 and 64) and Vista (32)". It is sad to see that succesful companies ignore the fact that other operating systems like MacOS and Linux are becoming increasingly popular. I'm not a Mac-fanboy, but MacOS has around 9% market share according to this site.
Once the USB driver is setup under Windows, you must use the complex Sunny Data Control application to gather the data. It is not a simple program to use, especially for small scaled solar systems. You really need to dive into the manual to get started. SMA informed me that a new, simpler application will be released in the beginning of 2009. According to the sma-america.com site, this new program will probably be called 'Sunny Beam WebConnect'.
I tried to install the Sunny Data Control under the latest version of Wine HQ. By default Ubuntu provides an outdated version of Wine, but upgrading is well documented on the Wine site. The application installs seamingly and starts fine; I even managed to register the USB driver with the following command:
wine rundll32 setupapi.dll,InstallHinfSection DefaultInstall 128 Desktop/sbeamdriver/SBeamUSB.inf Next, I found out that SMA provides a C library, YASDI -- 'Yet Another SMA Device Interface' -- for generic low-level access to their inverter interfaces. The Yasdi-API has everything to convince me: it's LGPL, linux is supported out-of-the box, Java JNI wrappers exist and it has a nice layered architecture where the technical drivers are decoupled from higher layers like the protocol-handling. Unfortunatly, Yasdi has no USB driver. I contacted the author of Yasdi, to check if the Sunny Beam fits in the standard Yasdi-API framework concept, so it would be easy to add the USB driver to it. Bad luck, Heiko informed me that the protocol used in SunnyBeam is a little bit different from other -- I suppose older -- devices. In order to add the SunnyBeam module to Yasdi, the protocol-layers will have to be adapted. I suppose the YASDI support and development is frozen at SMA.
The suggestion I got from Heiko is to use VMware or VirtualBox to run a Windows-system. Since my system is dual-boot (Windows is the last -- painful -- option) this doesn't add much value to the setup. Other suggestions are welkom.
The device presents itself to Ubuntu as follows:
$ lsusb -v
Bus 001 Device 002: ID 1587:002d SMA Technologie AG
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.10
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x1587 SMA Technologie AG
idProduct 0x002d
bcdDevice 2.00
iManufacturer 1 SMA Technologie AG
iProduct 2 Sunny Beam
iSerial 3 00024383
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 32
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xa0
(Bus Powered)
Remote Wakeup
MaxPower 200mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 255 Vendor Specific Subclass
bInterfaceProtocol 255 Vendor Specific Protocol
iInterface 2 Sunny Beam
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Device Status: 0x0000
(Bus Powered)
$ dmesg | tail
[ 3100.544038] usb 1-1: new full speed USB device using uhci_hcd and address 2
[ 3100.781392] usb 1-1: configuration #1 chosen from 1 choice
2008/12/04
Photovoltaic Panels on my Roof
At home 
I have a reef aquarium that consumes a lot of electricity. In order to cut my electric bill and thanks to local subsidies and regulations, I decided to install photo-voltaic panels on the southern part of my roof. The planning of the system was executed by Planet Eco.
Technical details:
the SunnyBoy 2500
the official 'green' current meter for getting subsidies
As an extra, I bought the wireless 'SunnyBeam' module. Once the wireless piggyback is installed on the inverter by a professional, the SunnyBeam offers a clean way to monitor the electricity production of the solar panels. The SunnyBeam itself uses solar energy to charge its batteries. The device maintains a detailed graph of the production for one day and the detailed total production for the last 31 days.
The connection from the inverter to the SunnyBeam is over wifi, although a proprietary protocol is used. There is also a USB connector, but unfortunately, only Windows drivers are available. The "Sunny Data Control" software is needed to retrieve the data over USB, but the software's interface is rather overloaded and non-intuitive, especially for small-scaled needs. You really need to delve into the manual to get started.
update 13/6/2009: check out my new sunnybeamtool command line application.
I have a reef aquarium that consumes a lot of electricity. In order to cut my electric bill and thanks to local subsidies and regulations, I decided to install photo-voltaic panels on the southern part of my roof. The planning of the system was executed by Planet Eco.Technical details:
- 17 panels from EcoStream ("Atrium");
single panel specs: - 185Wpeak
- 1628x840x42mm
- 16,2 kg
- polycrystalline cells
- frame: aluminium
- total: 3,145kWp
- minimum 2673 kWh
- The inverter is a SMA "SunnyBoy" 2500, with a wireless 'piggyback' connection to a standalone 'SunnyBeam' unit.
the SunnyBoy 2500
the official 'green' current meter for getting subsidiesAs an extra, I bought the wireless 'SunnyBeam' module. Once the wireless piggyback is installed on the inverter by a professional, the SunnyBeam offers a clean way to monitor the electricity production of the solar panels. The SunnyBeam itself uses solar energy to charge its batteries. The device maintains a detailed graph of the production for one day and the detailed total production for the last 31 days.
The connection from the inverter to the SunnyBeam is over wifi, although a proprietary protocol is used. There is also a USB connector, but unfortunately, only Windows drivers are available. The "Sunny Data Control" software is needed to retrieve the data over USB, but the software's interface is rather overloaded and non-intuitive, especially for small-scaled needs. You really need to delve into the manual to get started.
update 13/6/2009: check out my new sunnybeamtool command line application.
2008/12/01
The worst RIA-site ever
I've never been a big fan of the 'Rich Internet Application' movement, with it most obvious defender: Flash. There are two contenders that fight for the RIA dominance: Microsoft Silverlight and Suns' Java FX. Flash is currently the only platform that's ready for real use: Silverlight and Java FX are still in beta.
Java FX has a huge potential as being a real Open Source (GPL) RIA platform compared to its more or less closed source competitors. I know, Flash has some parts of it made 'open', but Flash basically remains a commercial platform, requiring commercial Adobe developer tools for the 'real' work (including the now classic video à la youtube). It remains to see if Sun will be able to maintain the full GPL license for Java FX, since video and sound are usually licensed from third party providers using patented technologies (e.g. On2).
Although RIA apps have a tremendous 'whaw' factor (e.g. parleys.com) , I'm quickly bored by all the transitions, fading in/out and other visual magic. It's also very hard developing good RIA applications that are both functionally relevant and appealing to the eye. Developers must combine design skills, good taste and programming talents. Otherwise you end up with disastrous websites like this amusement park. It takes about 15-20s of looking at a shiny progress bar, hearing silly music and looking at stupid animations to finally find the opening hours. Definitely in the top-10 of 'worst-RIA-sites-ever'.
RIA apps fail to embrace the strengths of the classic web:
I do think that Flash and Java FX have their value when used to enrich standard web pages. You'd be amazed to see what is possible in plain html/javascript/css. Look at the spectacular Apple ITunes store for example. In general everything that's not supported in html4 requires flash: audio/video or drawing. By putting those pieces in a classical webapp you get a tremendous combination. For example, Google Analytics or Google Finance are html pages, but the nice graphs are Flash. The integration is seamless, an end-user can see no difference between the html and the flash parts.
We'll see what the RIA-future brings: all Flash (flex/air), Java FX, or a "back to the web" reaction?
Java FX has a huge potential as being a real Open Source (GPL) RIA platform compared to its more or less closed source competitors. I know, Flash has some parts of it made 'open', but Flash basically remains a commercial platform, requiring commercial Adobe developer tools for the 'real' work (including the now classic video à la youtube). It remains to see if Sun will be able to maintain the full GPL license for Java FX, since video and sound are usually licensed from third party providers using patented technologies (e.g. On2).
Although RIA apps have a tremendous 'whaw' factor (e.g. parleys.com) , I'm quickly bored by all the transitions, fading in/out and other visual magic. It's also very hard developing good RIA applications that are both functionally relevant and appealing to the eye. Developers must combine design skills, good taste and programming talents. Otherwise you end up with disastrous websites like this amusement park. It takes about 15-20s of looking at a shiny progress bar, hearing silly music and looking at stupid animations to finally find the opening hours. Definitely in the top-10 of 'worst-RIA-sites-ever'.
RIA apps fail to embrace the strengths of the classic web:
- bookmarkability & addressability: In typical RIAs, only the main page has an address; forget to send a to-the-point URL to a friend... Bookmarking individual parts of a RIA app requires the oh-so-stupid 'link this' button. This is opposite to the current "REST" trend where everything is a URL, a resource.
- user driven action: RIA apps tend to limit the users' behavior, forcing them to follow a fixed pattern, where classic webshops let the user decide where to click. Have a look at the on-line promotion folder of a classic brick and mortar shop. It takes about 2-3 seconds to change to the next page; at least one or 2 minutes of clicking if you're searching a specific item.
- speed: it looks like every RIA app needs its shiny progressbar, tricking the user to wait much longer than he'd be willing to wait for a classic web-page
- search engine friendliness: by default RIA apps can not be indexed by google, unless tricking google by offering a 'ghost' site to the search robots. This is how parleys.com solves their search ranking problem. Someone said 'hacking'?
- OS-agnostic: only Java FX promises to target Windows, MacOS and Linux. Of course, Flash support under Linux has made huge progress, with e.g. the Flash 64bit version released under Linux first. But all RIA platforms try to address the 'fat client' offline problem: running apps while disconnected from internet, even outside the browser (e.g. Adobe AIR or Java Update 10). Unfortunatly, those 'fat client' standalone versions of RIA apps offer a consequent look on all OSes. This might be a plus, but the current trend is to adapt your UI to the OS. Firefox 3 for example has a different look and feel under Gnome than under Vista.
- hackability: I'm addicted to noscript, adblock and firebug, but those tools are useless in a RIA context. Even a simple 'save as', as Firefox provides, is not straightforward. Perhaps the development of gnash might bring some new hacker-tools to the scene.
- open standards: html4, css, ... why are they missing from the RIA picture? Yes, ActionScript is like EcmaScript, which is like JavaScript. Anyway, in real apps, ActionScript only runs in Flash.
I do think that Flash and Java FX have their value when used to enrich standard web pages. You'd be amazed to see what is possible in plain html/javascript/css. Look at the spectacular Apple ITunes store for example. In general everything that's not supported in html4 requires flash: audio/video or drawing. By putting those pieces in a classical webapp you get a tremendous combination. For example, Google Analytics or Google Finance are html pages, but the nice graphs are Flash. The integration is seamless, an end-user can see no difference between the html and the flash parts.
We'll see what the RIA-future brings: all Flash (flex/air), Java FX, or a "back to the web" reaction?
Subscribe to:
Posts (Atom)