Tech stuff: January 2009

2009/01/02

Devoxx: Developing Beyond localhost by Brian Leonard

Brian demoed how combining VirtualBox and ZFS can provide a very flexible and dynamic development and testing platform for web-developers. VirtualBox is a simple, free yet powerful virtualization tool. It allows you to setup very easily virtual machines having multiple OS's and configurations. You don't even have boot those virtual machines over and over again: a snapshot can be saved, and restored almost instantly. After setting up several virtual machines, you quickly get a relevant set of client-machines on which you can test your webapp. All virtual machines get a unique ip address, so connecting to the web server from you virtual client is as simple as finding out your local IP serving the web app. Since all vm's are simple files, you can backup the 'clean' vm's and start playing around with them. Restoring a vm is as simple as restoring the original snapshot file. Virtual Systems can run windows outside of the 'vm-box', or on contrary, you can run the Virtual machine full-screen; that way it's easy to forget that you were actually playing in a virtual environment.

I really do like the Virtual machine principle and Virtual Box is a pretty neat solution -- for free! The issues however start with closed OS's: Windows e.g. requires new license keys and Mac OS doesn't run on Virtual Box.

The ZFS (file system) is a crown-jewel of Sun. Its feature-set (security, backup ...) really stands out compared to legacy NFS, ext2/3 etc. In ZFS creating a volume is almost as cheap as creating a directory. Brian used this on his OpenSolaris laptop to create a dedicate ZFS volume under his home directory for Virtual Box snapshots. ZFS-volumes can be configured to take regular snapshots, or to take the snapshot on request. After messing around with his Virtual Box snapshots, he simply undoed everything with a single ZFS "restore snapshot" command in only a few seconds. Truly amazing.

I really think ZFS should be supported in Linux, although the CDDL license is not compatible with GPL v2. There is a zfs-on-Fuse project (=file system in userspace), but it is still in beta fase and performance should be lower than native kernel support. Although the excellent ntfs-support (ntfs-3g) also uses fuse with good performance.

references:

Devoxx: JRuby by Charles Nutter and Thomas Enebo

Charles and Thomas did a good job in getting the java-audience in touch with ruby's beauty. The demos looked really slick and development seems very efficient and intuitive. I still have the impression though that ruby works best if you completely adhere and embrace ruby's principles. The trouble starts when you want to take slightly different approaches than the 'sensible defaults'.

Some (J)Ruby's features:

closures
operator overloading
Ruby Gems is the repository
'module' = mixins of scala

Ruby on Rails web development platform:

convention over configuration
DRY don't repeat yourself
'agile': hot deploy, generate basic structure
persistence: ActiveRecord (=driven by DB structure)
rhtml + partial rhtml = (sort of similar to) jsp and jspf
multi-format support: e.g. xml and html based on http accept header
warble: create standard WAR based on bundle

examples of sites using JRuby:

references:

Devoxx: Introduction to Asynchronous I/O (NIO.2) using the Grizzly Framework by Jean-Francois Arcand

I was hoping to get a general 'New NIO' overview, but Jean-François concentrated most of his talk discussing low-level Asynchronous IO usage patterns in Grizzly using New-NIO:

waiting for Java 7
'Selector' and Notify principle
'CompletionHandler' that uses a 'Future' to listen to the socket without wasting threads
thread pools: fixed, cached thread pools built-in to the kernel (but: risk of blocking the kernel!)

Other New NIO features in java.nio.file:

FileSystem
FileRef for having native symbolic links in Java
Path
FileStore

references:

Devoxx: Towards a dynamic VM by Brian Goetz and Alex Buckley

The JVM and the Java Language should be treated as two decoupled things. The JVM provides the basic infrastructure on which the Java Language builds: Garbage collection, reflection, security, Hotspot ... All those things make the JVM appealing to other languages like JRuby, Jython, Groovy and Scala. Choosing for the JVM as a platform has also tradeoffs like for example the fact that the JVM is stack based (vs. the registry based VM in for example the Davlik VM of Android).

Dynamic language on the JVM problems and solutions (mostly focusing on JRuby). All this is quite low level for me, I just wrote down some keywords I might investigate further some day:

'Open classes' --> JSR 292 invokedynamic =consider argument types at runtime = invokevirtual optimized
overhead of reflection and BigInteger
'inline caching' trick to cache 'Method Objects'
'invoke interface'
tail calls
continuations
tuples
value objects

references:

http://www.devoxx.com/display/JV08/Towards+a+dynamic+VM
concerning register-based vs. stack-based vm's:

Devoxx Keynote: Java SE 7 Update by Mark Reinhold

In the online schedule this item is titled differently: 'The Modular Java Platform'. Anyway, Mark talks about both the Modular stuff and Java SE 7 features.

Concerning Modularity in Java 7:

Java 6 Update 10 couldn't add any features, but managed to speed up download and startup times thanks to, amongst others, the Java Kernel and pack200 compression. Still startup time can't compare to other more targeted environments like python for example.
(*) JSR 277 is on hold (frozen?). I suppose there was too much resistance from the OSGI camp as being competing technologies. I do think that explicit language-support for modules is needed in Java, but this is JSR-294's sweet spot.
(*) JSR 294 will contain all language modularization efforts for Java 7
(*) The scope of the 'Jigsaw project', which seems to run outside the JPC, wants to differentiate from OSGI by having different more low-level objectives in applying modularization to the JRE:

allow the java kernel itself to be modular: e.g. distribute targeted Java 'Profiles': Java FX, applets, full SE...
modularizing the kernel might be a solution to all the deprecated APIs in Java that still have to be provided for backwards compatibility. A possibility could be to provide a basic Java profile without the 'old' classes, and allow a simple module to extend the Java Runtime in order to add the deprecated stuff when needed.
integrate with OS package systems: cfr. JPackage project creates rpm's from jars. --> how to package / deploy reusable modules / JVMs?
what about version collisions?
integrate with the VM, language and native packaging
allow multi-module packages
support "friend" packages (for tighter integration)

Java SE 7 update:

JSR 292: better JVM support of dynamic language (JRuby, Jython ...)
JSR 203: New NIO: FileSystem, access to symbolic links ...
small language changes, too be discussed / decided --> Joe Darcy

safe rethrow
null dereference: shorthand to do chained tests on null in one line:
a?.b()?.c()?.d()
type inference
multicatch

others:

JSR 296: Swing application framework
upgrade Java 6 Update 10 to Java 7
update supported protocols
enhance classloaders
Unicode 5.0 support
improve HotSpot performance
new G1 garbage collector
compressed 64 bit pointers (use less memory)
support "Multiple VM's" (MVM) --> run light, isolated apps like Java FX applets
JSR 308: standard annotations eg. @NonNull used for static analysis
concurrency update: collections, fork/join ...

What will not be included in Java 7:

Closures: prototypes exist, but too early to dump in java
properties
beans binding: need more time to see if this dependency injection framework fits well in core java
operator overloading

references:

Parleys.com now published Mark's talk:

(*) modified thanks to comments / further reading...

Devoxx: Caching web contents in the browser by Ignacio Coloma

Always nice to refresh basic web skills like caching:

last-modified http header
e-tag header
cache-control header of http1.1
http 304 'not modified' indicator
refresh vs shift-refresh --> the latter ignores the cache

Reference:

http://www.devoxx.com/display/JV08/Caching+web+contents+in+the+browser

Devoxx: From Concurrent to Parallel by Brian Goetz

The trend of getting faster processors every year just stopped a few years ago. Modern CPUs now get more cores, but each single core is actually slower than its single-core ancestor. Buying new hardware won't automatically make your software run faster anymore, unless your software is built for parallel processing. The more cores are added to the cpu, the more important it becomes to find fine-grained parallelism in your apps, in other words, to split coarse-grained request into smaller chunks.

Java 5 brought the basic concurrency support beyond Threads into Java: Futures + Executors. Those are very powerful concepts, but the resulting code is complex to maintain. Java 7 builds on the existing java.util.concurrent foundation by providing the 'fork-join' framework to Java (JSR166).

The concept behind the fork-join framework is "divide and conquer": a job is recursively split in halves, until a manageable size for the sub-jobs is found. The correct size for the chunks is typically determined by the number of cores (=Runtime.availableProcessors()). Another key concept is "work stealing": if one thread finishes its job early, it 'steals' work from other 'double-ended queues' (Deque) of other threads. That way all cores can remain busy while limitting congestion for the queue.

Java 7 java.util.concurrent.forkjoin:

basic fork join classes: RecursiveAction, AsyncAction, CyclicAction ...
higher abstraction: ParallelArray offers parallel functions out of the box: filter, map, replace, aggregate ... I like the declarative style use for this class: it chains declarations before firing the action: students.withFilter(isSenior).withMapping(selectGpa).summary();

Definitively an interesting step for Java, although once you enter a multi-threaded environment, you loose many safety guarantees from Java and you as a developer has to ensure that concurrency doesn't break your app. The growing number of cores puts parallel programming research in the spotlight, with the renewed interest in functional languages like haskell, erlang and Scala for example.

References:

(very small) intro http://www.devoxx.com/display/JV08/From+Concurrent+to+Parallel
jsr-166: http://www.infoq.com/news/2008/03/fork_join
research paper: http://gee.cs.oswego.edu/dl/papers/fj.pdf
IBM: http://www.ibm.com/developerworks/java/library/j-jtp11137.html
IBM: http://www.ibm.com/developerworks/java/library/j-jtp03048.html

Devoxx Keynote: RFID and IBM

Devoxx experimented for the first time with RFID tags to monitor talk attendance. In fact, every Devoxx access-badge was an RFID tag and each room had an RFID reader on its entrance that monitored each individual RFID passing through.

IBM presented their backend processing infrastructure to capture, process and present the RFID information at Devoxx. Looked really very complex -- not sure if such a complex setup was needed or if it was just a way to demo their Tivoli monitoring infrastructure.

I was more interested by the RFID-technology itself:

the used RFID readers had a 5 meter detection range, but they stilled missed about 20% of all tags.
depending on the frequency used, some RFID tags can be 'shielded' from the readers by water. This can explain the 20% error range because a human body mostly consists of water.
the RFID readers can't tell if you're entering or leaving a room. This is deduced by the backend system by correlating the different individual tag-readings. E.g. if a tag is detected in room A and then in room B after 30 minutes, it supposes that you were out for those 30 minutes
RFID tags exist in several types, each with its own usage patterns and pros and cons:

'active': the tag has its own battery and radio
'active backscatter': the tag has a battery embedded
'passive': no battery nor radio

RFID tags come also in different form factors:

single use: as 'printed' labels. This was used for the Devoxx badge
rugged reusable tags.

the RFID readers are actually quite advanced systems that contain a built-in JVM. This provides opportunity to do some basic processing when retrieving the scanned data.
the classical questions on privacy / security / health remain.

It was an interesting experience to get 'live' in touch with the RFID technology, its pros&cons and its technical and ethical issues.

Devoxx Keynote: JavaFX - The Platform for Rich Internet Applications by Danny Coward

Java FX version 1 is out now and looks promising. Although it hits version 1 I don't think it is really ready for prime time:

no Linux (or OpenSolaris ;-) support yet. Really a missed opportunity to get 'Java' in a more positive spotlight in the Linux users' mindset.
native Mac and Windows video formats are supported, but to get real cross-platform media, FXM must be used (http://java.sun.com/javafx/faqs.jsp#19). Supporting native video formats but claiming full cross-platform support seems contradictive to me.
I could still see some small glitches like small GC-pauses and startup-waits. Not sure if those are relevant or just demo-quirks.
the promise of 'multiple devices, one language' is still very vague to me. I have no clue how full screen apps can run great on small phones without any change.
licensing is still blurred. Although Sun is committed to opensource the Java FX core (http://openjfx.org). Real open source of Java FX would be a big plus compared to Flash, especially for the Linux distro's, but I doubt Sun will be able to remove all encumbered stuff like the multimedia codec's.

I'm not really a "shiny flashy apps" developer, but once Linux support is there I might delve deeper into Java FX.

References: