2009/11/29

Accessing GSM SIM-Cards under Ubuntu

I borrowed a cheap GSM SIM-card reader and tried to use it under Ubuntu. The "Zolid" card reader is both a memory-card reader as well as a SIM card reader. The device presents itself to Ubuntu as follows (including the typo):
$lsusb
Bus 001 Device 005: ID 0bda:0169 Realtek Semiconductor Corp. Mass Stroage Device
The 'Mass Storage Driver' that mounts SD cards just works fine out-of-the-box. I concentrated my experiments on the SIM-card reader. I installed the 'MonoSIM' application to access the device.
In Applications/Office you can run the MonoSIM application and manage all phone numbers on it.

Some scripts that might be useful:
  • combine the 'monosim' file records to 1 line per record
    $ cat *mono* | while read a; do read b; echo "$a;$b"; done | sort -u > gsm.csv
  • I used OpenOffice to filter out old lines and save the file back to csv
  • converting the csv back to the 'monosim' format:
    $ while read a
    do
    echo $a | cut -d '"' -f 2
    echo $a | cut -d '"' -f 4
    done < gsm.csv > gsm.csv.monosim
The MonoSIM application works fine; the trick is to reconnect whenever a menu-item is greyed out.

see also: http://www.barrydegraaff.tk/index.php?page=files/Archive/Linux%20Sim%20Card%20reader/index.html

2009/11/24

Fixing Auto-mount Problems of a USB Device in Ubuntu

On one of my laptops with Ubuntu 9.10, USB memory sticks would only mount after several plug/unplug attempts. I investigated the problem and found a solution.

Diagnosis:

  • $ tail -f /var/log/messages (or call dmesg)
    ...
    Nov 24 21:33:41 michael-laptop2 kernel: [ 2592.748078] usb 2-1: new full speed USB device using uhci_hcd and address 2
    Nov 24 21:33:44 michael-laptop2 kernel: [ 2595.113326] usb 2-1: configuration #1 chosen from 1 choice

    → only after several attempts, the log would display "Initializing USB Mass Storage" and proceed with the auto-mount.

  • $ sudo lsusb
    Bus 002 Device 002: ID
    041e:200c Creative Technology, Ltd
    → in this case, the USB stick is an mp3 player

  • $ sudo lshal
    ...

    udi = '/org/freedesktop/Hal/devices/usb_device_
    41e_200c_4002FA76125D8F8A_if0'
    info.linux.driver = 'ndiswrapper'
    (string) info.parent = '/org/freedesktop/Hal/devices/usb_device_41e_200c_4002FA76125D8F8A'
    (string)
    info.product = 'USB Mass Storage Interface'
    (string)
    info.subsystem = 'usb' (string) ...
    → the "41e_200c" USB device has been wrongly associated to the ndiswrapper module.
Conclusion: my old experiments with ndiswrapper and the rtl8187b wifi device still left traces on my Ubuntu system I upgraded to 9.10.

Removing ndiswrapper was a bit tricky:

  • $ sudo ndiswrapper -l
    net8187b : driver installed
    device (0BDA:8189) present (alternate driver: rtl8187)
  • $ sudo ndiswrapper -r rtl8187
  • in synaptic: remove all the ndiswrapper-* packages
  • $ sudo rmmod ndiswrapper
  • $ sudo gedit /etc/modules
    → remove ndiswrapper line
  • $ sudo rm /lib/modules/$(uname -r)/kernel/ubuntu/ndiswrapper/ndiswrapper.ko

Now, usb sticks mount from the first time.
see also: https://help.ubuntu.com/community/WifiDocs/Driver/Ndiswrapper
update 25/11/2009: apparently, the ndiswrapper is part of the ubuntu distribution. After applying the latest security-fix, the ndiswrapper module re-appeared.
update 6/6/2010: the ndiswrapper bug still exists in Ubuntu 10.4.

2009/11/19

Devoxx 2009: Performance for the Performance-shy

19/11/2009, Holly Cummins, IBM

costs of bad performance
  • electricity
  • employee productivity
  • lost business (e.g. slow pages)
  • hard cash (e.g. trading)
tuning technique
  • find the bottleneck in the application
    • cpu → consistently high
    • I/O → cpu not consistently high
    • lock → cpu not consistently high
    • space ??? cpu can be low or high ...
  • what is the JVM doing? → tools … must have:
    • visual indicators
    • explanation
    • suggested solution (command line)
  • measure performance
    • risk to make it worse
    • get a baseline
    • system must be as similar as possible to production
    • how:
      • System.currentTimeMillis()
      • warm up your system
        • JIT
        • trigger caches
tools: IBM Perfomance Tools
  • for IBM jvm's
  • free
  • IBM Support Assistant: centralized repository for IBM tools
space-bound applications: heap
  • diagnose:
    • lower throughput
    • crash (out-of-memory exception)
    • trigger gc's → higher cpu
  • tools:
    • verbosegc: very low overhead → use on production
    • live memory monitoring → IBM Monitoring & Diagnostic tools for Java
      • GC and Memory Visualizer
        • for Websphere → also works on Solaris & HP-UX (i.e. sun jvm syntax)
        • heap details, including heap histogram
        • get dominator tree
        • search inefficiently used data structures & redundant data
        • recommendations
          • increase heap size
          • gc takes between 10% - 20% cpu → tune gc policy
      • Health Center
        • agent in JVM
        • client (IBM Support Assistant)
        • visualize gc
        • recommendations
        • check gc performance
    • Memory Analyzer (not from IBM → eclipse.org): diagnose footprint issues:
      • heapdumps
        • automatically on out-of-memory
        • trigger programmatically
        • tool support is essentialµ
    • OS-tools
      • track native memory: perfmon, vmstart, ps (os-specific)
      • use gcmv to visualize os-specific dump
      • some native structures have java-wrappers in the heap (e.g. Threads and NIO buffers)
cpu-bound applications
  • target your optimizations → measure
  • diagnose:
    • method trace:
      • System.out.println()
      • log4j
      • ibm-jvm param -Xtrace:print=mt,methods=(HW*)
    • method profiler:
      • health center: low overhead sampler
I/O bound applications
  • gc and memory analyser → long gc times might indicate paging
  • method trace for network and disk I/O
  • os-specific tools
synchronization issues (concurrency)
  • sample thread dumps
  • health center:
    • lock analysis
    • identify contended locks

Devoxx 2009: The not so Dark Art of Performance Tuning

19/11/2009, Kirk Pepperdine & Dan Hardiker
  • loadtest <> stresstest:
    • loadtest: test user experience
    • stresstest: find when the application breaks
  • performance anti-patterns & solutions:
    • lack of stress testing → introduce a stress test harness
      • repeatable test
      • control load
    • shot in the dark: fix ugly code
      • ugly code runs just fine
      • measure, don't guess!
  • Apache JMeter → root node: “Test Plan”
      • Thread Group
        • set looping / # threads
      • HTTP Request
        • → variables: ${var}
          • defeat cache
          • vary testdata
      • Timer child for think-time→ e.g. Uniform Random Timer
      • aggregate report
      • view results tree → captures data of requests: e.g. don't measure 404 errors
  • understand the environment, identify dominating consumer → look at the full system
    • actors: usage patterns
    • application
      • locks
      • external systems
    • jvm/os
      • memory
      • hardware management
      • tools e.g.:
        • vmstat on hpux
        • jps
        • jstack
        • visual vm (thread and heap dumps)
    • hardware
      • cpu
      • memory
      • disk I/O
      • network
  • response time budgets:
    • detail the time of the different components → add customizable time measures
      • database
      • application server
      • client
  • common problems, demo's:
    • poor response times, low cpu
      • thread pools
      • locks
      • VisualVM, thread dump + plugin thread dump analyzer, identify lock

Devoxx 2009: Using XML with Java: Sploit for Choice?

19/11/2009, Michael Kay, Saxonica

Xml interfaces in java, from low level to high level
  • SAX/Stax
    • high performance: +/- 40 MB/sec
    • low-level
    • SAX/Stax: comparable performance
    • use pipelines:
      • SAX: push
        • best for multiple output
      • Stax: pull:
        • best for multiple input
  • DOM: tree based
    • don't use plain DOM, prefer JDOM, DOM4J or XOM – I once experienced problems with the XOM-handling of unicode characters
    • uses a lot of memory: +/- 5 times the document size
    • complex, tedious usage
  • databinding: compile to / (un)marshall from java objects
    • like JAXB
    • fits for simple xml
    • not for complex xml or document xmls → create too much java classes
    • give 2 views on the same: xml <> java objects
  • language support:
    • doesn't exist in Java...
    • scala: XQuery-like capabilities
  • XSLT/XQuery
    • should be your default choice
    • good performance: +/- 10MB/sec
    • XSLT:
      • more for documents
      • transformations
      • richer functionality
    • XQuery:
      • more for data
      • xml-database
      • easier to optimize
Architectures
  • prefer xml pipelines: (xproc pipelines)xml database → xquery → xslt → rest → xforms
  • don't mix data: objects & xml

2009/11/18

Devoxx 2009: Project Coin

18/11/2009, Joe D. Darcy, Sun

slides: http://blogs.sun.com/darcy/entry/project_coin_devoxx_2009

Project Coin
  • small language changes in JDK7
  • keep broad consideration of java landscape → remain consistent with the Java Language
  • bigger language changes, not in scope of Project Coin
    • JSR308: annotations to improve static checkers (e.g. @NotNull)
    • JSR294: modularity
principles of Project Coin:
  • reading more important than writing
    • 'beautiful' code
    • don't hide what's happening
  • simplicity matters
  • no slash-and-burn: remember broad user-base
  • balance stability versus progress
  • prefer library / IDE changes over language changes (String.join)
  • no new keywords
  • no type system changes
  • small in specification, implementation and testing
methodology:
  • open: blog entries / white board on Devoxx2008
  • call for proposals
  • prototypes recommended
    • lot of work
    • very beneficial → learn / find issues
  • existing requests for enhancement: 120 changes → too much
final list of Project Coin enhancements
  • numbers:
    • binary notation: 0b00100100001
    • group numbers with '_': long big = 1_123_456_789;
  • switch on a String – really cool!
    • specification: only one word added
    • proposal contains a suggested implementation:
      • uses String.hashCode() for an initial switch
    • big patch in javac
    • lots of tests to validate change
      • reject pre -source 7 classes
      • null behavior
      • hash collisions
      • fall-throughs
      • → comparable to enum switch control flow
  • diamond <> operator : type interference → e.g. Map <String,String> map = new HashMap<>();
    • a lot of prototyping work
    • quantitative analysis
    • some concerns on language evolution were considered
  • JSR292: invokedynamic for dynamic languages
  • auto-close of Closable types within a special try block (ARM)
    • try(File f = new File) {} → f is automatically closed when leaving the block
  • simplified varargs method invocation, working along with generics
  • language support for Lists and Maps:
    • List<String> list = [“1”, “2”];
    • list[0]
    • map{“hello”}
not retained in Project coin:
  • Elvis operator:
    • use ?: as a shorthand for testing on null
    • conclusion from building a prototype→ doesn't fit well in java
  • multiple exception handling
    • disjunctive means of | operator
  • properties:
    • changes the type system
    • a lot of work (e.g. reflection api, corner cases …) – the 'enum' specification is demonstrated as a comparable change that involved a lot of rework
    • pseudo-properties workaround: via annotation-processing → Project Lombok
  • reified generics
    • source language compatibility issues – java generic erasure was chosen for backward compatibility
    • C# created a separate generic library
    • difficult for other languages to target the a platform with reified generics
  • extensible enums
    • most enums are constant
    • low utility

Devoxx 2009: Solr Power with Lucene

18/11/2009, Erik Hatcher

Lucene core
  • full-text search library
  • concepts
    • inverted index:
      • term + proximity
    • documents
    • fields
      • field-ids: e.g. category, title, name...
      • types: number, date, text...
      • unique keys: unique id per document
    • terms (aka tokens):
      • processed through filters
        • synonyms
        • ignore words
        • stemming
    • scoring relevancy
      • term frequency
      • inverse document frequency
      • field length normalization → control how field length / # occurrences affects scoring
      • boost factors: favor or boost some fields (e.g. titles)
  • core
    • standalone jar
    • core index
Apache Solr properties:
  • search server
  • based on Apache Lucene
    • → Lucene exposed over http
    • spell checking
    • highlighting
  • extensible
  • scalable
    • caching
    • replication
    • master/slave distributed search → sharding
  • multiple inputs
  • version 1.4
Using Solr
  • setup:
    • solrconfig.xml:
      • cache settings
      • Lucene indexing parameters
  • API:
    • RequestHandlers:
      • mini-servlets,
      • flexible responses:
        • http GET/POST
        • JSON
        • SolrJ
        • ruby, php, …
        • content streams (must be shielded)
    • indexing / deleting a document
      • through api: xml document with commands
      • POST or GET with request parameters
    • other actions:
      • commit / rollback: batching document indexing
      • optimize
    • search request: simple GET, with optional parameters
      • debug
      • lucene explanation
      • pagination: start / raws
      • score: lucene score
  • DataImportHandler
    • import from RDBMS, xml and e-mail
    • incremental indexing
    • extensible
    • debug console
  • Solr Cell: uses Lucene Tika:
    • index Word, pdf, html ...
    • ExtractingRequestHandler
  • Query parser framework with plugable parsers:
    • Lucene syntax:
      • powerful
      • but user-unfriendly syntax
      • exceptions visible to end-users
    • Dismax query parser
      • simplified syntax
Advanced Solr: Search Components
  • standard: query, facet, mlt, highlight, stats, debug
  • others: elevation, clustering, term, term vector
  • faceting
    • counts subset within results
    • group 'facets' of a document (like a category field)
  • spell checking
  • pluggable distance algorithms: Levenstein or JaroWinkler
  • highlighting: custom prefix and suffix → response is highlighted
  • query elevation → elevate.xml: boost or exclude a document
  • clustering: grouping of documents into labeled sets
  • enumerate terms for a field
  • term vectors: term frequency, document frequency, position, offset
  • statistics: stats.jsp (in RAM); returns xml
  • scaling:
    • replication:
      • master is polled
      • replicant pulls Lucene index / config files
      • replicate + load balance
    • distributed search: single index is too large → sharding
Staring with Solr
  • agile, iterative process works best:
    • basic schema
    • bring in data
    • check requirement gaps
    • adjust solr

Devoxx 2009: Architecting Robust Applications for Amazon EC2

18/11/2009, Chris Richardson, SpringSource (VMware) www.cloudtools.org

Amazon EC2 cloud computing solution:
  • choose your virtual machine, with machine images of standard OS's
  • cloud services
    • SQS: Simple Queue Services
    • SimpleDB: alternative to relational DB
    • S3: storage
    • EC2 firewall: security groups
    • Elastic Block Storage:
      • replicated, mounted as device
      • snapshots → backup
    • Elastic Relational DB Service:
      • MySQL 5.2
      • maintenance window: 4hour/week
      • backup window
  • standard software: simply install (with yum or apt) your packages
  • console
  • local storage:
    • plenty (160GB → 1690 GB)
    • ephemeral
    • slow down on first access
    • → Elastic Block Service
  • easy upgrades:
    • clone your production environment
    • upgrade the clone
    • test the clone
    • if OK: put the clone in production
    • remove the old clone
  • regions:
    • US
    • EU
    • Asia
    • sub-zones:
      • “availability zones” that are isolated from each other.
      • better availability
      • more expensive
issues:
  • security:
    • not PCI-compliant
    • discomfort of putting your core data on the cloud
    • where is your data really?
    • but:
      • Amazon experience
      • unlimited logging
      • standard best practices for security
      • isolated instances
      • put servers in different security groups → Amazon EC2 firewall
      • encrypt your data and backups
  • cost:
    • expensive for large servers
    • bandwidth
    • but: no extra cost of cooling, power etc.
  • machines:
    • no very small, very big machines
    • maintenance windows
    • load-balancing:
      • ipbased: no support for sessions
      • dns (cname)
  • is it reliable?
    • is internal IT better?
    • Amazon expertise
    • easy to launch a new instance within a few minutes
future:
  • Platform as a Service:
    • complete stack
    • simpler scaling, without being involved into the technical details
    • Google Application Engine
    • Cloud Foundry:
      • Spring & Grails

Devoxx 2009: JDK 7 Update

18/11/2009, Mark Reinhold

higlight: Closures in java are BACK!

Aims of JDK7
  • modular platform
    • JDK7 status:
      • 13MB download with pack200 compression
      • difficult to support smaller (embedded) hardware
    • → Jigsaw project
      • low-level modules, at the rt.jar level
      • language support: module-info.java file
        • explicit module / version requirements
        • create simpler OS-specific package, with correct dependencies (e.g. deb-packages)
      • OSGi interoperability
  • multi-lingual support
    • better JRuby, Jython support on the JVM
    • InvokeDynamic
    • “Da Vinci Machine project”
  • productivity enhancement
    • “Project Coin”
    • language changes that improve programming experience. E.g.:
      • type inference with the <> (aka. 'diamond') operator: Map <String,String> m = new <> HashMap();
      • split numbers with '_' for better readability: int a=123_456_789;
      • support binary notation: int b=0b00100100001;
      • Lists: direct construction and access to set/get with []: a[4] = b[5];
      • Map: direct construction and access to set/get with {}: a{“b”}=”xyz”;
      • auto-close Closeable interfaces: try(File f = new File()) where f is always closed correctly.
  • performance
    • multi-cores are there! → java support needed.
    • → fork/join framework
    • Dual-Pivot Quicksort
    • Closures: “It's time to add them” – I thought this was dropped from Java7 plans...
      • needed:
        • function literals
        • function types
        • extension methods
      • “doesn't fit in java”:
        • non-local control transfer
        • capture non-final vars
        • library-defined control structures
When?
  • Planning shifted
  • Milestone 10: 2010/09/09: release candidate

Devoxx 2009: Keynotes 18/11/2009: Parleys.com, Oracle, Sun (JEE6) and Adobe

Stephan Janssen: Devoxx organizer,Parleys.com founder
  • presented his new Parleys v3
    • support for 'spaces' and channels in spaces → propose parleys.com as a platform
    • monetize:
      • subscription for impatient viewers (i.e. for immediate access to the talks); talks with free access are provided within a year
      • give private spaces to companies for education purposes
    • still presented in the over-Flashy way
    • I hope they'll get the podcast feed – finally – fixed.
Oracle slot
  • mostly boring management talk, with lots of empty words
  • WebLogic DM:
    • heavily based on OSGI
      • 'Profiles' group technical bundles
      • 'Bundles' are the application-specific modules
    • possibility to run on a “bare-metal” hypervisor VMWare: http://en.wikipedia.org/wiki/Hypervisor– new concept to me
Sun:
  • JEE6
    • final release scheduled for 10/12/2009
    • → enterprise development made simpler
    • basic contents (a.o.)
      • new stuff:
        • JAX-RS: annotation-based RESTfull API
        • Bean validation, used in the full stack (JSF and JPA)
        • Dependency Injection (DI) 1.0
      • updates:
        • EJB 3.1
        • Servlet 3.0
        • JSF 2.0
    • support for technical profiles: set of basic, but quite complete JEE components.
      • e.g. WebProfile: Servlet 3.0, JSF, EJB 3.1 lite, DI
    • pluggable, modular web applications
      • web.xml is now optional
      • web-fragment.xml
      • annotations:
        • @WebServlet
        • @WebFilter
      • resource jars → finally a standard way to package static resources
      • programmable registration API
    • Dependency Injection 1.0:
      • @Resource
      • @Inject → Guice-like standard solution
      • Inject metamodel
    • EJB3.1
      • @Singleton
      • @Startup
      • @Asynchronous
      • EJBContainer API, useable in Java SE (for testing)
      • EJB's, directly useable in webapps
    • JSF 2.0
      • standard facelets
      • autodiscovery of component libraries
      • Ajax & partial view loading
      • JavaScript API
      • composite components
  • Glassfish demo
    • Eclipse & NetBeans support
    • not sure how Oracle will handle the GlassFish versus WebLogic proposal
    • great programmers' experience:
      • fast startup
      • incremental deploys applied to a running application without restarting → JRebel-like feature.
Adobe stuff
  • skipped this one – too much flash for me :-)

2009/11/17

Devoxx 2009: A Year of Monitoring with Java-Monitor

17/11/2009, Tools in Action, Kees Jan Koster

Slides are available at: http://java-monitor.com/forum/showthread.php?t=646

Case-by-case approach from experiences reported on java-monitor.com, from an Operations' perspective.

Memory Leak:
  • diagnose:
    • Scavenger gc gives up and stop the world gc (= mark and sweep gc) takes over
    • monitor # scavenger gc's versus mark and sweep gc's
  • solution:
    • profile your app and fix your app
    • workaround: increase heap
Misbehaving GC
  • problem: System.gc() in code freezes the JVM!
  • diagnose:
    • many stop the world gc's (= mark and sweep gc)
    • low heap usage (e.g. 20%)
  • solution:
    • workaround: -XX-DisableExplicitGC
    • use findbugs and fix code
Native threads in 32 bit environments
  • problem:
    • OutOfMemoryException: Couldn't allocate native Thread
    • 32bit Systems share 4GB with OS Kernel, JVM (heap + perm space) and thread stack space.
  • solution:
    • workaround: -Xss limitThreadStack
    • migrate to 64 bit!
In verboseGC: "promotion failed"
  • problem:
    • verboseGC logs "promotion failed" --> stop-the-world GC (mark and sweep) is triggered
    • CMS is non-compacting: the heap can end up to be fragmented. Fragmentation can prevent promotion of young heap (eden spaces) to old heap.
  • solution:
    • bigger heap
    • no real solution
File Descriptors
  • problem:
    • "too many open files"
    • files not properly closed on exceptions --> file description leakage
    • GlassFish opens file-descriptors for each static file served (efficient usage of FileChannels)
  • solution:
    • check ulimit / limits of your OS
    • fix your application (findbugs)
others
  • hyperthreading doesn't make any significant difference
  • use the magic "-server" switch

Devoxx 2009: Full Text Search for Hibernate

17/11/2009, University sessions, Emmanuel Bernard

Search solutions:
  • categorize upfront
  • show detailed search screen
  • use single search box (preferred)
Plain SQL search limits:
  • performance: like '%...%' causes a full table scan
  • no support for approximation nor synonyms
  • no proximity concept
  • lacking relevance scoring
  • no simple multi-column search
Full-text search solutions:
  • word based
  • captures / indexes frequency and position
  • solutions:
    • RDMS: (like Oracle Text):
      • less flexible
      • not portable (vendor-specific API and behavior)
    • standalone: Lucene
      • text-only
      • no synchronization with model objects
Hibernate Search, general features:
  • LGPL
  • uses Hibernate core
  • uses Lucene under the hood
  • solves object vs text mismatch
  • convert object to text document (+reverse) → Hibernate application uses objects, not text
    documents
  • convention over configuration
  • heavily built on annotations
  • Optimize Lucene access:
    • update Lucene docs on commit
    • object graphs are consolidated to single Lucene docs to provide relevant searches
    • avoid flooding Lucene indexer:
      • batch Lucene updates on commit
      • optionally trigger the Lucene indexer asynchronously
    • support clustering (JMS)
Hibernate Search Annotations:
  • @Indexed
  • @Field: tunable how to convert to text with, among others, @FieldBridge. E.g. convert number to 0-padded number.
  • @IndexedEmbedded
  • @Boost: promote a particular field in the relevance score (can be at indexing time or at query time)
  • @Analyzer: e.g. anagram-support
Lucene Index as used by Hibernate Search:
  • event based
  • batches updates per transaction (=at commit time)
  • sync or async mode (optimize Lucenes' locking mechanism)
Query:
  • HQL
  • Full Text (Lucene syntax) e.g. with the ~ opperator
  • JPA2 criteria
  • native SQL
  • → always returns Objects, not Lucene documents.
Advanced stuff:
  • tokenizer: split text in words, remove common words
  • complex searches: combination of indexing and querying
  • fuzzy search:
    • “Levenstein distance”: quantifies similarity
    • “n-gram”: word is split in groups of 3 letters → matching groups determines score. (demo looked a bit hackery)
  • phonetic search (soundex-like): disappointing in practice
  • synonyms: use your application-specific list
  • stemming: → 'reduction'
    • Porter Algorithm
    • Snowball stemmer
  • filters: provide efficient an pluggable support for
    • security, categories, temporal data, caching...
  • “explain” query result
  • clustering / Scaling Lucene
    • one Lucene writer at a given time
    • use a JMS queue for indexing (→ 'Master')→ small delay, but very scalable.
    • Distributed in-memory index (Infinispan 4.0) – technical preview
    • index optimizations:
      • sharding
      • defragmenting or re-indexing

Devoxx 2009 – Google Application Engine - A Real Live Voyage to The Cloud

17/11/2009, University Sessions, Sam Brodkin and Scott Stevenson

Google Application Engine for Java (GAE/J) is a cloud computing solution that provides
  • a JVM
  • OS support
  • access to the Servlet API
  • a management console
  • access to Google utilities:
    • Logon / Sign up → com.google...User
    • image service: resize images
    • mails
  • building blocks:
    • DataStore
    • MemCache
Cost model is based on actual usage, with a free quota for easy starting:
  • bandwidth usage (in/out)
  • cpu usage
  • datastore size
  • emails sent
Management console:
  • access to logs
  • shows bandwidth / cpu usage → cost
  • users with admin role
  • statistics
  • manage multiple versions: one default, and many test versions (beware of data-incompatibilities)
Limits of GAE/J:
  • requests timeout: 30sec
  • no background processes allowed
  • no server push
  • no threads
  • read-only file-system
  • no transaction
  • only access to white-listed classes (e.g. ImageIO class of java is not accessible → use Google Image service)
  • a maximum of 3000 static resources per application
  • Migrating your application off the GAE/J cloud: difficult, but not impossible → google for “GEA Bar”
DataStore for data persisency
  • not a relational database
  • BigTable → like a big HashMap
  • no transaction
  • JDO or JPA access
Development Environment:
  • eclipse plugin
  • Jetty based emulator → fair emulation, but not 100% relevant
  • MemCache
  • disk-based datastore
  • compile-time checks
  • no maven support yet
  • DIY ant scripts are possible and recommended.
Demo: “swagswap.org”
  • 3 tiers:
    • DAO
    • service layer
    • 4 frontends that use the same
      service layer:
      • jsf 2.0 (+IPhone version)
      • gwt 1.7
      • SmartGwt
      • Spring MVC
  • wired through Spring 3.0
  • get detailed cost of a request by setting a special HTTP header
Quirks:
  • BLOB type
  • Text type for long Strings
  • white-listed classes
    • → find workarounds for blocked classes (e.g. Image services)
    • → some thirdparty libraries don't work
  • JSF2.0:
    • disable threading in web.xml
    • no JNDI: hack WebConfiguration class
    • server side state doesn't work → use client side state
    • RichFaces, IceFaces and Commons FileUpload use non-whitelisted classes → incompatible with GAE
  • GWT:
    • BLOB and Text are non-serializable
    • GWT <> Spring integration requires a special service class
  • SmartGWT:
    • requires 1000 static resources → closer to the max of 3000 of GAE → workaround with zip-file and cache, but increases cpu-cost

2009/11/11

Sound in Ubuntu 9.10 - Intel 82801G

The sound on my Medion MD96290 didn't work after upgrading to Ubuntu 9.10 (Karmik Koala). I experience this glitch on every Ubuntu upgrade - see my previous post.

My sound chip:
$ sudo lspci -v
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 02)
Subsystem: Wistron Corp. Device 4071
Flags: bus master, fast devsel, latency 0, IRQ 21
Memory at d8440000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2
Capabilities: [60] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable-
Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [130] Root Complex Link
Kernel driver in use: HDA Intel
Kernel modules: snd-hda-intel
...
Fixing is easy:
  • gksudo gedit /etc/modprobe.d/alsa-base
  • add one line to the end of the file:
    options snd-hda-intel model=6stack-dig
  • save and reboot
  • rightclick on the volume icon and select Sound Preferences
  • in the hardware tab, select a profile with "Surround" in it.
update 6/6/2010: this tip also works for ubuntu 10.4.

2009/11/08

Tweaking the Full-screen Mode of Firefox

I like to use Firefox in full-screen mode (hotkey: F11) on my netbook. Unfortunately, Firefox hides the tabs and navigation bar and only shows the bar when your mouse pointer is on the top of the screen.

Luckily, it's easy to modify this default setting:
  • enter 'about:config' in the URL-field
  • confirm that you really will be careful
  • set the 'browser.fullscreen.autohide' setting to false (double-click on the line)
This gives me a Chrome-like browser experience with Firefox on Ubuntu, while maximally using the small screen size.