OAUTH 2 with google-oauth-java-client


This post is a quick reference for running the dailymotion-cmdline-sample, a Java console sample app for the google-oauth-java-client OAUTH 2 library.

The sample app performs authorization code grant specified in the OAUTH 2 RFC, and receives user’s authorization code via an embedded Jetty HTTP server. That code is then exchanged for an access token.

Clone the google-oauth-java-client repo at GitHub.

git clone git@github.com:google/google-oauth-java-client.git

Install Apache Maven if you don’t already have it. You’ll need Oracle’s JDK to use Maven and compile the sample app. On Mac OS X Maven can be installed using Homebrew.

brew install maven

Head into the cloned repo and compile.

cd google-oauth-java-client
mvn compile

Head into the sample app folder and execute the sample app.

cd samples/dailymotion-cmdline-sample
mvn -X exec:java -Dexec.mainClass="com.google.api.services.samples.dailymotion.cmdline.DailyMotionSample"

The first execution will fail. You’ll need to create a dailymotion.com developer account, which can be created at http://www.dailymotion.com/profile/developer. Create a new API key. The only value that really matters is the Callback URL which should be http://127.0.0.1:8080/Callback.

Enter your API credentials in rc/main/java/com/google/api/services/samples/dailymotion/cmdline/OAuth2ClientCredentials.java. Compile and exec again. The application should list your favorite videos if all goes well.

Tackling OAuth 2.0 in an Android app


This post shows how to perform OAuth 2.0 authorization in an Android app using WebView. It is meant for those who need to tackle OAuth 2.0 themselves, probably because their identity provider does not provide an Android library.

OAuth 2.0 defines a two-step process for obtaining an access token from an authorization server (aka identity service), that can subsequently be used to obtain resources from resource servers that trust the authorization server.

  1. Obtain authorization code from authorization server. This step is usually carried out once, within the browser, to mitigate the need for user credentials to be handled by clients. The user authenticates with the identity service, and authorizes requested scopes. The authorization service grants an authorization code as a result, and redirects the browser to a redirect URI specified by the client.
  2. Client uses the authorization code obtained through previous step, and performs a token request to authorization server with its own credentials such as client_id and client_secret. This step usually happens in a server application, but it’s done here on device. Some OAuth 2.0 providers will enable a simpler implicit grant flow, where step 1 above returns an access token, dispensing the need for step 2.

Here’s one way to carry out Step 1 in a WebView

// define REDIRECT_URI
final WebView webView = (WebView)findViewById(R.id.webView);
WebSettings webSettings = webView.getSettings();
webSettings.setJavaScriptEnabled(true);
WebViewClient webViewClient = new WebViewClient() {
    @Override
    public boolean shouldOverrideUrlLoading(WebView view, String url) {
        if (url.startsWith(REDIRECT_URI)) {
            Pattern p = Pattern.compile(".+code=(.+?(?=&|$))");
            Matcher m = p.matcher(url);
            if (m.matches()) {
                acquireAccessToken(m.group(1));
                // update UI
            }
            return true; // we've handled the url
        } else {
            return false;
        }
    }
};
webView.setWebViewClient(webViewClient);
// Prepare loginURL
webView.loadUrl(loginURL);

I intercept browser navigation using shouldOverrideUrlLoading. Upon detecting the redirect URI, I look for the authorization code, extract it using a simple regular expression, and initiate the procedure to obtain the access token, as described in step 2. It can be carried out using a simple REST request such as

    private void acquireAccessToken(String code) {
        // prepare url

        AsyncTask task = new AsyncTask<Object, Integer, String>() {
            @Override
            protected String doInBackground(Object[] urls) {
                return executeRequest((String) urls[0], "POST", "");
            }

            @Override
            protected void onPostExecute(String result) {
                try {
                    JSONObject json = new JSONObject(result);
                    String accessToken = (String)json.get("access_token");
                } catch(Exception ex) {
                    Log.e(TAG, "Request failed.", ex);
                }
            }
        };
        task.execute(url);
    }

To perform the HTTP POST request above, I use the HttpsURLConnection class, in the executeRequest convenience method, implemented as follows

    private String executeRequest(String url, String method, String content) {
        StringBuilder buffer = new StringBuilder();
        try {
            URL connUrl = new URL(url);

            HttpsURLConnection conn = (HttpsURLConnection)connUrl.openConnection();
            conn.setSSLSocketFactory(sslContext.getSocketFactory());

            if (content != null) {
                conn.setRequestMethod(method);
                conn.setRequestProperty("Content-Type", "application/json");
                conn.setRequestProperty("Content-Length", String.valueOf(content.length()));
                OutputStreamWriter writer = new OutputStreamWriter(conn.getOutputStream());
                for (int i = 0; i < content.length(); i++)
                    writer.write(content.charAt(i));
            }

            InputStreamReader reader = new InputStreamReader(conn.getInputStream());
            int c = reader.read();
            while (c != -1) {
                buffer.append((char)c);
                c = reader.read();
            }

            conn.disconnect();
        } catch (Exception ex) {
            Log.e(TAG, "Request failed.", ex);
        }
        return buffer.toString();
    }

Since my authorization server uses SSL certificates with custom CAs, I have need for a customized SSLContext that can perform SSL handshake using the custom CAs (certificate pinning). This is how sslContext above may be initialized

        // Create a KeyStore containing our trusted CAs,
        // see http://developer.android.com/training/articles/security-ssl.html
        String keyStoreType = KeyStore.getDefaultType();
        KeyStore keyStore = KeyStore.getInstance(keyStoreType);
        keyStore.load(null, null);
        CertificateFactory cf = CertificateFactory.getInstance("X.509");
        Certificate ca = cf.generateCertificate(getResources().openRawResource(R.raw.cert_1));
        keyStore.setCertificateEntry("ca1", ca);
        ca = cf.generateCertificate(getResources().openRawResource(R.raw.cert_2));
        keyStore.setCertificateEntry("ca2", ca);

        // Create a TrustManager that trusts the CAs in our KeyStore
        String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
        TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
        tmf.init(keyStore);

        TrustManager[] trustManagers = tmf.getTrustManagers();
        sslContext = SSLContext.getInstance("TLS");
        sslContext.init(null, trustManagers, null);

Certificates are packaged as raw resources under res folder. Android has special naming restrictions for raw resource file names; it only allows lower-case letters and underscores. Binary DER or corresponding textual PEM certificates work fine.

Hibernate 2


What to me seems a long while back, I wrote an article on Hibernate. Spring was just beginning to take root, and everyone was raving about POJOs. I am studying Spring and Hibernate, and thought posting this bit of historical trivia would be a good starting point. Don’t use it as is, or use it at your own peril.

The ideal persistence mechanism

Ideally, objects would not need to be persisted if memory was non-volatile. Creating objects and establishing associations would suffice. Thus, a system would be a soup of objects, each having to deal with their own responsibilities, and not having to bother about external persistence.

O/R mapping

The closest we get to our ideal persistence is by using an O/R mapping framework such as Hibernate. Typically, external persistence is done by saving object state to relational databases. Thus, after we create an object, we have to persist it. The following steps are typically performed when a user interacts with a web based data entry use case:

  • Create new objects (transient) or query existing objects (persistent)
  • Begin a transaction
  • Change object properties and associations
  • Save objects to persistent storage
  • Commit the transaction

A simple persistence framework using Hibernate

In our architecture, development is driven by use cases. The flow of events of one or more use cases are orchestrated by a controller object. Each method of a controller object may initiate a new, or participate in an existing, database transaction. The controller saves and retrieves persistent objects using a Hibernate utility class, which in turn directly uses the Hibernate Session object.

import java.io.Serializable;
import java.lang.reflect.Method;
import java.net.URL;
import java.util.Date;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import net.sf.cglib.proxy.Enhancer;
import net.sf.cglib.proxy.MethodInterceptor;
import net.sf.cglib.proxy.MethodProxy;
import net.sf.hibernate.CallbackException;
import net.sf.hibernate.HibernateException;
import net.sf.hibernate.Interceptor;
import net.sf.hibernate.LockMode;
import net.sf.hibernate.ObjectNotFoundException;
import net.sf.hibernate.ReplicationMode;
import net.sf.hibernate.Session;
import net.sf.hibernate.SessionFactory;
import net.sf.hibernate.Transaction;
import net.sf.hibernate.cfg.Configuration;
import net.sf.hibernate.metadata.ClassMetadata;
import net.sf.hibernate.type.Type;

/**
 * This utility class provides a simple framework for using Hibernate
 * by implementing some <a href="http://www.hibernate.org/40.html">well known</a>
 * design patterns or by being useful with some of these patterns:
 *  - Thread Local session
 *  - AspectJ Hibernate aspect
 *  - Root Persistent Class
 *
 * In conjunctions with CGLIB and the TransactionConfiguration class it allows
 * any class to be extended so that methods of it's instances are wrapped with
 * appropriate begin and commit transaction invocations. The transaction
 * requirement can specified in an external xml configuration file.
 *
 * @see net.sf.hibernate.Session
 * @see net.sf.hibernate.SessionFactory
 */
public final class HibernateUtil implements MethodInterceptor, Interceptor {
// methods in various code snippets below
}

Persistent Objects

Persistent objects inherit from a common root class called PersistentObject as shown below.

/**
 * Super-class of all basic / entity classes.
 */
public abstract class PersistentObject implements Serializable {
    // flag indicating that object is persistent / saved
    private boolean _saved = false; 

    /**
     * This method is called when the object is saved to persistent
     * storage.
     */
    public void onSave() {
        _saved = true;
    }

    /**
     * This method is called when the object is loaded from persistent
     * storage.
     */
    public void onLoad() {
        _saved = true;
    }

    /**
     * This method is used to discover the saved status of the object.
     *
     * @return True if the object is exists in persistent storage
     */
    public boolean isSaved() {
        return _saved;
    }
}

Hibernate Utility needs the isSaved() method to hint a hibernate session on whether an object is persistent. This is required in the special case where the child object has the primary key of the parent object and not a direct reference to it. This is how the Hibernate utility, which needs to implement the Hibernate Interceptor interface, makes use of this method.

    /**
     * If the specified object is an instance of PersistentObject
     * sets the state of the object to saved.
     * @see net.sf.hibernate.Interceptor#onLoad(java.lang.Object,
     * java.io.Serializable, java.lang.Object[], java.lang.String[],
     * net.sf.hibernate.type.Type[])
     */
    public boolean onLoad(Object object, Serializable id, Object[] state,
            String[] propertyNames, Type[] types)
            throws CallbackException {

        if (object instanceof PersistentObject) {
            ( (PersistentObject) object ).onLoad();
        }
        return false; // we did not modify state
    }

    /**
     * If the specified object is an instance of AuditedPersistentObject
     * sets the dateLastUpdated attribute.
     */
    private boolean setDateLastUpdated(Object object, Object[] state,
            String[] propertyNames) {
        if (object instanceof AuditedPersistentObject) {
            for (int i = 0; i < propertyNames.length; i++) {
                if ("dateLastUpdated".equals(propertyNames[i])) {
                    state[i] = new Date();
                    return true;
                }
            }
        }
        return false; // we did not modify state
    }

    /**
     * @see net.sf.hibernate.Interceptor#onFlushDirty(java.lang.Object,
     * java.io.Serializable, java.lang.Object[], java.lang.Object[],
     * java.lang.String[], net.sf.hibernate.type.Type[])
     */
    public boolean onFlushDirty(Object object, Serializable id,
            Object[] currentState, Object[] previousState,
            String[] propertyNames, Type[] types)
            throws CallbackException {

        return setDateLastUpdated(object, currentState, propertyNames);
    }

    /**
     * @see net.sf.hibernate.Interceptor#onSave(java.lang.Object,
     * java.io.Serializable, java.lang.Object[], java.lang.String[],
     * net.sf.hibernate.type.Type[])
     */
    public boolean onSave(Object object, Serializable id, Object[] state,
            String[] propertyNames, Type[] types)
            throws CallbackException {

        boolean result = setDateLastUpdated(object, state, propertyNames);

        if (object instanceof PersistentObject) {
            ( (PersistentObject) object ).onSave();
        }

        return result;
    }

    /** (non-Javadoc)
     * @see net.sf.hibernate.Interceptor#onDelete(java.lang.Object,
     * java.io.Serializable, java.lang.Object[], java.lang.String[],
     * net.sf.hibernate.type.Type[])
     */
    public void onDelete(Object object, Serializable id, Object[] state,
            String[] propertyNames, Type[] types)
            throws CallbackException {
    }

    /** (non-Javadoc)
     * @see net.sf.hibernate.Interceptor#preFlush(java.util.Iterator)
     */
    public void preFlush(Iterator objects) throws CallbackException {

    }

    /** (non-Javadoc)
     * @see net.sf.hibernate.Interceptor#postFlush(java.util.Iterator)
     */
    public void postFlush(Iterator objects) throws CallbackException {

    }

    /**
     * If the object is a PersistentObject returns its saved status.
     * @return true if the PersistentObject already exists in the database.
     * @see net.sf.hibernate.Interceptor#isUnsaved(java.lang.Object)
     */
    public Boolean isUnsaved(Object object) {
        if (object instanceof PersistentObject) {
            return new Boolean( !( (PersistentObject) object ).isSaved() );
        } else {
            return null; // let Hibernate assume default behavior
        }
    }

    /**
     * @see net.sf.hibernate.Interceptor#findDirty(java.lang.Object,
     * java.io.Serializable, java.lang.Object[], java.lang.Object[],
     * java.lang.String[], net.sf.hibernate.type.Type[])
     */
    public int[] findDirty(Object object, Serializable id,
            Object[] currentState, Object[] previousState,
            String[] propertyNames, Type[] types) {
        return null; // let Hibernate assume default behavior
    }

    /**
     * @see net.sf.hibernate.Interceptor#instantiate(java.lang.Class,
     * java.io.Serializable)
     */
    public Object instantiate(Class classObj, Serializable id)
            throws CallbackException {
        Object object = null; // let Hibernate assume default behavior
        return object;
    }

Persistent objects that have composite keys must also override the equals and hashCode methods. The equals method must return true if the corresponding primary key attributes of two objects of the same class pass the equals test.

Intercepting objects about to be saved

The onSave method shown in the previous code listing also forms the basis for making changes to objects before they are saved. In our case we need to update the time the object was lasted updated. We have done that by creating a base class called AuditedPersistentObject, for all objects that need this new attribute. The implementation of this class is shown below. In the onSave method of Hibernate Utility we detect whether the object is an instance of this class and set the dateLastUpdated attribute.

/**
 * This class and it's sub-classes represent persistent objects with
 * audit information such as the date/hour the object was last updated.
 */
public abstract class AuditedPersistentObject extends PersistentObject {
    protected Date dateLastUpdated = null;

    /**
     * Recover the date/hour of last update.
     * @return Date/Hour
     */
    public Date getDateLastUpdated() {
        return dateLastUpdated;
    }

    /**
     * Set the date/hour of last update.
     * @param dateLastUpdated Date/hour
     */
    public void setDateLastUpdated(Date dateLastUpdated) {
        this.dateLastUpdated = dateLastUpdated;
    }
}

Declarative transactions

As mentioned previously, the methods in the controller can initiate transactions. To do this declaratively, we create every controller object using the following method of Hibernate Utility.

    /***
     * This method returns a CGLIB extended object of the
     * specified class. Each method call on the extended object
     * results in a callback to the intercept method.
     * @param c The class to extend.
     * @return A new extended instance of the specified class
     */
    public Object extendClass(Class c) {
        Object newObject = null;
        newObject = Enhancer.create(c, null, this);
        return newObject;
    }

Now, when any method of the controller is called, the intercept method of Hibernate Utility is called automatically. For this, Hibernate Utility must implement the net.sf.cglib.proxy.MethodInterceptor interface of CGLIB. The intercept method itself, and all the other methods it calls, and the attributes they use, are shown below.

    // Private attributes

    // Transaction configuration
    private TransactionConfiguration transactions;
    // Hibernate Session Factory
    private SessionFactory sessionFactory = null;
    // Number of transactions in progress
    private int transactionCount = 0;             

    // Private Thread Local attributes

    // Session local to current thread
    private ThreadLocal session = new ThreadLocal();
    // Active transaction local to current thread
    private ThreadLocal transaction = new ThreadLocal();
    // Depth of the active database transaction
    private ThreadLocal transactionDepth = new ThreadLocal(); 

    /**
     * Wraps the call of an intercepted method with begin and
     * commit transactions if it supports transactions.
     * @see net.sf.cglib.proxy.MethodInterceptor#intercept(java.lang.Object, java.lang.reflect.Method, java.lang.Object[], net.sf.cglib.proxy.MethodProxy)
     */
    public Object intercept(Object obj, Method method, Object[] args,
            MethodProxy proxy) throws Throwable {

            boolean useTransaction;
            Object retValFromSuper = null;

            useTransaction = transactions.isTransaction(method);
            try {
                if (useTransaction) beginTransaction();

                retValFromSuper = proxy.invokeSuper(obj, args);

                if (useTransaction) commitTransaction();

            }catch (Throwable e) {
                if (useTransaction) rollbackTransaction();
                throw e;
            }
            return retValFromSuper;
    }

    /**
     * Initiates a new database transaction and associates the
     * Hibernate Transaction instance with the current thread only on the
     * first call to this method. Successive calls to this method will
     * not initiate any transactions and result in incrementing a
     * stack counter. This stack counter is later used to determine
     * the outer-most commit so that the transaction can be committed.
     *
     * @throws DataPersistenceException If the beginTransaction method
     * of hibernate returns a HibernateException.
     */
    public void beginTransaction() throws DataPersistenceException {
        Integer depth = (Integer) transactionDepth.get();
        if (depth == null) {
            // first call to begin transaction in this thread
            Session s = (Session) session.get();
            if (s == null) {
                currentSession();
            }
            s = (Session) session.get();
            try {
                Transaction t = (Transaction) s.beginTransaction();
                transaction.set(t);
                transactionDepth.set(new Integer(0));
                incrementTransactionCount();
            } catch (HibernateException e) {
                throw new DataPersistenceException(e);
            }
        } else {
            // increment depth
            transactionDepth.set(new Integer(depth.intValue() + 1));
        }
    }

    /**
     * Commits a database transaction and dissociates the
     * Hibernate Transaction instance associated with the
     * current thread only if this is the outer-most commit request.
     *
     * @throws DataPersistenceException If the commit method
     * of hibernate returns a HibernateException or there is
     * no existing transaction to be committed.
     */
    public void commitTransaction() throws DataPersistenceException {
        Integer depth = (Integer) transactionDepth.get();
        if (depth == null) {
            // no begin transaction issued
        } else {
            if (depth.intValue() == 0) {
                // reached the outer most call to begin transaction
                Transaction t = (Transaction) transaction.get();
                if (t == null) {
                    // no begin transaction issued or rolled back
                    throw new DataPersistenceException("Assertion Failed: A transaction should be present but was not found.", null);
                } else {
                    try {
                        t.commit();
                    } catch (HibernateException e) {
                        throw new DataPersistenceException(e);
                    } finally {
                        transaction.set(null);
                        transactionDepth.set(null);
                        decrementTransactionCount();
                    }
                }
            } else {
                // decrement depth, ignore commit request
                transactionDepth.set(new Integer(depth.intValue() - 1));
            }
        }
    }

    /**
     * Rolls back a database transaction and dissociates the
     * Hibernate Transaction instance associated with the
     * current thread.
     *
     * @throws DataPersistenceException If the rollback method
     * of hibernate returns a HibernateException or there is
     * no existing transaction to be rolled back.
     */
    public void rollbackTransaction() throws DataPersistenceException {
        Integer depth = (Integer) transactionDepth.get();
        if (depth == null) {
            // no begin transaction issued
        } else {
            Transaction t = (Transaction) transaction.get();
            if (t == null) {
                // no transaction
                throw new DataPersistenceException("Assertion Failed: A transaction should be present but was not found.", null);
            } else {
                try {
                    t.rollback();
                } catch (HibernateException e) {
                    throw new DataPersistenceException(e);
                } finally {
                    transaction.set(null);
                    decrementTransactionCount();
                    closeSession();
                }
            }
            // decrement depth
            transactionDepth.set(null);
        }
    }

    /**
     * Recovers a Hibernate session associated with the current thread.
     * Opens a new session if one doesn't already exist and associates
     * it to the thread. As long as you don't call the closeSession
     * or the rollbackTransaction methods the session remains
     * open and associated with the current thread.
     *
     * @return A Hibernate session
     *
     * @throws DataPersistenceException If the openSession method
     * of the Hibernate session factory fails.
     */
    public Session currentSession() throws DataPersistenceException {

        Session s = (Session) session.get();
        // Open a new Session, if this Thread has none yet
        if (s == null) {
            try {
                s = sessionFactory.openSession(this);
                session.set(s);
            } catch (HibernateException e) {
                e.printStackTrace();
                throw new DataPersistenceException(e);
            }
        }
        return s;
    }

    /**
     * Closes and dissociates the Hibernate session associated with
     * the current thread.
     * @throws DataPersistenceException If the close method of the
     * Hibernate session fails.
     */
    public void closeSession() throws DataPersistenceException {
        Integer depth = (Integer) transactionDepth.get();
        if (depth != null) {
            // We are within a transaction, ignore close request
        } else {
            try {
                Session s = (Session) session.get();
                session.set(null);
                if (s != null) {
                    s.close();
                }
            } catch (HibernateException e) {
                e.printStackTrace();
                throw new DataPersistenceException(e);
            }
        }
    }

The TransactionConfiguration class, used in the intercept method above, parses an XML file that specifies which methods of any class need database transactions. The isTransaction methods of this class return true for methods that need database transactions. See the code listing below for implementation of these methods.

import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Method;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;

import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

/**
 * Utility class used to represent transaction configuration.
 */
public class TransactionConfiguration extends DefaultHandler {
    private static String className;
    Map classes = new HashMap();
    private static TransactionConfiguration instance;

    public static TransactionConfiguration getInstance() {
        if (instance == null) {
            instance = new TransactionConfiguration();
        }
        return instance;
    }

    /**
     * The method returns true if the method of the specified class is
     * transactional.
     *
     * @param className
     *            Fully qualified name of the class
     * @param methodName
     *            Method name
     * @return true if the method is specified as transactional.
     */
    public boolean isTransaction(String className, String methodName) {
        return hasMethod(className, methodName);
    }

    /**
     * The method returns true if the given Method instance is transactional.
     *
     * @param method
     *            Method instance
     * @return true if the Method instance is specified as transactional.
     */
    public boolean isTransaction(Method method) {
        boolean val;
        val = isTransaction(method.getDeclaringClass().getName(), method
                .getName());
        if (val == false) {
            // let us check if super classes / interfaces are transactions
            for (int i = 0; i < method.getDeclaringClass().getInterfaces().length; i++) {
                val = isTransaction(
                        method.getDeclaringClass().getInterfaces()[i].getName(),
                        method.getName());
                if (val == true)
                    break;
            }
        }
        return val;
    }

Hibernate Utility encapsulates the most commonly used Hibernate Session and SessionFactory methods, so you don’t need to obtain a Session by calling the currentSession method and call the methods of the Session.

Declarative transactions using AspectJ

The following aspect can be used to extend desired methods to add declarative transactions.

public abstract aspect TransactionAspect {

    public abstract pointcut transactedMethods();

    Object around() throws SystemRuntimeException : transactedMethods() {
        boolean useTransaction = false;
        Object retValFromSuper = null;

        String methodName = thisJoinPoint.getSignature().getName();
        String className = thisJoinPoint.getSignature()
                .getDeclaringType().getName();

        // The TransactionConfiguration determines if a method
        // should be made transactional or not.
        useTransaction = TransactionConfiguration.getInstance()
                .isTransaction(className, methodName);

        if ( useTransaction ) {
            System.out.println("TransactionAspect: Method '"
                    + className + "." + methodName
                    + "' is transactional.");
            try {
                // begin the transaction.
                HibernateUtil.getInstance().beginTransaction();

                // "proceed()" allows the method to continue as it normally
                // would. The return value is saved.
                retValFromSuper = proceed();

                HibernateUtil.getInstance().commitTransaction();

                return retValFromSuper;
            } catch (RuntimeException e) {
                System.out.println("TransactionAspect: Method '"
                        + className + "." + methodName
                        + "' threw an exception.");
                HibernateUtil.getInstance().rollbackTransaction();
                throw e;
            }
        }

        // This means the method was not transactional.
        System.out.println("TransactionAspect: Method '"
                + className + "." + methodName
                + "' is NOT transactional.");

        // Return the result of the action method as normal.
        return proceed();
    }
}

A concrete aspect must extend the abstract aspect above and implement the abstract join point transactedMethods as follows.

public aspect AppTransactionAspect extends TransactionAspect {
    public pointcut transactedMethods() :
        (target(PersistentObject)
        || target(SomeOtherClass))
        && execution(public * *( .. ))
        && !execution(* PersistentObject.*( .. ))
        && !execution(public * find*( .. ))
        && !execution(public * get*( ))
        && !execution(public * set*( .. ));
}

The transactedMethods point cut implementation above executes the around advice implemented in the abstract aspect, for all methods of classes that are sub classes of SomeOtherClass and PersistentObject, except the following:

  • All methods of the class PersistentObject itself.
  • All methods beginning with set.
  • All methods beginning with get.
  • All methods beginning with find.

Concurrency

While an object is being prepared for updating, the persistent representation of the object in the database may be changed by another transaction. Hibernate automatically adds version number checking to all UPDATE commands if the class mapping has a version element, as shown below.

<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD//EN"     "http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd">

<hibernate-mapping package="test.app">

    <class name="Customer" table="CUSTOMER">

        <id name="id">
            <column name="id" sql-type="integer" not-null="true"/>
            <generator class="native"/>
        </id>

        <version name="version" column="version" type="integer" />

        <property name="name">
            <column name="name" sql-type="varchar(20)" not-null="true"/>
        </property>

        <bag name="orders" table="ORDER" cascade="all">
            <key column="customer_id"/>
            <one-to-many class="Order"/>
        </bag>

    </class>

</hibernate-mapping>

The application is responsible for checking that the object information it has is not stale, before updating the object.

Closing Hibernate sessions

Closing hibernate sessions is a must to liberate JVM heap and avoid an OutOfMemoryError, and more importantly, close database connections. In web applications one useful strategy is to use a servlet filter to close a Hibernate session as shown below.

public class CloseDataSession implements Filter {

    private static ThreadLocal stackCounter = new ThreadLocal();
        // for closing sessions only when exiting from a client request

    public void destroy() {
    }

    public void doFilter(ServletRequest req, ServletResponse resp,
            FilterChain chain) throws IOException, ServletException {
        try {
            increment();
            chain.doFilter(req, resp);
        } finally {
            if (decrement() <= 0) {
                HibernateUtil.getInstance().closeSession();
            }
        }
    }

    public void init(FilterConfig config) throws ServletException {
    }

    private int increment() {
        int count = 0;
        if (stackCounter.get() == null) {
            stackCounter.set(new Integer(++count));
        } else {
            count = ((Integer) stackCounter.get()).intValue();
            stackCounter.set(new Integer(++count));
        }
        return count;
    }

    private int decrement() {
        int count = 0;
        if (stackCounter.get() == null) {
            stackCounter.set(new Integer(count));
        } else {
            count = ((Integer) stackCounter.get()).intValue();
            stackCounter.set(new Integer(--count));
        }
        return count;
    }
}

The filter CloseDataSession is mapped to all server side resources that directly or indirectly access any persistent objects. This mapping is done in the web module descriptor (web.xml) of a J2EE web application. Some application servers execute a filter several times within the same web request, for example, every time when the request is forwarded. To close the Hibernate session only once, we use a simple filter execution counter associated with the current thread. Only when the counter goes to zero do we close the session.

For those adopting Hibernate today

I recommend using Hibernate with JPA, or Spring, and using annotations instead of manually coding configuration files.

Microsoft.NET vs Java on portability


Portability is a much sought after objective of high-level (as opposed to machine-level) languages. Portability reached a water-shed moment with Java, which defined an intermediate language (IL) that programs can be compiled to (called byte code) and a Virtual Machine that executes the IL. By abstracting the machine-level aspects like memory, processor and operating system, Java brought portability to the mainstream.

Earlier attempts at portability in languages like C and C++ were based on being able to compile the same high-level code to different machine-level code, using different compiler implementations for each machine architecture, and portable run-time libraries.

IL code is translated to machine code at run-time. That requires a capable machine. IL code allows access to machine-specific features (like graphics processors) using non-portable extension mechanisms. Java has been relegated to the background in mobile devices due to these factors. Microsoft.NET has however managed to remain relevant by embracing machine-specific extensions.

Microsoft.NET began as a Windows only run-time. Microsoft had the foresight to release its specifications as open standards. Now, due to Mono, .NET is available on a number of different machine architectures including ARM (most mobile devices) and x86 (most PC devices). It has clearly overtaken Java in that sense. Java is still relevant in the mobile devices space due to Android and that too only as a high-level language. Java is compiled to run on Android’s own VM called Dalvik.

Java is however the quintessential portable language on desktop PCs and servers. It will remain relevant there for a long time due to strong backing from the likes of Apache, Google, IBM, Oracle, SAP, and due to the large number of applications and frameworks written in it. Developer usage trends show Java as stagnant (surprising considering Android) but C# and Visual Basic .NET on a steady rise.

I suspect however that JavaScript will overtake the incumbents with respect to portability. Commenting on that though will be a topic of some future post.

Use Tile Studio to construct tiled maps


Tile Studio is a free and open source utility for making tile based games. The key features of Tile Studio are:

  1. Manage several tile sets in a single project. Multiple maps can be created with each tile set.
  2. Bitmap editor for editing tiles.
  3. Specify bounds for collision detection. The left, right, top, bottom and diagonals of a cell can be marked. The bounds can be exported, along with other map data, to be used in games or applications.
  4. Specify a one byte map code for any cell in the map. This can be used to tag each cell with a special value which can indicate, among other things, enemy positions in games.
  5. Create and use animated tiles.
  6. Export tile sets to image files in bitmap or PNG format.
  7. Export map data to binary files.
  8. Generate game code using templates.

Exporting the tile map from Tile Studio

Tile Studio provides means to export map data to files by means of templates [5]. These templates can produce complete source code or just simple text or binary files. In the example template (see export.tsd) below we do two things:

  1. Export the tile set images to PNG files
  2. Produce a binary file containing a sequence of 32 bit integer values, by reading the map data row by row. Each 32 bit integer is a result of the concatenation of the tile index, the bound values and the map code. We assume that we will not need more than 2^15 – 1 tiles in each tile set (32767 tiles is quite a large number of tiles!).

The images created by Tile Studio can and should be optimized further using image manipulation tools to optimize the color depth (palette size) and so on. Programs such as pngout, which optimize the image size further by using better compression, can also be used.

export.tsd

;
; Create the tile set bitmap. The width by default has been limited to 160
; pixels. Adjust this as required.
;
#tileset
#tilebitmap <TileSetIdentifier>.png 160
#end tilebitmap
#end tileset

;
; Generate binary files containing map data. The file format is an
; ordinary sequence of 32 bit values.
;
#tileset
#map
; concatenate tile number, bounds data and map value into a 32 bit value
#binfile <MapIdentifier>.bin 32
#mapdata
<TileNumber:"16"><BoundMapValue:"16">
#end mapdata
#end binfile
#end map
#end tileset

Compressing map data

The binary file exported from Tile Studio in the previous section can be optimized further. This may need to be done to keep the resources of J2ME games and applications to the smallest size possible. Unfortunately the template language provided by the tool itself is quite limited so we have built a simple java utility for this task. The source code of the utility is shown below (see TileStudioUtil.java). Have a look at the javadoc of the writeSparseMatrix method, to get a notion of the format of the output file created by this utility. The utility can be executed in the following manner to produce the binary file (diags.map) used in the example in the next section:

java TileStudioUtil diags.bin diags.map 20 20 1

TileStudioUtil.java

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Enumeration;
import java.util.Hashtable;
import java.util.Iterator;
import java.util.Vector;

public class TileStudioUtil {

    private static final int INFILE_INDEX = 0;

    private static final int OUTFILE_INDEX = 1;

    private static final int HEIGHT_INDEX = 2;

    private static final int WIDTH_INDEX = 3;

    private static final int INDEXSIZE_INDEX = 4;

    private static final int NUM_ARGUMENTS = 5;

    /**
     * Main method.
     *
     * @param args
     *            Arguments to the main method
     */
    public static void main(String[] args) {
        try {
            System.out.println(args.length + " arguments received.");
            if (args.length < NUM_ARGUMENTS) {
                System.out
                        .print("Command line options: ");
                System.out
                        .println("infile outfile height width indexSize");
                System.out
                        .print("infile\t\t");
                System.out
                        .println("The binary file exported from Tile Studio");
                System.out
                        .print("outfile\t\t");
                System.out
                        .println("The binary file exported by this program");
                System.out.println("height\t\tThe height of the matrix");
                System.out.println("width\t\tThe width of the matrix");
                System.out
                        .print("indexSize\tThe storage size of the matrix");
                System.out
                        .println("index in bytes");
                System.exit(0);
            }

            int indexSize = Integer.parseInt(args[INDEXSIZE_INDEX]);
            int width = Integer.parseInt(args[WIDTH_INDEX]);
            int height = Integer.parseInt(args[HEIGHT_INDEX]);
            int a[][] = new int[height][width];
            InputStream in;
            OutputStream out;

            System.out.println("Reading file " + args[INFILE_INDEX]);
            in = new FileInputStream(args[INFILE_INDEX]);
            readMatrix(in, a, height, width);
            in.close();

            System.out.println("Writing file " + args[OUTFILE_INDEX]);
            out = new FileOutputStream(args[OUTFILE_INDEX]);
            writeSparseMatrix(out, a, height, width, indexSize);
            out.close();

            System.out.println("Testing file written");
            int b[][] = new int[height][width];
            in = new FileInputStream(args[OUTFILE_INDEX]);
            readSparseMatrix(in, b, height, width, indexSize);
            in.close();

            // Compare
            for (int i = 0; i < height; i++) {
                for (int j = 0; j < width; j++) {
                    if (a[i][j] != b[i][j]) {
                        System.out.println("Different data at row " + i
                                + ", column " + j);
                    }
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
            System.exit(-1);
        }
    }

    /**
     * Read an int value from an array of bytes.
     *
     * @param data
     *            an array of bytes, only the first four elements are read
     * @param littleEndian
     *            Pass true if the least signficant byte is stored in the first
     *            element of the byte array
     *
     * @return int value
     * @throws IllegalArgumentException
     */
    public static int toInt(byte[] data, boolean littleEndian)
            throws IllegalArgumentException {
        int val = 0;

        if (littleEndian) {
            for (int i = 0; i < data.length; i++) {
                val += (data[i] & 0xFF) << (i * 8);
            }
        } else { // big endian
            for (int i = 0; i < data.length; i++) {
                val += (data[i] & 0xFF) << ((data.length - i - 1) * 8);
            }
        }

        return val;
    }

    /**
     * Write an int to an array of bytes.
     *
     * @param i
     *            The int value to write
     * @param data
     *            The byte array to write to
     * @param littleEndian
     *            If set to true the least significant byte is stored first
     * @throws IllegalArgumentException
     */
    public static void toByteArray(int i, byte[] data, boolean littleEndian)
            throws IllegalArgumentException {
        if (data.length < 4) {
            throw new IllegalArgumentException("Need at least 4 bytes");
        }

        if (littleEndian) {
            data[3] = (byte) ((i >>> 24) & 0xFF);
            data[2] = (byte) ((i >>> 16) & 0xFF);
            data[1] = (byte) ((i >>> 8) & 0xFF);
            data[0] = (byte) (i & 0xFF);
        } else {
            data[0] = (byte) ((i >>> 24) & 0xFF);
            data[1] = (byte) ((i >>> 16) & 0xFF);
            data[2] = (byte) ((i >>> 8) & 0xFF);
            data[3] = (byte) (i & 0xFF);
        }
    }

    /**
     * Reads an int matrix in binary format into an array. The file structure is
     * a simple sequence of int values. Each int is little endian. All int
     * values in the first row are stored sequentially followed by all int
     * values in the second row and so on.
     *
     *
     * @param file
     *            Binary file
     * @param array
     *            Data array
     * @param width
     *            The array width
     * @param height
     *            The array height
     * @throws IOException
     */
    public static void readMatrix(InputStream in, int array[][], int rows,
            int columns) throws IOException {
        byte[] data = new byte[4];
        int n, r, c;

        r = 0;
        while (r < rows) {
            c = 0;
            while (c < columns) {
                n = in.read(data);
                if (n == -1)
                    throw new IOException("Error reading map data at row " + r
                            + " and column " + c);
                array[r][c] = toInt(data, true);
                c++;
            }
            r++;
        }

    }

    /**
     * Writes a sparse matrix to a binary format. The binary format is a simple
     * structure as represented in the BNF notation below:
     *
     * <pre>
     *       number_of_types (type number_of_positions (positions)+)+
     *       number_of_types ::= int value
     *       type ::= int value
     *       number_of_positions ::= int value
     *       positions ::= row column
     *       row ::= int value stored in indexSize bytes
     *       column ::= int value stored in indexSize bytes
     * </pre>
     *
     * A type is any unique value, other than zero, that occurs at least once
     * in the array.
     *
     * @param out
     *            Binary output stream
     * @param array
     *            Data array
     * @param rows
     *            The number of rows in the matrix
     * @param columns
     *            The number of columns in the matrix
     * @param indexSize
     *            Size in bytes of row or column index
     *
     * @throws IOException
     */
    public static void writeSparseMatrix(OutputStream out, int array[][],
            int rows, int columns, int indexSize) throws IOException {
        byte[] data; // a byte array to store int
        Hashtable table = new Hashtable(); // a hashtable to store type
                                            // locations
        int r, c;
        Integer type; // A type (unique value) in the array
        Vector positions; // a vector to store positions of a type
        Enumeration keys;
        Iterator p;

        // Analyze data and store it in an optimized format in memory
        for (r = 0; r < rows; r++) {
            for (c = 0; c < columns; c++) {
                if (array[r][c] == 0)
                    continue;

                type = new Integer(array[r][c]);
                if (table.containsKey(type)) {
                    positions = ((Vector) table.get(type));
                } else {
                    positions = new Vector();
                    table.put(type, positions);
                }
                data = new byte[4];
                toByteArray(r, data, true);
                positions.add(data);
                data = new byte[4];
                toByteArray(c, data, true);
                positions.add(data);
            }
        }

        // Dump sparse matrix to binary file

        data = new byte[4];

        // write number of types
        toByteArray(table.size(), data, true);
        out.write(data);

        keys = table.keys();
        while (keys.hasMoreElements()) {
            type = (Integer) keys.nextElement();

            // write type
            toByteArray(type.intValue(), data, true);
            out.write(data);

            positions = (Vector) table.get(type);

            // write number of positions the type occurs
            toByteArray(positions.size() / 2, data, true);
            out.write(data);

            // write positions of type
            p = positions.iterator();
            while (p.hasNext()) {
                out.write((byte[]) p.next(), 0, indexSize);
            }
        }

    }

    /**
     * Read a sparse matrix from a binary format.
     *
     * @param file
     *            File with binary data
     * @param array
     *            Array to write to
     * @param rows
     *            Rows in the matrix
     * @param columns
     *            Columns in the matrix
     * @param indexSize
     *            The storage size of the matrix index in bytes
     * @throws IOException
     *
     * @see #writeSparseMatrix(OutputStream, int[][], int, int, int)
     */
    public static void readSparseMatrix(InputStream in, int array[][],
            int rows, int columns, int indexSize) throws IOException {
        byte[] data = new byte[4]; // a byte array to store int
        byte[] pos = new byte[indexSize]; // a byte array to store position
        int bytesRead, numTypes, t, type, numPositions, n, r, c;

        // Read sparse matrix from binary file

        // read number of types
        bytesRead = in.read(data);
        if (bytesRead == -1)
            throw new IOException("Error reading number of types");
        numTypes = toInt(data, true);

        for (t = 0; t < numTypes; t++) {
            // read type
            bytesRead = in.read(data);
            if (bytesRead == -1)
                throw new IOException("Error reading type");
            type = toInt(data, true);

            // read number of positions
            bytesRead = in.read(data);
            if (bytesRead == -1)
                throw new IOException("Error reading number of positions");
            numPositions = toInt(data, true);

            for (n = 0; n < numPositions; n++) {
                // read row
                bytesRead = in.read(pos);
                if (bytesRead == -1)
                    throw new IOException("Error reading row");
                r = toInt(pos, true);
                // read column
                bytesRead = in.read(pos);
                if (bytesRead == -1)
                    throw new IOException("Error reading column");
                c = toInt(pos, true);

                array[r][c] = type;
            }

        }

    }
}

Tiled is another neat editor for creating tiled maps.

Web chat using Strophe and Apache Vysper


I was told about Apache Vysper by a colleague at work and thought I’d give it a try. It is very much in development and lacks documentation, but it is very easy to get started. It does not have clustering support though.

In this post I’ll replace Openfire in the web chat setup I did in a previous post. You’ll need a more feature-rich client than trophyim to add users to a roster (contact list) and such. I use Jitsi.

Setup Vysper

Grab a copy of Vysper version 0.7. Extract it to any folder. You’ll need a JVM to run the server. Before doing that, let us configure a few things by editing the config file spring-config.xml in the config folder.

Locate the following lines and change the domain name (your machine name should do fine or localhost):

<!-- TODO change domain name to your domain -->
<constructor-arg value="localhost"/>

Locate the following lines and change the admin user domain and password. Add more users if you wish.

<bean id="addUsers" class="org.apache.vysper.spring.AddUserHelper">
    <constructor-arg index="0">
        <map>
            <entry key="admin@localhost" value="passw0rd" />
            <entry key="friend@localhost" value="passw0rd" />

Uncomment the following line (thus enabling the BOSH):

<ref bean="boshEndpoint"/>

Uncomment the BOSH endpoint configuration:

<bean id="boshEndpoint" class="org.apache.vysper.xmpp.extension.xep0124.BoshEndpoint">
    <property name="accessControlAllowOrigin">
        <list><value>*</value></list>
    </property>
    <property name="port" value="8080" />
    <property name="contextPath" value="/bosh" />
</bean>

Start Vysper

Head to the command line and execute run.bat or run.sh from the bin folder. If all goes well you can access http://localhost:8080/bosh/ from the browser.

Execute trophyim

You’ll need to change the variable TROPHYIM_BOSH_SERVICE in the trophyim.js file, assign 'http://localhost:8080/bosh/'. Open index.html in the browser (using a file:// url), that should launch trophyim. You can use a client like Jitsi to setup contacts, and then test chatting with them using trophyim.

Note that we do not require an http server and proxy, as in the previous post, I suspect Vysper supports W3C CORS out of the box. That is good news indeed.

Web chat using Strophe and Openfire


XMPP is now widely used to implement messaging and presence services. Popular applications and services such as Google Voice, Google Talk, Jabber etc are based on it. Openfire is one really popular cross-platform Java server infrastructure for XMPP. On the client side there is an increasing trend towards applications done in HTML and JavaScript. There now exist some pure JavaScript client libraries such as Strophe that support this trend. These work with the HTTP binding provided by Openfire based on BOSH.

In this post I explain how I setup a simple chat application called trophyim to run with Openfire. I’ll use Apache httpd along the way to proxy access to the http binding service provided by Openfire, to get around cross-origin errors. Openfire should soon implement W3C CORS support so we don’t need to proxy HTTP requests.

Obtain and build Strophe

Download Strophe from github. Build it using make (I usually have a Linux VM around for such occasions).

Setup Openfire and Apache httpd

Download and setup Openfire (version 3.7.0 used here) and Apache httpd. Openfire provides a web browser based administration console at port 9090 (e.g. http://localhost:9090) that you can use to add users. The BOSH based http binding service runs at port 7070. A Strophe connection needs to be opened to url http://localhost:7070/http-bind/.

Setup trophyim

Obtain trophyim (version 0.3 used here) provide access to it through Apache httpd, I’ll assume that accessing http://localhost/trophyim.0.3/index.html from the browser runs the chat client. Copy strophe.js to a folder called strophejs within trophyim. You’ll need to change the variable TROPHYIM_BOSH_SERVICE in the trophyim.js file to http://localhost:7070/http-bind/.

Direct access to that URL from the chat client will result in a cross-origin request, this fails in most browsers. In Chrome you should see a message such as:

XMLHttpRequest cannot load http://localhost:7070/http-bind/.
Origin http://localhost is not allowed by Access-Control-Allow-Origin.

Change the TROPHYIM_BOSH_SERVICE variable to http://localhost/bosh. We’ll now configure Apache httpd to proxy that URL to http://localhost:7070/http-bind/.

Configure Apache httpd as a proxy

Edit conf/httpd.conf located in the Apache httpd installation folder. Uncomment the following lines (remove #):

LoadModule proxy_module modules/mod_proxy.so

LoadModule proxy_connect_module modules/mod_proxy_connect.so

LoadModule proxy_http_module modules/mod_proxy_http.so

Include conf/extra/httpd-vhosts.conf

Next, edit conf/extra/httpd-vhosts.conf. Remove any stray VirtualHost directives there and add:


    ProxyPass /bosh http://localhost:7070/http-bind/
    ProxyPassReverse /bosh http://localhost:7070/http-bind/

Restart Apache httpd and reload page http://localhost/trophyim.0.3/index.html. You should now have a running chat client. Log in. Run another instance of the chat client and log in with a different user. If both users are buddies you should be able to chat.

Happy chatting!