Scott Leberknight's Weblog

I've just published the second article of a two-part series in the December 2009 NFJS Magazine on Hibernate Performance Tuning. Here's the abstract:

Tuning performance in Hibernate applications is all about reducing the number of database queries or eliminating them entirely using caching. In the first article in this two part series, you saw how to tune object retrieval using eager fetching techniques to optimize queries and avoid lazy-loads. In this second and final article, I’ll show you how inheritance strategy affects performance, how to eliminate queries using the Hibernate second-level cache, and show some simple but effective tools you can use to monitor and profile your applications.

If you are using Hibernate and want to know more about how inheritance affects performance, how to use the second-level cache, and some simple monitoring and profiling techniques, check it out and let me know what you think. Note that NFJS Magazine does require a subscription.

I've just published an article in the November 2009 NFJS Magazine on Hibernate Performance Tuning. Here's the abstract:

Many developers treat Hibernate like a "black box" and assume it will simply "Do the Right Thing" when it comes to all things related to the underlying database. This is a faulty assumption because, while Hibernate is great at the mechanics of database interaction, it cannot and will likely not ever be able to figure out the specific details of your domain model and discern the most efficient and best performing data access strategies. In this first article of a two part series, I'll show you how to achieve better performance in your Hibernate applications by focusing on tuning object retrieval, which forms the basis of your "fetch plan" for finding and storing objects in the database.

If you are using Hibernate and want to know more about how to change how objects are fetched from the database, check it out and let me know what you think. Note that NFJS Magazine does require a subscription.

When you have collections of associated objects in domain objects, you generally want to specify some kind of default sort order. For example, suppose I have domain objects Timeline and Event:

@Entity
class Timeline {

    @Required 
    String description

    @OneToMany(mappedBy = "timeline")
    @javax.persistence.OrderBy("startYear, endYear")
    Set<Event> events
}

@Entity
class Event {

    @Required
    Integer startYear

    Integer endYear

    @Required
    String description

    @ManyToOne
    Timeline timeline
}

In the above example I've used the standard JPA (Java Persistence API) @OrderBy annotation which allows you to specify the order of a collection of objects via object properties, in this example a @OneToMany association . I'm ordering first by startYear in ascending order and then by endYear, also in ascending order. This is all well and good, but note that I've specified that only the start year is required. (The @Required annotation is a custom Hibernate Validator annotation which does exactly what you would expect.) How are the events ordered when you have several events that start in the same year but some of them have no end year? The answer is that it depends on how your database sorts null values by default. Under Oracle 10g nulls will come last. For example if two events both start in 2001 and one of them has no end year, here is how they are ordered:

2001 2002  Some event
2001 2003  Other event
2001       Event with no end year

What if you want to control how null values are ordered so they come first rather than last? In Hibernate there are several ways you could do this. First, you could use the Hibernate-specific @Sort annotation to perform in-memory (i.e. not in the database) sorting, using natural sorting or sorting using a Comparator you supply. For example, assume I have an EventComparator helper class that implements Comparator. I could change Timeline's collection of events to look like this:

@OneToMany(mappedBy = "timeline")
@org.hibernate.annotations.Sort(type = SortType.COMPARATOR, comparator = EventCompator)
 Set<Event> events

Using @Sort will perform sorting in-memory once the collection has been retrieved from the database. While you can certainly do this and implement arbitrarily complex sorting logic, it's probably better to sort in the database when you can. So we now need to turn to Hibernate's @OrderBy annotation, which lets you specify a SQL fragment describing how to perform the sort. For example, you can change the events mapping to :

@OneToMany(mappedBy = "timeline")
@org.hibernate.annotations.OrderBy("start_year, end_year")
 Set<Event> events

This sort order is the same as using the JPA @OrderBy with "startYear, endYear" sort order. But since you write actual SQL in Hibernate's @OrderBy you can take advantage of whatever features your database has, at the possible expense of portability across databases. As an example, Oracle 10g supports using a syntax like "order by start_year, end_year nulls first" to order null end years before non-null end years. You could also say "order by start_year, end year nulls last" which sorts null end years last as you would expect. This syntax is probably not portable, so another trick you can use is the NVL function, which is supported in a bunch of databases. You can rewrite Timeline's collection of events like so:

@OneToMany(mappedBy = "timeline")
@org.hibernate.annotations.OrderBy("start_year, nvl(end_year , start_year)")
 Set<Event> events

The expression "nvl(end_year , start_year)" simply says to use end_year as the sort value if it is not null, and start_year if it is null. So for sorting purposes you end up treating end_year as the same as the start_year if end_year is null. In the contrived example earlier, applying the nvl-based sort using Hibernate's @OrderBy to specify SQL sorting criteria, you now end with the events sorted like this:

2001       Event with no end year
2001 2002  Some event
2001 2003  Other event

Which is what you wanted in the first place. So if you need more complex sorting logic than what you can get out of the standard JPA @javax.persistence.OrderBy, try one of the Hibernate sorting options, either @org.hibernate.annotations.Sort or @org.hibernate.annotations.OrderBy. Adding a SQL fragment into your domain class isn't necessarily the most elegant thing in the world, but it might be the most pragmatic thing.

In almost every application I've done, the database tables have some kind of audit trail fields. Sometimes this is a separate "audit log" table where all inserts, updates, deletes, and possibly even queries are logged. Other times there are the four typical audit trail fields in each table, for example you might have created_by, created_on, updated_by, and updated_on fields in each table. The goal in the latter case is to update those four fields with the appropriate information as to who created or updated a record and when they did it. Using s simple Hibernate Interceptor this can be accomplished with no changes to your application code (with several assumptions which I'll detail next). In other words, you won't need to and definitely should not be manually setting those audit properties littered around your application code.

The basic assumptions I'll make for this simple audit interceptor are that: (1) model objects contain the four audit properties mentioned above, and (2) there is an easy way to obtain the current user's information from anywhere in the code. The first assumption is needed since you need some way to identify which properties constitute the audit trail properties. The second assumption is required because you need some way to obtain the credentials of the person making the change in order to set the createdBy or updatedBy property in your Hibernate Interceptor class.

So, for reference purposes, assume you have a (Groovy) base entity like this with the four audit properties:

@MappedSuperclass
class BaseEntity implements Serializable {
  String createdBy
  Date createdOn
  String updatedBy
  Date updatedOn
}

I'm using the Hibernate ImprovedNamingStrategy so that camel case names are translated to underscored names, e.g. "createdBy" becomes "created_by". Next assume there is a BlogEntry entity class that extends BaseEntity and inherits the audit trail properties:

@Entity
class BlogEntry extends BaseEntity {
  @Id @GeneratedValue (strategy = GenerationType.IDENTITY)
  Long id

  @Version
  Long version

  String title

  @Column (name = "entry_text")
  String text

  @Temporal (TemporalType.TIMESTAMP)
  Date publishedOn
}

To implement the interceptor, we need to implement the aforementioned Interceptor interface. We could do this directly, but it is better to extend EmptyInterceptor so we need only implement the methods we actually care about. Without further ado, here's the implementation (excluding package declaration and imports):

class AuditTrailInterceptor extends EmptyInterceptor {

  boolean onFlushDirty(Object entity, Serializable id, Object[] currentState,
                      Object[] previousState, String[] propertyNames,
                      Type[] types) {
    setValue(currentState, propertyNames, "updatedBy", UserUtils.getCurrentUsername())
    setValue(currentState, propertyNames, "updatedOn", new Date())
    true
  }

  boolean onSave(Object entity, Serializable id, Object[] state,
                 String[] propertyNames, Type[] types) {
    setValue(state, propertyNames, "createdBy", UserUtils.getCurrentUsername())
    setValue(state, propertyNames, "createdOn", new Date())
    true
  }

  private void setValue(Object[] currentState, String[] propertyNames,
                        String propertyToSet, Object value) {
    def index = propertyNames.toList().indexOf(propertyToSet)
    if (index >= 0) {
      currentState[index] = value
    }
  }
}

So what did we do? First, we implemented the onFlushDirty and onSave methods because they are called for SQL updates and inserts, respectively. For example, when a new entity is first saved, the onSave method is called, at which point we want to set the createdBy and createdOn properties. And if an existing entity is updated, onFlushDirty is called and we set the updatedBy and updatedOn.

Second, we are using the setValue helper method to do the real work. Specfically, the only way to modify the state in a Hibernate Interceptor (that I am aware of anyway) is to dig into the currentState array and change the appropriate value. In order to do that, you first need to trawl through the propertyNames array to find the index of the property you are trying to set. For example, if you are updating a blog entry you need to set the updatedBy and updatedOn properties within the currentState array. For a BlogEntry object, the currentState array might look like this before the update (the updated by and on propertes are both null in this case because the entity was created by Bob but has not been updated yet):

{ 
  "Bob",
   2008-08-27 10:57:19.0,
   null, 
   null, 
   2008-08-27 10:57:19.0, 
   "Lorem ipsum...",
   "My First Blog Entry",
   0
}

You then need to look at the propertyNames array to provide context for what the above data represents:

{
  "createdBy",
  "createdOn",
  "updatedBy",
  "updatedOn",
  "publishedOn",
  "text",
  "title",
  "version"
}

So in the above updatedBy is at index 2 and updatedOn is located at index 3. setValue() works by finding the index of the property it needs to set, e.g. "updatedBy," and if the property was found, it changes the value at that index in the currentState array. So for updatedBy at index 2, the following is the equivalent code if we had actually hardcoded the implementation to always expect the audit fields as the first four properties (which is obviously not a great idea):

// Equivalent hard-coded code to change "updatedBy" in above example
// Don't use in production!
currentState[2] = UserUtils.getCurrentUsername()

To actually make your interceptor do something, you need to enable it on the Hibernate Session. You can do this in one of several ways. If you are using plain Hibernate (i.e. not with Spring or another framework) you can set the interceptor globally on the SessionFactory, or you can enable it for each Session as in the following example code:

// Configure interceptor globally (applies to all Sessions)
sessionFactory =
  new AnnotationConfiguration()
    .configure()
    .setNamingStrategy(ImprovedNamingStrategy.INSTANCE)
    .setInterceptor(new AuditTrailInterceptor())
    .buildSessionFactory()

// Enable per Session
Session session = getSessionFactory().openSession(new AuditTrailInterceptor())

If you enable the interceptor globally, it must be thread-safe. If you are using Spring you can easily configure a global interceptor on your session factory bean:

<bean id="sessionFactory"
      class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
  <property name="entityInterceptor">
    <bean class="com.nearinfinity.hibernate.interceptor.AuditTrailInterceptor"/>
  </property>
  <!-- additional Hibernate configuration properties -->
</bean>

On the other hand, if you would rather enable the interceptor per session, you either need to use the openSession(Interceptor) method to open your sessions or alternatively implement your own version of CurrentSessionContext to use the getCurrentSession() method in order to set the interceptor. Using getCurrentSession() is preferable anyway since it allows several different classes (e.g. DAOs) to use the same session without needing to explicitly pass the Session object around to each object that needs it.

At this point we're done. But, if you know about the Hibernate eventing system (e.g. you can listen for events such as inserts and updates and define event listener classes to respond to those events), you might be wondering why I didn't use that mechanism rather than the Interceptor. The reason is that, to the best of my current knowledge, you cannot alter state of objects in event listeners. So for example you would not be able to change an entity's state in a PreInsertEventListener implementation class. If anyone knows this is incorrect or has implemented it, I'd love to hear about it. Until next time, happy auditing!

This is the final (and way, way overdue) article in a series of blogs describing how you can effectively use Hibernate validators. The fifth article described how to bypass Hibernate validation in specific use cases, for example if you need to save a "draft" object that should not be validated yet. In this article I'll describe how the Hibernate Validator can be integrated into web applications so that validation errors propagate seamlessly from the data access code back up through the web tier and to the end user as nicely formatted error messages. For this article I'm using Spring MVC, but the basic concept should be applicable no matter which of the 100 billion Java web frameworks you are using.

The basic concept is this: When a user submits a form, you first want to bind the form values to your domain object (making sure of course that you only allow user-editable fields to be bound to the object). You then want to validate the domain object with the updated values. Finally, you want the Hibernate Validator validation errors translated into your web framework's native error validation mechanism so it can inform the user of the errors and do things like display the errors to the user next to the problematic fields. Stated more succinctly, the following steps must occur: submit form, data binding to domain object, validate domain object using Hibernate Validator, translate Hibernate Validator errors to web framework errors, re-display form with nicely formatted error messages to user for correction.

The only piece we don't have is the translation of Hibernate Validator errors into web framework errors. Since I'm using Spring MVC in this case, I'll need to take the Hibernate Validator errors and translate them into a Spring Errors object. For this I can use Spring MVC's Validator interface to implement the error translation in a generic fashion. For this blog the implementation is going to be simple and naive, and won't take into account things like nested properties or internationalization because I want to keep things relatively simple.

So, let's look at the HibernateAnnotationSpringValidator class which is responsible for validating any of our domain objects (which are all assumed to ultimately extend from a custom BaseEntity class for this example) and then translating the Hibernate Validator InvalidValue objects into Spring MVC Errors.

package com.nearinfinity.common.spring.validation.hibernate;

// imports...

public class HibernateAnnotationSpringValidator implements Validator {

  private Map validatorCache = new HashMap();

  public boolean supports(Class clazz) {
    return BaseEntity.class.isAssignableFrom(clazz);
  }

  @SuppressWarnings(CompilerWarnings.UNCHECKED)
  public void validate(Object value, Errors errors) {
    Class type = value.getClass();
    ClassValidator validator = validatorCache.get(type);
    if (validator == null) {
      validator = new ClassValidator(type);
      validatorCache.put(type, validator);
    }
    InvalidValue[] invalidValues = validator.getInvalidValues(value);
    translateToSpringValidationErrors(invalidValues, errors);
  }

  private void translateToSpringValidationErrors(InvalidValue[] invalidValues, Errors errors) {
    for (InvalidValue invalidValue : invalidValues) {
      String propertyName = invalidValue.getPropertyName();
      if (propertyName == null) {
        errors.reject(null, invalidValue.getMessage());
      }
      else {
        String titleCasedPropertyName = StringUtils.camelCaseToTitleCase(propertyName);
        String errorMessage = titleCasedPropertyName + " " + invalidValue.getMessage();
        errors.rejectValue(invalidValue.getPropertyPath(), null, errorMessage);
      }
    }
  }

}

The most important things in the above code are the validate and translateToSpringValidationErrors methods. As expected, validate expects a domain object that extends from BaseEntity and uses Hibernate Validator to validate it. This validator caches Hibernate ClassValidator instances and so one instance of HibernateAnnotationSpringValidator could be used in all of your Spring MVC controllers if desired. It then validates the object and gets back an array of InvalidValue objects from Hibernate Validator. Finally, it calls translateToSpringValidationErrors.

The translateToSpringValidationErrors method iterates through the InvalidValues and transforms them into Spring MVC errors. This implementation is very simplistic and not i18n-ready as it uses the domain object property names to create the error messages using a utility method I wrote called camelCaseToTitleCase. For example, if the property name is "firstName" then the "titleCasedPropertyName" is simply "First Name." So, if the InvalidValue's getMessage() method returns "is required" then the error message would be "First Name is required." Obviously for production use you'll want to make something more robust than this and support i18n if you need to.

Now you have all the pieces and simply need to plug-in the HibernateAnnotationSpringValidator and use as you would any Spring Validator which is documented extensively in the Spring docs and lots of books, blogs, twitters, etc. Obviously this example only works with Spring MVC. To make it work for the web framework you are using, you'll need to hook into your framework's validation mechanism and do something similar. Again, the two most important things are (1) using Hibernate Validator to validate an object (doesn't even need to be a domain object for that matter since Hibernate Validator can be used on its own independent of Hibernate even) and (2) translating the Hibernate validation errors into your web framework's native errors. That's it!