dagblog.
← Archive

Java 6 — what makes it faster?

Roman Pichlík ·

There's no doubt that Java 6 takes another step forward in performance, as David Dagastine's tests show, for example. I couldn't stop thinking about Java 6 performance, so I set out to find what was behind it. Alongside reading up on the theory, I've spent the last few days measuring how the JVM's new synchronization techniques perform.

Type Checking Verifier

I'll start with something I didn't test but that certainly affects performance. Every JVM has a bytecode verifier that does two basic things: type checking and type inference. Why type inference? Bytecode includes type information for method parameters and return values, but it doesn't contain it for local variables in methods. The verifier has to infer those types.

Type inference is slow, memory-intensive and rather complicated. Starting with Java 6.0, the verifier changed so that type inference moved from the JVM to the compiler. It now happens during compilation to bytecode. This split made the verifier twice as fast.

The new verifier, the Type Checking Verifier, can still run in its old type inference mode if the bytecode lacks the necessary type information or the information is incorrect. So if you're testing your code on Java 6, remember to compile it with the Java 6 compiler too, or you won't see the effect of the new verifier.

Sources

Synchronization optimizations — the theory

A great deal of work went into HotSpot, specifically its JIT compiler, which compiles to native code. The effect is especially noticeable for synchronized access. By synchronized access I mean using the Java keyword synchronized. Combined with the object used for synchronization, it provides mutual exclusion. In the critical section defined by the synchronized block and that object, exactly one thread can operate at a time.

The object (a monitor) used for synchronization has a lock. To the JVM, locks come in two categories, contended and uncontended. The terms might be translated into Czech as “competing” and “noncompeting,” but I'll stick with the English terminology.

contended
 With these locks, threads frequently overlap and compete. They can also be called hot locks. In applications, these are typically locks protecting parts of the code that many threads use at the same time. A database connection pool is a typical example.
 
 uncontended
 There's hardly ever any overlap on these locks. In other words, a thread trying to acquire the lock doesn't find another thread holding it.

Most locks are uncontended, according to the available information. The JVM has two different ways to acquire contended and uncontended locks. The JVM uses the “fast path” for a contended lock and the “slow path” for an uncontended lock. The difference is that on the “fast path,” two instructions are enough to acquire or release a lock (a simple owner flag that's null or not null). The “slow path” is much more complicated, since it has to manage threads by putting them to sleep or waking them up.

As we've said, most locks are uncontended, so the speed of the “fast path” matters. It only uses one or two instructions, but those instructions are atomic with respect to memory and therefore incur compare-and-swap (CAS latency). So any speedup for uncontended locks counts.

Java 6 has three new ways to speed up uncontended locking and therefore most programs. As we'll see, you have lots of uncontended locks in your programs, and you may not even know it.

Lock elision

There's strength in simplicity. Many uncontended locks aren't even necessary because the synchronized object never becomes visible beyond the thread's own stack. Suppose we have the following code.

public String doSomething() {
    Object lock = new Object();
    synchronized(lock){
      //do something
      return result;           
    }
}

The block is synchronized, but every thread has its own object to synchronize on. All the variables are local too, so synchronization is unnecessary. The threads can't affect each other. If you're wondering why anyone would use such unnecessary synchronization, here's an example with real code.

public String doSomething() {
    StringBuffer s = new StringBuffer();
    s.append("foo");
    s.append("hoo");
    s.append("bar");
    return s.toString();
}

For synchronization purposes, this code is equivalent to the previous example. Many core Java classes are thread-safe: they have the synchronization needed for safe use across multiple threads. The synchronization in StringBuffer is overhead we don't need in this particular case. Again, each thread gets its own s object, so the threads can't affect each other.

From version 6 on, the JVM can detect that the reference to object s never leaves the method doSomething. It can therefore remove the unnecessary synchronization at runtime. To decide whether it can do that, the JVM performs what's called escape analysis. I didn't invent that term after watching Prison Break for three days straight. It's called “Escape analysis” in the original English.

Escape analysis would let the JVM remove unnecessary synchronization and make a remarkable optimization possible: allocating objects on the stack. Allocating objects on the heap is much slower than allocating them directly on the stack. Java allocates all objects on the heap. Only primitive values and references to objects are on the stack.

With escape analysis, the JVM could optimize the creation of objects that don't leave local scope by allocating them on the stack. I say “could” deliberately. Although Java can do escape analysis and use it to eliminate unnecessary synchronization, it unfortunately can't do stack allocation yet.

Lock coarsening

Lock coarsening roughly translates into Czech as making a lock coarser. It reduces the overhead of locking and unlocking when sections can be grouped under one lock. A slight variation on the previous example will illustrate it.

public void doSomething(StringBuffer s) {
    s.append("foo");
    s.append("hoo");
    s.append("bar");
}

Each call to append locks and unlocks object s. The JVM can optimize the code so it doesn't lock and unlock on each call to append but instead, inside doSomething, creates one larger lock spanning all three calls. That reduces synchronization overhead.

Lock coarsening can have a side effect. It reduces synchronization overhead, but if an uncontended lock becomes contended it may increase the sequential portion of the program and reduce the portion that can run in parallel. That can have an unfortunate effect on parallelism. See Amdahl's law (the section on parallelization).

Lock biasing

Lock biasing again takes advantage of the fact that a lock is usually held by just one thread. In effect, it reserves the lock. When a thread first acquires the lock, a special flag is set. The next time the thread requests the lock, the flag is checked along with the thread ID. If the thread owns the lock, the flag is set, making the lock active.

This is also called the “ultra fast path” because it eliminates the CAS latency of the “fast path.” Lock biasing is enabled by default in Java 6. In Java 5, from version 5.0_06, you can enable it with -XX:+UseBiasedLocking. Biased locking makes the most difference on multiprocessor systems, where you'd otherwise need a locked version of the CAS instruction that would serialize processing.

Actual effect on performance

To test these features, I wrote a microbenchmark to check what lock elision does for performance. I won't keep you in suspense. It turned out just as the theory predicts. With the locks removed at runtime, the result was the same as if there had been no locking or synchronization in the first place. If you're interested in the results, see the detailed test description and microbenchmark measurements.

Sources

Medium version

Related articles

Related CZ Podcast episodes