talkingCode

Java, SSL, and the Keystore of Doom

In a break from our Haskell programming…

I used to be a staunch defender of Java as a language. I still think it’s relatively good (though mostly for the tool support), but there are things about it that make me want to scream.

Imagine, for example, you’d like to make an SSL authenticated fetch from a webserver. You have a client certificate to authenticate your client, and a server certificate to authenticate the server, and you’ve generated them both from your own CA. Shouldn’t be that hard, right? Wrong Everything SSL has to be configured via the key stores, so you need to import your private certificate and the server’s public certificate in to your key store in order to make anything go.

There are two stores - the Trust Store and the Key Store. The Trust store contains the certificates you trust (CAs, etc.). The Key Store contains certificates for which you have the private key and against which you’ll encrypt challenges to verify your identity. All you have to do is populate them…

Step 1: Don’t use GCJ
There are a lot of great things to be said for the Open Source outlook on life. GCJ isn’t one of them. It works quite like Java, except when you try and run anything. Unfortunately it works sufficiently like Java that you don’t necessarily know you’re using it, and it’s installed as the default on a lot of Debian machines.

keytool error: java.lang.IllegalStateException: masked envelope

That was the first cryptic clue that I was using GCJ. Other clues are random GC messages on the console. Here’s a quick way to tell if you’re infected:

$ ls -l /etc/alternatives/ | grep -c java-gcj
24

The number you’re looking for is ‘0′ on a correctly configured system. Specifically you want to see:

$ chase `which keytool`
/usr/lib/jvm/java-1.5.0-sun-1.5.0.15/jre/bin/keytool
$ chase `which java`
/usr/lib/jvm/java-1.5.0-sun-1.5.0.15/jre/bin/java

If you’re not getting that, reconfigure the alternative:

# update-alternatives --config java

There are 7 alternatives which provide `java'.

  Selection    Alternative
-----------------------------------------------
          1    /etc/alternatives/kaffe-system/bin/java
          2    /usr/bin/gij-wrapper-4.0
*         3    /usr/lib/jvm/java-1.5.0-sun/jre/bin/java
          4    /usr/bin/gij-4.1
          5    /usr/bin/gij-4.3
 +        6    /usr/lib/jvm/java-gcj/jre/bin/java
          7    /usr/bin/gij-4.2

Step 2: Import the CA
Now we’re running the right JVM, it should be a simple matter of:

$ #Create a trust store with a CA Cert in it (teststore.jks doesn't yet exist)
$ keytool -import -v -trustcacerts -alias myalias -file cacert.pem -keystore teststore.jks
Enter keystore password:
keytool error: java.lang.NullPointerException

Oh. There may be a way to use blank passwords on keystores, but keytool ain’t it. Let’s try again with a password:

$ keytool -import -v -trustcacerts -alias myalias -file cacert.pem -keystore teststore.jks
Enter keystore password: password

Smashing. That’ll mean we can at least connect to the remote host. But the SSL handshake will still fail when the host sees our lack of client certificate.

main, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Step 3: Import the Client certificate
My client certificate comes as two files - a certificate PEM file (the public part) and a key PEM file (the private part). Naïvely, I tried just installing the PEM part:

$ keytool -import -v -alias myalias2 -file signup1-cert.pem -keystore teststore.jks
Enter keystore password:  password
keytool error: java.lang.Exception: Input not an X.509 certificate

*sigh*. That’ll happen. Fortunately, we can convert from PEM to ‘DER’, which is something that keytool understands, using ‘openssl’:

$ openssl x509 -in signup1-cert.pem -inform PEM -out signup1-cert.der -outform DER
$ keytool -import -v -alias myalias2 -file signup1-cert.der -keystore teststore.jks
Enter keystore password:  password
Certificate was added to keystore
[Storing teststore.jks]

It’s stored, but unfortunately a) it doesn’t work and b) keytool thinks this is a ‘trustedCertEntry’ rather than a ‘keyEntry’:

$ keytool -v -list -keystore teststore.jks
...
Entry type: trustedCertEntry
...

Now, we can use ‘openssl’ to convert our certificate and key into a PKCS#12 combined key file:

$ openssl pkcs12 -export -in signup1-cert.pem -inkey signup1-key.pem -out signup1.p12

Even better, according to the documentation, PKCS#12 format files are valid key stores… unless you try and use them:

default context init failed: java.io.IOException: Invalid keystore format

Right. Let’s try that as a combined PEM format file then:

$ openssl pkcs12 -in mykey.p12 -out keystore.pem -nodes
$ keytool -import -v -alias clientcert -file keystore.pem -keystore keystore.jks
Enter keystore password:  password
keytool error: java.lang.Exception: Input not an X.509 certificate

True. (Incidentally, that’s No DES, not ‘nodes’) But we can convert PEM files to DER files

$ openssl x509 -in keystore.pem -inform PEM -out keystore.der -outform DER
$ keytool -import -v -alias clientcert -file keystore.der -keystore keystore.jks
Enter keystore password:  password

Step 4: Using the keystore in your program
You can configure the keystore at runtime as follows:

    System.setProperty("javax.net.ssl.keyStore", context.getRealPath(KEYSTORE));
    System.setProperty("javax.net.ssl.keyStorePassword", "password");
    System.setProperty("javax.net.ssl.trustStore", context.getRealPath(TRUSTSTORE));
    System.setProperty("javax.net.debug", "ssl");
    HttpClient httpClient = new HttpClient();
    GetMethod httpGet = new GetMethod("https://something.com");
    httpClient.executeMethod(httpGet);
    return new String(httpGet.getResponseBody());

javax.net.debug=ssl truly is a magic rune. I don’t know if you can get a list of such runes, but commit that one to memory. The debug output is pretty handy, if I little hard to follow.

Step 5: Become frustrated
What I didn’t mention, and perhaps should have mentioned above, is that I still hadn’t managed to import my key as a keyEntry, so this code still didn’t work. I downloaded the source code to the JDK and tried single-stepping KeyTool, but that also didn’t help. [Aside: People who create compressed archives without a top-level folder should be shot].

Step 6: Use KeyMan
KeyMan can be downloaded from IBM Alphaworks at time of writing:
http://www.alphaworks.ibm.com/tech/keyman/download
It ‘just works’. It lets you import your certificate and create a valid key store. Thanks IBM! (Source code plz).

Buzzword Bingo - Tomcat, Jetty, Cactus, Derby, Velocity, Maven

I’ve been writing a webapp at work recently. Because I’m writing it in Java, the scope for me to post interesting articles about what I’m doing has been pretty limited - I don’t know that much about enterprisey Java, and I spend most of my time bashing my head against a Java-shaped wall. A quick overview is probably worth doing though.

Apache
If you’ve not heard of the Apache Foundation (outside the context of their fine webserver), go visit their site now. Amongst other things, Apache are creating an incredible resource on that site in the form of a huge code commons. If you’re not sure what a commons is, a) shame on you and b) read this fine book (in fact, read it even if you are sure). The majority of the code on the Apache site is Java, so if you’re developing an application in Java it’s well worth checking the links on the left-hand side of the main page to see if they’ve already written what you’re writing, or something that will help you. The code is all Apache-licensed - a BSD-style license that is amenable to reuse in commercial projects as well as Free Software projects. It’s also very high quality code. You basically can’t lose.

strace is your friend
One of the biggest irritations, I’ve found, in using the Apache Java projects (and this is more of a function of Java than Apache) is trying to make Java find the appropriate config files. It’s pretty difficult to tell which config files Java is seeing, if any. You can download the source to the component you’re using and single step it in Eclipse, but this tends to get tedious at about the point you hit the class loader, which is exactly the point at which it’ll do any resource location. What you can do, though, is:

strace -f -p[YOUR_PID] 2>&1 | grep “your_config_file.xml”

You’re looking for things like calls to ’stat’ or results of ‘ENOENT’. This will generally tell you what Java’s looking for and where. That is, I grant you, a sledgehammer-nut solution, but I’ve found it quicker than anything else. If you know a better way to work out where I should be putting my Velocity toolbox.xml in my Maven tree for the Webapp to run in Jetty under Cactus please let me know. It’s not in the FAQ.

My environment
What am I doing with all these projects?

• Maven - Your lifecycle management tool (build and dependency management). Maven2 is much improved over Maven for having the package repositories work 99% of the time rather than the 50% that was more common in the original. The whole POM thing is… errr… a little obtuse, I grant you, but you can use Maven in a dumb way without too much trouble. The Eclipse plugin is a great way to find the packages you need too.
• Tomcat - The application server. Can’t really do a Webapp without one of those. Tomcat isn’t the only server out there, but it’s pretty well used and, as at 5.5-ish, not too difficult to make go.
• Velocity - The templating engine. The web is awash with webapp ‘frameworks’; Apache have about four of them for starters. Sometimes, though, you’re not writing for web browsers. Sometimes, you just want a way to turn your POJOs into arbitrary markup. Velocity is fast, uncomplicated and fully featured.
• Cactus - The Servlet test framework. I’ve still not completely wrapped my head around Cactus, but basically for each of your unit tests you have setUp and tearDown to manipulate state on the server, and begin and end to manipulate state on the client, and Cactus will run web requests inside a test harness for you. Trés handy.
• Jetty - A lightweight servlet container. Controversially not an Apache project, Jetty is really quite handy for running your Cactus tests. Instead of pushing the WAR all the way to Tomcat, you can new up a Jetty server inside the Maven test task and run your functional tests there.
• Derby - The lightweight database. To avoid having to have a real database available in order to run your unit tests, I find it convenient to put some test fixtures in a Derby DB and run the servlet off the back of that. This plays really nicely with the Cactus / Jetty setup and has the added advantage that (unlike MySQL) if you try and create a prepared statement that your schema can’t possibly run, Derby will warn you about that at preparation time rather than execution time. With a bit of tweaking, then, you can statically check your statements against the schema.

Making all that play together has been nightmarish, either because of my relative inexperience or because it’s genuinely difficult. I’ve embarrassingly failed to write down all the errors I encountered and how to fix them, but I may cover some of the more tricky parts of the setup in future articles. For all the setup work, though, being able to type mvn install and feel confident that your latest refactoring hasn’t broken the complex application you’ve written is worth almost any amount of blood, sweat and tears.