talkingCode

Archive for the debian category

Java, SSL, and the Keystore of Doom

posted by codders in code, debian, java

In a break from our Haskell programming…

I used to be a staunch defender of Java as a language. I still think it’s relatively good (though mostly for the tool support), but there are things about it that make me want to scream.

Imagine, for example, you’d like to make an SSL authenticated fetch from a webserver. You have a client certificate to authenticate your client, and a server certificate to authenticate the server, and you’ve generated them both from your own CA. Shouldn’t be that hard, right? Wrong :( Everything SSL has to be configured via the key stores, so you need to import your private certificate and the server’s public certificate in to your key store in order to make anything go.

There are two stores - the Trust Store and the Key Store. The Trust store contains the certificates you trust (CAs, etc.). The Key Store contains certificates for which you have the private key and against which you’ll encrypt challenges to verify your identity. All you have to do is populate them…

Step 1: Don’t use GCJ
There are a lot of great things to be said for the Open Source outlook on life. GCJ isn’t one of them. It works quite like Java, except when you try and run anything. Unfortunately it works sufficiently like Java that you don’t necessarily know you’re using it, and it’s installed as the default on a lot of Debian machines.

keytool error: java.lang.IllegalStateException: masked envelope

That was the first cryptic clue that I was using GCJ. Other clues are random GC messages on the console. Here’s a quick way to tell if you’re infected:

$ ls -l /etc/alternatives/ | grep -c java-gcj
24

The number you’re looking for is ‘0′ on a correctly configured system. Specifically you want to see:

$ chase `which keytool`
/usr/lib/jvm/java-1.5.0-sun-1.5.0.15/jre/bin/keytool
$ chase `which java`
/usr/lib/jvm/java-1.5.0-sun-1.5.0.15/jre/bin/java

If you’re not getting that, reconfigure the alternative:

# update-alternatives --config java

There are 7 alternatives which provide `java'.

  Selection    Alternative
-----------------------------------------------
          1    /etc/alternatives/kaffe-system/bin/java
          2    /usr/bin/gij-wrapper-4.0
*         3    /usr/lib/jvm/java-1.5.0-sun/jre/bin/java
          4    /usr/bin/gij-4.1
          5    /usr/bin/gij-4.3
 +        6    /usr/lib/jvm/java-gcj/jre/bin/java
          7    /usr/bin/gij-4.2

Step 2: Import the CA
Now we’re running the right JVM, it should be a simple matter of:

$ #Create a trust store with a CA Cert in it (teststore.jks doesn't yet exist)
$ keytool -import -v -trustcacerts -alias myalias -file cacert.pem -keystore teststore.jks
Enter keystore password:
keytool error: java.lang.NullPointerException

Oh. There may be a way to use blank passwords on keystores, but keytool ain’t it. Let’s try again with a password:

$ keytool -import -v -trustcacerts -alias myalias -file cacert.pem -keystore teststore.jks
Enter keystore password: password

Smashing. That’ll mean we can at least connect to the remote host. But the SSL handshake will still fail when the host sees our lack of client certificate.

main, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Step 3: Import the Client certificate
My client certificate comes as two files - a certificate PEM file (the public part) and a key PEM file (the private part). Naïvely, I tried just installing the PEM part:

$ keytool -import -v -alias myalias2 -file signup1-cert.pem -keystore teststore.jks
Enter keystore password:  password
keytool error: java.lang.Exception: Input not an X.509 certificate

*sigh*. That’ll happen. Fortunately, we can convert from PEM to ‘DER’, which is something that keytool understands, using ‘openssl’:

$ openssl x509 -in signup1-cert.pem -inform PEM -out signup1-cert.der -outform DER
$ keytool -import -v -alias myalias2 -file signup1-cert.der -keystore teststore.jks
Enter keystore password:  password
Certificate was added to keystore
[Storing teststore.jks]

It’s stored, but unfortunately a) it doesn’t work and b) keytool thinks this is a ‘trustedCertEntry’ rather than a ‘keyEntry’:

$ keytool -v -list -keystore teststore.jks
...
Entry type: trustedCertEntry
...

Now, we can use ‘openssl’ to convert our certificate and key into a PKCS#12 combined key file:

$ openssl pkcs12 -export -in signup1-cert.pem -inkey signup1-key.pem -out signup1.p12

Even better, according to the documentation, PKCS#12 format files are valid key stores… unless you try and use them:

default context init failed: java.io.IOException: Invalid keystore format

Right. Let’s try that as a combined PEM format file then:

$ openssl pkcs12 -in mykey.p12 -out keystore.pem -nodes
$ keytool -import -v -alias clientcert -file keystore.pem -keystore keystore.jks
Enter keystore password:  password
keytool error: java.lang.Exception: Input not an X.509 certificate

True. (Incidentally, that’s No DES, not ‘nodes’) But we can convert PEM files to DER files

$ openssl x509 -in keystore.pem -inform PEM -out keystore.der -outform DER
$ keytool -import -v -alias clientcert -file keystore.der -keystore keystore.jks
Enter keystore password:  password

Step 4: Using the keystore in your program
You can configure the keystore at runtime as follows:

    System.setProperty("javax.net.ssl.keyStore", context.getRealPath(KEYSTORE));
    System.setProperty("javax.net.ssl.keyStorePassword", "password");
    System.setProperty("javax.net.ssl.trustStore", context.getRealPath(TRUSTSTORE));
    System.setProperty("javax.net.debug", "ssl");
    HttpClient httpClient = new HttpClient();
    GetMethod httpGet = new GetMethod("https://something.com");
    httpClient.executeMethod(httpGet);
    return new String(httpGet.getResponseBody());

javax.net.debug=ssl truly is a magic rune. I don’t know if you can get a list of such runes, but commit that one to memory. The debug output is pretty handy, if I little hard to follow.

Step 5: Become frustrated
What I didn’t mention, and perhaps should have mentioned above, is that I still hadn’t managed to import my key as a keyEntry, so this code still didn’t work. I downloaded the source code to the JDK and tried single-stepping KeyTool, but that also didn’t help. [Aside: People who create compressed archives without a top-level folder should be shot].

Step 6: Use KeyMan
KeyMan can be downloaded from IBM Alphaworks at time of writing:
http://www.alphaworks.ibm.com/tech/keyman/download
It ‘just works’. It lets you import your certificate and create a valid key store. Thanks IBM! (Source code plz).

Normalising MP3s

posted by codders in debian

I’m changing the way I blog, and possibly starting to do it again. But I digress…

The OpenRightsGroup blog linked a recording of an interesting talk with Jonathan Zittrain (whose new book is getting some press and whose e-book PDF is available for free download), but the recording was a bit quiet and I wanted to listen to it while cooking my lunch…

# sudo apt-get install mp3gain
# mp3gain /tmp/org.mp3

/tmp/org.mp3
Recommended “Track” dB change: 25.760000
Recommended “Track” mp3 gain change: 17
WARNING: some clipping may occur with this gain change!
Max PCM sample at current gain: 21450.428803
Max mp3 global gain field: 190
Min mp3 global gain field: 129

Recommended “Album” dB change for all files: 25.760000
Recommended “Album” mp3 gain change for all files: 17
WARNING: with this global gain change, some clipping may occur in file /tmp/org.mp3

# mp3gain -g 17 /tmp/org.mp3
Applying gain change of 17 to /tmp/org.mp3…

done

…which was nice.

Talk:
http://www.openrightsgroup.org/2008/06/06/the-future-of-the-internet-in-focus/

Book:
http://futureoftheinternet.org/

Building your own USB pendrive Linux image

posted by codders in debian

I don’t know about you, but whenever I leave the house I have the same six items in my pockets (in order of certainty with which I believe I’m going to need them) - keys, wallet, phone, handkerchief, penknife and, as of the end of December, a 1GB USB stick with a basic Debian Live system on it.

Before December, had I been asked at a party “Exactly how geeky are you?”, I’d have meekly to have replied “Fairly.”. Henceforth, though, I shall be able to reach in to my pocket and say “Well, if you have a computer that can boot from USB mass storage and that’s configured to do so in the BIOS, I can show you…”. Of course, having such a device won’t just make you popular with the ladies - there’s a chance it’ll also have practical applications. Without further ado then…

First thing you’ll need is (at time of writing) a Debian Lenny (or more recent) install. Real men run unstable, but if you’re sitting there with Etch (running Firefox 0.0.1a on your Linux 2.0.34 machine) you can easily enough conjure up a suitable chroot with the standard:

debootstrap lenny chroot
chroot chroot

(There is, I should warn, some scope for confusion when you attempt the following inside a chroot, because the scripts create their own chroot)

Next thing to do is to install the helper packages:

apt-get install live-helper

We’re going to build the image in a fresh folder somewhere. You’ll want to pick a filesystem with plenty of disk space (~5GB) just to be on the safe side. In the fresh folder, typing lh_config will create the config directory for the image building scripts:

mkdir live
cd live
lh_config

We want to tweak some of the config files - it’d be pretty dull to go to this effort just to build a vanilla image. Below I’ve listed the changes I made to build my ‘dream’ install. The variables listed already exist in the files - you just need to change them to match what’s below:

# In 'config/binary'
LH_BINARY_IMAGES="usb-hdd"
LH_BOOTAPPEND_LIVE="locale=en_GB.UTF-8 keyb=uk"

# In 'config/bootstrap'
LH_SECTIONS="main contrib non-free"

# In 'config/chroot' (you won't want the linebreaks)
LH_PACKAGES="dns2tcp doc-base dsniff ettercap ettercap-common
less vim wireless-tools iceweasel icedove gaim curl openssh-client
openssh-server irssi centerim hexdump iproute iptables nemesis nmap
ntop privoxy socat tcpdump tor wireshark wireshark-common
firmware-ipw3945 ipw3945d ipw3945-modules-2.6-486 ncftp telnet
netcat fluxbox eterm xserver-xorg xfonts-base"

The idea with those package selections is a) to make you dangerous and b) to enable you to punch a hole through to the internet wherever possible. Once online, you can obviously install whatever else you need.

You’ll also want to:

mkdir -p config/chroot_local-includes/etc/skel/
echo fluxbox > config/chroot_local-includes/etc/skel/.xinit

This image boots to console, but typing startx will give you an X session (assuming it can detect and load the correct graphics driver).
To build the image, simply type

lh_build

Your completed image will appear as binary.img in that folder (weighing in at around 235Mb). In the event that something screws up, it’s important to understand that the state for the build process is tied up in the .stage folder. Until you notice that hiding there, the whole system can seem a bit mysterious.

All that remains is to transfer the image to your pendrive.
CAUTION: This next instruction is a dd onto a block device. My pendrive is /dev/sdb, but for all I know on your system that’s the SATA hard disk containing the only copy of your doctoral thesis.

dd if=binary.img of=/dev/sdb

You’re done. You should be able to reboot into that (assuming your BIOS supports it and is correctly configured). Couple of extra things to point out. The dd copies across a boot sector and the first partition. You can use fdisk to create further partitions on the device if you want a little read-write space. Specifically, if you type live persistent at the SysLinux prompt:

live-initramfs will look for persistent and snapshot partitions or files labeled “live-rw”, “home-rw”, and files called “live-sn*”, “home-sn*” and will try to, in order: mount as /cow the first, mount the second in /home, and just copy the contents of the latter in appropriate locations (snapshots). Snapshots will be tried to be updated on reboot/shutdown. Look at live-snapshot(1) for more informations.

More details in the manpage. To label a partition, you’re looking for:

mke2fs -L your_label /dev/sdbX

after you’ve created it in fdisk

The astute among you may spot the live-magic package. This is a GUI front-end to the process I’ve just described, but not something I’ve actually tried to use.

ERROR 2026 (HY000): SSL connection error - the joy of MySQL SSL on Debian

posted by codders in debian, linux, mysql, sysadmin

OpenSSL has some issues. It can’t be linked against GPL software, and Debian only includes free software (in its main archive). So when, in order to encrypt communications to your MySQL server, you issued the magic:

mysql> GRANT ALL PRIVILEGES ON database.* TO 'someuser'@'%' IDENTIFIED BY 'somepassword' REQUIRE SSL;

And tried to connect to the server (an empty certificate suffices for this purpose) with the rune:

# mysql -u someuser -psomepassword -h my.server.com database --ssl --ssl-ca=/dev/null

you might well have been frustrated to see the cryptic

ERROR 2026 (HY000): SSL connection error

Sucks to be you. (N.B. In order for ‘REQUIRE SSL’ to have any effect, you need to have enabled SSL on the server. See /etc/mysql/my.cnf) There are at least two possible causes. One is that the certificates you’ve generated for the server are in some way broken, and that can be true on any system. The other, which plagues the current Debain packages (5.0.32-7etch1 at time of writing) is the OpenSSL linking issue in the client. So what’s to be done? Well the long and the short of it is that if you’re on Debian, you’re at least going to have to recompile the mysql-server package with OpenSSL support, depressing as that undoubtedly is. For reasons of hygiene in linking, we’ll need to do this in a chroot. Don’t worry - it won’t hurt a bit:

cd /usr/local
mkdir chroot
debootstrap etch chroot
# Make yourself a drink.
mount -t proc none chroot/proc/
chroot chroot
# If you've not already got a 'src' URL:
echo deb-src http://ftp.uk.debian.org/debian etch main >> /etc/apt/sources.list
apt-get update
apt-get install devscripts
# At this point, you may start to see
# 'perl: warning: Setting locale failed.'
# If so...
apt-get install locales
dpkg-reconfigure locales
# ... and select the missing locale.
# Doesn't really hurt if you don't do that though.
cd /usr/src
apt-get build-dep mysql-server
apt-get source mysql-server
cd mysql-dfsg-5.0-5.0.32/
# either ...
wget http://talkingcode.co.uk/wp-content/2007/11/patch.txt
patch -p0 < patch.txt
# ... or change the line 'without-openssl' in debian/rules to 'with-openssl'
# and 'with-yassl' to 'without-yassl'
apt-get install libssl-dev
# Change the version:
debchange -v 5.0.32-7etch1+ssl-1 "Added SSL"
dpkg-buildpackage
# Time to go get another drink. Consider getting a biscuit too.
cd ..
ls *ssl*.deb
echo "That's handy"
exit

So now you have your SSL enabled packages, it’s a simple matter of installing them on the target machine:

dpkg -i *.deb

(though you could reasonably skip installing the server if you don’t need it).

And there you have it - you should now be able to connect over SSL to your server (if your certificates are okay).

If you want to connect from a Python or Perl script using SSL, you’re going to need to install the fresh .debs inside the chroot and recompile the appropriate Python and Perl MySQL binding packages in the same chroot so as to make them link the modified libmysqlclient.

New Laptop

posted by codders in debian, linux, sysadmin

What I _actually_ wanted to write about was installing Debian on my new laptop, in a geeky kind of way.

It’s been about 2 years since I did anything other than a clean install on a machine, and in that time it seems things have come along a little. The new machine is a Thinkpad, so all the dull hardware compatibility stuff is up on ThinkWiki as per. They also have some handy instructions on how not to destroy all your data.

Good news!!! QtParted is now able to resize NTFS, which has been a long time coming and means I no longer have to blitz the XP install that came with the laptop. It’s supported on the most recent Knoppix CD and, being determined and stubborn as is my wont, I decided I’d try and do the install manually from there instead of trying something more conventional like a Netinst CD.

Step 1: Burn the Knoppix CD. Straightforward enough

Step 2: QtParted. The Thinkpad comes with an NTFS partition at the start of the drive and a recovery partition at the end, so the rest is yours to play with. My first thought was just to make the rest LVM, but I couldn’t for the life of me make grub install to an LVM partition. Second attempt involved a little boot partition and an LVM root, which seems to work better.

Step 3: debootstap debian into the LVM partition. Something like…

mke2fs /dev/sda2 # The boot partition
tune2fs -j /dev/sda2
pvcreate /dev/sda3 # The LVM partition
vgcreate vg /dev/sda3
lvcreate -L10G -n slash vg
mke2fs /dev/vg/slash
tune2fs -j /dev/vg/slash
mkdir /tmp/bootstrap
mount /dev/vg/slash /tmp/bootstrap
mkdir /tmp/bootstrap/boot
mount /dev/sda2 /tmp/bootstrap/boot
debootstrap etch /tmp/bootstrap http://ftp.uk.debian.org/debian
chroot /tmp/bootstrap
apt-get update
apt-get install linux-image-2.6.18-5-686
exit
grub install --directory=/tmp/bootstrap/boot /dev/sda

et voila, as ze French would ‘ave it. Well, ish. Turns out that doesn’t work. Even if you persuade grub to do the right thing it is, it seems, the devil’s own job to create an initrd that’ll boot with an LVM rootfs (can’t find a good link to explain rootfs). I’m sure there are people that can make that happen. I didn’t really have the patience or the knowhow.

Step 4: Give up, install from the Netinst CD. It just works, and supports all the LVM goodness you could ever wish for.

Step 5: Copy /home from your old machine. One of the really lovely things about Linux is that once you’ve copied your home directory to your new machine, the place really does feel like Home (fsvo ‘once’. You still need to make sure you have roughly the same packages installed).
What most impressed me, though, was that having allocated only 10GB as the initial rootfs I was able, while everything was mounted and running, to resize things. Turns out that lvextend and resize2fs can both be run on the mounted filesystem, which makes having to choose the ‘right’ size for the initial partitions completely redundant. I now have nice /home, /var, /, and /var/warez partitions and needn’t worry about any of them running out and time soon since I still have 30GB unused in the LVM. Should also prevent the ol’ “/var/cache/apt/archives ate my entire disk and I didn’t realise” problem that I often have with Debian. It ought to be as simple as just removing the files in there from time to time, but I never seem to realise until the disk is full. Limiting /home also has the advantage that I’ll know when it becomes un-backup-able in advance of my disk filling up.

So there was that. I’m a happy bunny now with my new lappy. In other news, alsaconf autodetects soundcards and s2disk suspends to disk first time (and resumes!). I could bitch about having to compile my own kernel because of cutting-edge wireless ipw3945 stupidity, but I won’t. I’ve entertained you long enough.

That was me writing about Linux. Because we’re still in a ‘getting to know you’ phase, I’ve been heavy on the hyperlinks. I don’t know who you are or how much you know, and I wouldn’t want anyone to feel excluded by the use of jargon, idiom and turn of phrase. I’ll stop extending that courtesy just as soon as it gets boring (in about 10 minutes). If in doubt, Wikipedia, then Google, then Urban Dictionary. Can’t lose.

Recent Posts
Recent Comments
About Us
Franta: and Step 7: Become frustrated again...
Dave: hey, just wondering if there is a working demo somewhere. The above demo does not se...
Flemming Frandsen: Hi, I'd just like to thank you for posting this, it was an imeasureable help to me, s...
qbJim: Doing it with C++ iostreams would have saved remembering the parameter list to read a...
C-rat: I better put the Prelude on my reading list too. I might use init as a good example o...

This is the personal blog of a professional software engineer. This site and the views expressed on it are in no way endorsed by the RIAA.