talkingCode

Archive for November, 2007

Byte-packing / reading binary data in Ruby

posted by codders in code, ruby

A lot of high-level languages don’t have wonderful native support for bit-level manipulation of data, so when you find yourself having to implement a proprietary wire protocol or parse a custom file header, you often feel a little lost. Fortunately you’re not the first person to feel lost, and some kind person has, by and large, already solved the problem for you. In Java, for example, there’s the fine commons.io library (in fact, all the commons libraries are pretty fine), and in Ruby you have BitStruct:

# Syntax here is 
#    parsed-datatype
#    :symbol_name for the parsed data
#    field size in bits
#    comment
class BinaryHeader < BitStruct
  default_options :endian => :native
  char     :id, 4*8, "File ID"
  char     :format_string, 4*8, "Format String"
  unsigned :remainder, 32, "Remaining bytes"
  unsigned :trackId, 32, "Track ID"
  unsigned :formatId, 32, "Format ID"
  unsigned :codecId, 32, "Codec ID"
  unsigned :major, 16, "Major Version"
  unsigned :minor, 16, "Minor Version"
  unsigned :validation, 32, "Validation Data"
  unsigned :size, 32, "File Size"
end

There are a number of ways you can find binary data on your hands. You could read directly from a file / socket, you could generate some, or you could receive it Base64 encoded from a Webservices request:

def get_header(file)
  header_bytes_base64 = @webservice.getFileHeader(file.trackId)
  header_bytes = Base64.decode64(header_bytes_base64)
  if header_bytes.size != HEADER_SIZE
    puts "Invalid header size: #{header_bytes.size}"
    return nil
  end
  header = BinaryHeader.new(header_bytes)
  puts "Got Header:"
  puts header.inspect_detailed
  if header.id != "FILE"
    puts "Unknown file ID: #{id}"
    return nil
  end
  if header.format_string != "MP3X"
    puts "Unknown file format: #{format}"
    return nil
  end
  return header
end

Problem solved.

JSON “RPC” from Perl to Ruby / Webrick

posted by codders in code, perl, ruby

So I have a bunch of software I’d written in Ruby because it was “The Right Choice™” and I need to make it talk to a stack of software I’d written in Perl (because “I Was An Idiot™”). Specifically, I need the Perl to be able to call the Ruby. I had a quick dig around, and there’s a Perl module Inline::Ruby which, on the face of it, would do the job. Unfortunately, Inline::Ruby is version 0.0.2 software and “Doesn’t Work So Good™”. Not only that, but I’d really like some persistence in the Ruby code so that I don’t have to new up the state every time I make a call. What this calls for, then, is some IPC

Linux IPC comes in three varieties – Sockets, Files, and Shared Memory. Files is obviously a pretty poor idea in that you’ll either be polling a lot or writing nasty dnotify stuff. Shared Memory is okay but extremely unportable. Sockets are a pretty good idea, and if you choose an IP socket you get the advantage that you can run the communicating processes on different machines (assuming they don’t need to share other local resources).

So you’ve selected TCP/IP as a transport, but you’ve then got all sorts of irritating high-level protocol implementation to do. Unless…

#!/usr/bin/ruby
# A Simple Webserver for JSON RPC

require 'webrick'
require 'json'
require 'yaml'
include WEBrick
include YAML

server = HTTPServer.new(:Port => 8000)
server.mount_proc("/rhapsodise") do |request, response|
  response['Content-Type'] = "application/json"
  response.body = handle_json_request(request)
end

trap("INT") { server.shutdown }

def handle_json_request(request)
  object = JSON.parse(request.body)
  YAML.dump(object)
end

server.start

and then…

#!/usr/bin/perl -w
#
# A Simple Perl Client for JSON RPC

use strict;
use JSON;
use LWP;

my $actionurl = "http://localhost:8000/rhapsodise";

my $ua = LWP::UserAgent->new();
$ua->agent("JSONClient/0.1");

my $object = { test => "fish",
               wibble => ["meep", "flange" ] };

my $json = objToJson($object);
print "$json\n";
my $req = HTTP::Request->new(POST => $actionurl);
$req->content_type('application/json');
$req->content($json);

my $res = $ua->request($req);
if ($res->is_success)
{
  print "Succeeded:\n" . $res->content;
  my $result = jsonToObj($res->content);
}
else
{
  print "Failed\n";
}

… et voila. For less code than it might cost you to bind the socket in C(!) you’ve got a nice, portable way of making Ruby calls from your Perl through the Webrick webserver. Even better, you get to use Perl and JSON, as exhorted in my previous post. A dash more code for the server, and it’s very nearly useful:

def error_object(message)
  return { :error => message }.to_json
end

def handle_json_request(request)
  command = JSON.parse(request.body)
  if command["method"] == nil
    return error_object("No method suppled")
  end
  if command["method"] =~ /^rd_/
    return self.send(command["method"], command)
  end
  return error_object("No matching method")
end

def method_missing(m, *args)
  return error_object("Invalid command: #{m}")
end

Notice that I’m prefixing my RPC-able methods with ‘rd_’. This probably isn’t much more secure than not bothering, but is a useful kind of Hungarian Notation for the methods. I’m not worrying too much about security on this one since I write both ends and I trust the link across which the packets run – you’d need to take appropriate precautions if that wasn’t true.

Perl and JSON: heaven =~ m/made/ ?

posted by codders in ajax, code, perl

If you read the above as ‘Heaven Made’, I’m afraid you lose code Dingbats. Try harder.

Perl is a nice language and JSON is a good way to structure data if you’re about to send over your favourite IPC mechanism. And there are, it seems, many similarities between the way JSON structures data and the way Perl structures data. Unfortunately, I don’t have an awful lot to write on the topic, so as well as briefly introducing the basics of creating JSON in Perl I will, with the irrefutable logic of a professor of philosophy and all the religious sensitivity of a Dan Brown novel, be examining the evidence for Larry Wall being God.

We’re going to start with a motivating example. I want to represent the following JSON object in Perl:

{
  keywords:['cat', 'fish', 'pig'],
  somekey: 'somevalue',
  someobject: {
    nested: 'value',
    another: 'nestedvalue',
    a_number: 4
  }
  simple:true
}

I don’t know if you’ve come across JSON before, but it’s basically the way Javascript (ECMAScript) represents objects. Your modelling tools there are strings, numbers and booleans, as you might expect, and some collecting constructs. Specifically, JSON has a way to associate string keys with values – within the ‘{}’, thing on the left of the ‘:’ is the key and the thing on the right is the value, with commas seperating key-value pairs – and a way to create lists of values – inside the ‘[]‘, again seperated by commas.

Now, if you’re familiar with Hashes and Lists (or Arrays), you’ll see that JSON’s collection structures are exactly those. How do we represent those in Perl? Well, we use ‘{}’ for a hash and ‘[]‘ for a list. What’s more, we use commas to separate are values. I know what you’re thinking – “It’s mere coincidence”. The kind of coincidence that sees the Earth just the right temperature to support life, as a function of its being in just the right place in our solar system? Yes. _That_ kind of mere coincidence.

So how, naively, would I go about representing the above in Perl?

my $object = {
  keywords => ['cat', 'fish', 'pig'],
  somekey => 'somevalue',
  someobject => {
    nested => 'value',
    another => 'nestedvalue',
    a_number => 4
  }
  simple => true
};

But all I’ve done so far is replace the ‘:’ with ‘=>’. It can’t be that simple, can it? That would be too easy. Almost as though, at the start of the universe, someone had determined that JSON and Perl would have basically the same syntax. Could it be that this synergy is part of God’s great plan? Well, not quite. Turns out you can’t just whack ‘true’ in there for your boolean value. I’ve defined a little function here to return ‘true’:

sub json_true()
{
  return bless ( {value => "true"}, "JSON::NotString" );
}

That’s ugly as sin, I know. (AS SIN! And it’s BLESSED! This conspiracy writes itself.) Anyway. Why would a God, in designing this synergy, allow such evil? Here I distract you by shouting “Look out! Earthquake!”, say “Natural Evil” three times, and we sweep that under the carpet.

The only piece missing from this puzzle are the final bits of fairy dust that’ll actually let you convert JSON to a format suitable for sending over the wire. May I introduce:

use JSON;

my $obj = { test => 'object' };
my $json_string = objToJson($obj);
my $rebuilt_object = jsonToObj($json_string);

Simple as. But… is Larry Wall God? Could anyone but the supreme being have designed this happy coincidence into the starting conditions of the universe? That a humble pre-WWW scripting language would turn out to have approximately similar syntax to one of the high-level interchange protocols powering the modern web? Well, if Larry Wall were God I’d certainly be disappointed, and that’s to say nothing of the fuss that world religions would kick up. And, in fairness, this isn’t a coincidence that has to have been orchestrated from the dawn of the universe. You’d really only need to start thinking about it in… say… 1962.

Maybe Larry isn’t God. Maybe he’s just Al Gore.

Playing the organ is odd

posted by codders in music, organ, rant

I was planning to post something this morning – another thrilling technical article, no doubt – but it’s Advent this coming Sunday (at least, in college it is) and preparations for that have sucked up all my free time. Fortunately, while writing detailed technical articles requires research and effort, ranting mindlessly doesn’t. Yay.


The thing with playing the organ is that if you’ve done it right, nobody should notice you’ve done anything at all. Now, that’s true of quite a lot of things in life – flying a plane, removing an appendix, baking a soufflĂ©, packing a parachute, preventing a terror attack (don’t even get me started) – but playing the organ is different in that it’s something I do every other week so, you know, I care about it n’that. You’ve got two or three keyboards, a board of pedals which, for the uninitiated, act as another keyboard for the feet, and a whole pile of buttons, and for up to five minutes at a time all you have to do is put both hands, both feet, and all ten digits in the right place at the right time (to within, say, 5mm and 0.25 seconds).


So this week, I’ll mostly be practising. I’ll be playing the same few difficult bars over and over again knowing full well that the chances of them being right on the night are slim to nil, and that even if they’re right nobody will notice. I’m really not sure what the appeal is except to say that there’s something so much more fun about making music than listening to it and that all the practising and rehearsing is … err … fun, I guess. Personally, I prefer it when nobody’s listening, but I’m a lazy man and if it weren’t for the fact that I’ll have to play in front of 50-100 people on Sunday I’d probably never learn the pieces of music that I have. That and I don’t think I could justify the 10 hours a week of practise if it was every week and just for my own enjoyment.


Anyway. That’s why I’m not posting anything useful this week. Mr. SEO will no doubt berate me, complaining that the audience of Debian-using software-writing web-developing MySQL-administering geeks was niche enough before I made it about organ-playing as well, not to mention the fact that I’m (heaven forfend) writing about myself. He can bite me.

Drag and Drop table rows with AJAX, Scriptaculous and Prototype (Part II)

posted by codders in ajax, code

Last time, we saw how to render the UI for drag and drop table rows. In this post we’ll have a look at making the reordering that the user selects persistent, so that when they come back to the page (having cleared their cookies – this won’t be some session[] hack) they see the ordering they selected. What’s more, the Ajax stuff even works in Internet Explorer this time (assuming you’re using lists, not tables). If you’ve already worked it out, please stop reading now :)

I said last time that

new Ajax.Request('/some/action/url', 
    { method:'post', parameters:Sortable.serialize('table1') })

was the answer. To explain why, we’ll need a bit of preamble. A good general strategy for writing software when you’re not sure what you’re doing is to assume that whatever you’re trying to write has already been written. In this case, we know that we need to have the order of the rows stored in the database, so lets assume I have a database table from which I’m generating my entries:

CREATE TABLE my_table (
  id INT AUTO_INCREMENT,
  column1_value TEXT,
  column2_value TEXT,
  order INT,
  PRIMARY KEY (id)
);

I’m going to be generating the table using the results from the following query:

SELECT id, column1_value, column2_value 
FROM my_table 
ORDER BY order;

We’ll use table rows for this next bit, but you could use <li>s or whatever works for your browser. In whatever your favourite language is (let’s pretend you like Ruby/Rails/RHTML):

<% for row in rows %>
  <tr id="row_<%= row.id %>">
    <td><%=h row.column1_value %></td>
    <td><%=h row.column2_value %></td>
  </tr>
<% end %>

For balance, let’s pretend you like PHP too (disclosure – I don’t, nor do I really know PHP):

<?php
while ($row = mysqli_fetch_array($result))
{
?>
  <tr id="row_<?php echo $row["id"]; ?>">
    <td><?php echo $row["column1_value"]; ?></td>
    <td><?php echo $row["column2_value"]; ?></td>
  </tr>
<?php>
}
?>

So we’ve displayed the elements we want to be able reorder, sorted by the order specified in the database table. Making a call to Sortable.create as per the previous post will allow the user to reorder it, but what we really want to so is save the ordering. Sortable.create allows us to specify a callback function which will be executed every time the user changes the ordering (as soon as they release the mouse):

Sortable.create("table1", {tag:"tr", containment:["table1","table2"], onUpdate:sendUpdate})

So all we have to do is ensure that ‘sendUpdate’ sends the new order back to the user. At this point, installing FireBug is a really good idea. If you visit, for example, the previous post and enable FireBug, you can use the console tab to execute JavaScript on the page. Try the following:

>>> Sortable.serialize("table1")
"table1[]=1&table1[]=2"
>>> Sortable.serialize("table2")
"table2[]=1&table2[]=2&table2[]=3"

The Sortable.serialize function turns our rows into a string suitable for form POST / GET. You’ll notice that I gave my table rows “id”s like “row_1″ and “row_2″. If you name the tags you want sorting in the same way, the result of the Sortable.serialize call will be a POST / GET array whose elements are the “id” numbers of your sortable elements and whose order is the order that the elements appear after the user has made their change.

Prototype provides the Ajax.Request function for making XmlHttpRequests. We can use this to make our ‘sendUpdate’ function:

function sendUpdate(updatedElement)
{
  new Ajax.Request("/my/action/url.php", { 
method:"post", parameters:Sortable.serialize(updatedElement.id) 
})
}

Now, when the user changes the ordering, our “url.php” script will receive a POST from their browser containing an array whose name is the ID of the updated element and whose values are the rows. Back on the server side, then:

def action_function
  table1 = params[:table1]
  for i in (0..(table1.size - 1))
      t = TableRow.find(table1[i])
      t.update_attributes(:order => i)
  end
end

or, for devilment

function action_function()
{
  global $link;
  $table1 = $_POST["table1"]
  for ($i=0; $i<count($table1); $i++)
  {
    $link->query("UPDATE my_table SET order=$i WHERE id="
               . mysql_escape($link, $table1[$i]))
  }
}

Hopefully that’s all perfectly clear and mostly correct. Please let me know if not.

Drag and Drop table rows with AJAX, Scriptaculous and Prototype

posted by codders in ajax, code

* UPDATE: This’ll only work in Firefox (at time of writing. Other browsers may catch up)

Tables are like sooo last century, I know. Sometimes, though, you have data that is genuinely tabular and that you want the user to be able to reorder. I had this most recently with a tabular todo list that I was writing where I wanted the user to be able to priority order the actions by drag and drop.

Below, you’ll see four tables. You can drag items back and forth between tables 1 and 2, and between tables 3 and 4, but not 1 and 3 or 2 and 4 or 1 and 4 or 2 and 3. I’ve enabled the ‘ghosting’ effect on tables 1 and 2, but not on tables 3 and 4 – ghosting appears to screw up the layout a little once you’ve done some dragging.

Table 1

Col1 Col2
Value1Table1 Value2Table1
Value3Table1 Value4Table1

Table 3

Col1 Col2
Value1Table3 Value2Table3
Value3Table3 Value4Table3

Table 2

Col1 Col2
Value1Table2 Value2Table2
Value3Table2 Value4Table2
Value5Table2 Value6Table2

Table 4

Col1 Col2
Value1Table4 Value2Table4
Value3Table4 Value4Table4
Value5Table4 Value6Table4


Neat. The question is how, right? The answer is Sciptaculous, which is a handy Javascript library for Ajax and special effects that takes most of the misery and the having-to-know-any-javascript out of writing dynamic web pages.

<!-- Javascript Includes -->
<script type="text/javascript" src="/script/prototype.js"></script>
<script type="text/javascript" src="/script/scriptaculous.js"></script>
<!-- Standard HTML -->
<table>
<tr><td>
<p>Table 1</p>
<table padding="10" border="1" bgcolor="#aaaaff">
<tr><th>Col1</th><th>Col2</th></tr>
<tbody id="table1">
<tr id="row_1"><td>Value1Table1</td><td>Value2Table1</td></tr>
<tr id="row_2"><td>Value3Table1</td><td>Value4Table1</td></tr>
</tbody>
</table>
</td>
<td>
<p>Table 3</p>
<table border="1" bgcolor="#aaaaff">
<tr><th>Col1</th><th>Col2</th></tr>
<tbody id="table3">
<tr id="row_1"><td>Value1Table3</td><td>Value2Table3</td></tr>
<tr id="row_2"><td>Value3Table3</td><td>Value4Table3</td></tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td>
<p>Table 2</p>
<table border="1" bgcolor="#aaaaff">
<tr><th>Col1</th><th>Col2</th></tr>
<tbody id="table2">
<tr id="row_1"><td>Value1Table2</td><td>Value2Table2</td></tr>
<tr id="row_2"><td>Value3Table2</td><td>Value4Table2</td></tr>
<tr id="row_3"><td>Value5Table2</td><td>Value6Table2</td></tr>
</tbody>
</table>
</td><td>
<p>Table 4</p>
<table border="1" bgcolor="#aaaaff">
<tr><th>Col1</th><th>Col2</th></tr>
<tbody id="table4">
<tr id="row_1"><td>Value1Table4</td><td>Value2Table4</td></tr>
<tr id="row_2"><td>Value3Table4</td><td>Value4Table4</td></tr>
<tr id="row_3"><td>Value5Table4</td><td>Value6Table4</td></tr>
</tbody>
</table>
</td></tr></table>
<!-- Scriptaculous Magic -->
<script>
Sortable.create("table1", { tag:"tr", containment:["table1", "table2"], ghosting:true })
Sortable.create("table2", { tag:"tr", containment:["table1", "table2"], ghosting:true })
Sortable.create("table3", { tag:"tr", containment:["table3", "table4"]})
Sortable.create("table4", { tag:"tr", containment:["table3", "table4"]})
</script>

(sed -e ‘s/</\&lt;/g;s/>/\&gt;/g’ is my new bff)

Simple, right? Only slight oddness is the use of the <tbody> tag, which is a kink of the Scriptaculous library. In the call to ‘Sortable.create’, we’ve passed the “id” of the <tbody> element in as the first parameter, identified <tr> tags as the things we want to be able to reorder with the ‘tag’ option, specified the Sortable elements between which rows can be dragged with the ‘containment’ option, and turned on the ghosting effect in the obvious way. Only gotchas are that the includes at the top have to be done in the order shown, and that the calls to Sortable.create have to appear after the element definitions (not only of the Sortable in question, but also of any elements identified in the ‘containment’ option).
But you’re thinking “this is worse than useless – when I refresh the page, the sorting resets”. And you’d be right. This is where Prototype comes in. A copy of Prototype is included when you download Scriptaculous, and it implements all the handy little Ajax bits. This post is getting a bit long now, so next time I’ll explain why:

new Ajax.Request('/some/action/url', { 
method:'post', parameters:Sortable.serialize('table1') 
})

is basically all you need to know to be able to persist the reordering.

ERROR 2026 (HY000): SSL connection error – the joy of MySQL SSL on Debian

posted by codders in debian, linux, mysql, sysadmin

OpenSSL has some issues. It can’t be linked against GPL software, and Debian only includes free software (in its main archive). So when, in order to encrypt communications to your MySQL server, you issued the magic:

mysql> GRANT ALL PRIVILEGES ON database.* TO 'someuser'@'%' IDENTIFIED BY 'somepassword' REQUIRE SSL;

And tried to connect to the server (an empty certificate suffices for this purpose) with the rune:

# mysql -u someuser -psomepassword -h my.server.com database --ssl --ssl-ca=/dev/null

you might well have been frustrated to see the cryptic

ERROR 2026 (HY000): SSL connection error

Sucks to be you. (N.B. In order for ‘REQUIRE SSL’ to have any effect, you need to have enabled SSL on the server. See /etc/mysql/my.cnf) There are at least two possible causes. One is that the certificates you’ve generated for the server are in some way broken, and that can be true on any system. The other, which plagues the current Debain packages (5.0.32-7etch1 at time of writing) is the OpenSSL linking issue in the client. So what’s to be done? Well the long and the short of it is that if you’re on Debian, you’re at least going to have to recompile the mysql-server package with OpenSSL support, depressing as that undoubtedly is. For reasons of hygiene in linking, we’ll need to do this in a chroot. Don’t worry – it won’t hurt a bit:

cd /usr/local
mkdir chroot
debootstrap etch chroot
# Make yourself a drink.
mount -t proc none chroot/proc/
chroot chroot
# If you've not already got a 'src' URL:
echo deb-src http://ftp.uk.debian.org/debian etch main >> /etc/apt/sources.list
apt-get update
apt-get install devscripts
# At this point, you may start to see
# 'perl: warning: Setting locale failed.'
# If so...
apt-get install locales
dpkg-reconfigure locales
# ... and select the missing locale.
# Doesn't really hurt if you don't do that though.
cd /usr/src
apt-get build-dep mysql-server
apt-get source mysql-server
cd mysql-dfsg-5.0-5.0.32/
# either ...
wget http://talkingcode.co.uk/wp-content/2007/11/patch.txt
patch -p0 < patch.txt
# ... or change the line 'without-openssl' in debian/rules to 'with-openssl'
# and 'with-yassl' to 'without-yassl'
apt-get install libssl-dev
# Change the version:
debchange -v 5.0.32-7etch1+ssl-1 "Added SSL"
dpkg-buildpackage
# Time to go get another drink. Consider getting a biscuit too.
cd ..
ls *ssl*.deb
echo "That's handy"
exit

So now you have your SSL enabled packages, it's a simple matter of installing them on the target machine:

dpkg -i *.deb

(though you could reasonably skip installing the server if you don't need it).

And there you have it - you should now be able to connect over SSL to your server (if your certificates are okay).

If you want to connect from a Python or Perl script using SSL, you're going to need to install the fresh .debs inside the chroot and recompile the appropriate Python and Perl MySQL binding packages in the same chroot so as to make them link the modified libmysqlclient.

My publicist, and the right way to blog

posted by codders in meta

When I thought about setting up a blog I immediately went to Blogger because I knew the name and because, since they’re part of Google’s happy family, I could use my existing Gmail account to sign in. I wrote my first posts, then spoke to designer, SEO wizard and philanderer Mr. Rahman, who was both shocked AND appalled that I would write content for Blogger rather that host it myself.

See, I’m mostly writing this as therapy and experiment but in the unlikely event that I want to ‘make something’ of this site, it’s apparently crucially important that I host the blog myself. In the spirit of experiment, then, and in order to fully immerse myself in the whole blogging experience, I’ll be taking advice from Maruf as to the best way to make my blog a commercially viable project. I won’t give away all the man’s secrets, but the following is obvious. You will need:

  • A domain. 34sp are cheap and cheerful but (it turns out) lock you in to hosting with them if you buy a domain from them
  • Hosting. 34sp kinda have you over a barrel on this one, but that actually turns out to be no bad thing. They have a nice Plesk control panel, host on Apache (supporting things like mod_rewrite), and you can even run Rails (FastCGI) on their platform if you so choose. Access is all FTP / Web based, which is sucky but functional. Free e-mail and databases to boot, so that’s all pretty satisfactory
  • Blog Software. 34sp also provide a WordPress installer, so that’s what I’ve used.
  • Comment spam protection. The reCaptcha plugin for WordPress does this nicely
  • Some SEO tweaks, apparently
  • Time. I don’t have a lot of this, so how often I’ll post remains to be seen
  • Inclination. I’ve got at least some of this

So here we are, rehosted and ready for action. No doubt in the coming weeks I’ll be persuaded to plaster the page in adverts for Viagra and enter into dodgy reciprocal linking agreements with notorious spammers – can’t wait. All in the name of science, don’t you know.

New Laptop

posted by codders in debian, linux, sysadmin

What I _actually_ wanted to write about was installing Debian on my new laptop, in a geeky kind of way.

It’s been about 2 years since I did anything other than a clean install on a machine, and in that time it seems things have come along a little. The new machine is a Thinkpad, so all the dull hardware compatibility stuff is up on ThinkWiki as per. They also have some handy instructions on how not to destroy all your data.

Good news!!! QtParted is now able to resize NTFS, which has been a long time coming and means I no longer have to blitz the XP install that came with the laptop. It’s supported on the most recent Knoppix CD and, being determined and stubborn as is my wont, I decided I’d try and do the install manually from there instead of trying something more conventional like a Netinst CD.

Step 1: Burn the Knoppix CD. Straightforward enough

Step 2: QtParted. The Thinkpad comes with an NTFS partition at the start of the drive and a recovery partition at the end, so the rest is yours to play with. My first thought was just to make the rest LVM, but I couldn’t for the life of me make grub install to an LVM partition. Second attempt involved a little boot partition and an LVM root, which seems to work better.

Step 3: debootstap debian into the LVM partition. Something like…

mke2fs /dev/sda2 # The boot partition
tune2fs -j /dev/sda2
pvcreate /dev/sda3 # The LVM partition
vgcreate vg /dev/sda3
lvcreate -L10G -n slash vg
mke2fs /dev/vg/slash
tune2fs -j /dev/vg/slash
mkdir /tmp/bootstrap
mount /dev/vg/slash /tmp/bootstrap
mkdir /tmp/bootstrap/boot
mount /dev/sda2 /tmp/bootstrap/boot
debootstrap etch /tmp/bootstrap http://ftp.uk.debian.org/debian
chroot /tmp/bootstrap
apt-get update
apt-get install linux-image-2.6.18-5-686
exit
grub install --directory=/tmp/bootstrap/boot /dev/sda

et voila, as ze French would ‘ave it. Well, ish. Turns out that doesn’t work. Even if you persuade grub to do the right thing it is, it seems, the devil’s own job to create an initrd that’ll boot with an LVM rootfs (can’t find a good link to explain rootfs). I’m sure there are people that can make that happen. I didn’t really have the patience or the knowhow.

Step 4: Give up, install from the Netinst CD. It just works, and supports all the LVM goodness you could ever wish for.

Step 5: Copy /home from your old machine. One of the really lovely things about Linux is that once you’ve copied your home directory to your new machine, the place really does feel like Home (fsvo ‘once’. You still need to make sure you have roughly the same packages installed).
What most impressed me, though, was that having allocated only 10GB as the initial rootfs I was able, while everything was mounted and running, to resize things. Turns out that lvextend and resize2fs can both be run on the mounted filesystem, which makes having to choose the ‘right’ size for the initial partitions completely redundant. I now have nice /home, /var, /, and /var/warez partitions and needn’t worry about any of them running out and time soon since I still have 30GB unused in the LVM. Should also prevent the ol’ “/var/cache/apt/archives ate my entire disk and I didn’t realise” problem that I often have with Debian. It ought to be as simple as just removing the files in there from time to time, but I never seem to realise until the disk is full. Limiting /home also has the advantage that I’ll know when it becomes un-backup-able in advance of my disk filling up.

So there was that. I’m a happy bunny now with my new lappy. In other news, alsaconf autodetects soundcards and s2disk suspends to disk first time (and resumes!). I could bitch about having to compile my own kernel because of cutting-edge wireless ipw3945 stupidity, but I won’t. I’ve entertained you long enough.

That was me writing about Linux. Because we’re still in a ‘getting to know you’ phase, I’ve been heavy on the hyperlinks. I don’t know who you are or how much you know, and I wouldn’t want anyone to feel excluded by the use of jargon, idiom and turn of phrase. I’ll stop extending that courtesy just as soon as it gets boring (in about 10 minutes). If in doubt, Wikipedia, then Google, then Urban Dictionary. Can’t lose.

ftw?

posted by codders in meta, rant

Right. So. Here’s the thing. I work in ‘internet stuff’, software, digital media, etc.. Actually doing that for a day job does two things for you – it keeps you looking for The Next Big Thing for the purposes of exploiting it, and makes you feel anxious about your own obsolescence. The only way you’re going to keep up is to give things a go and find out what, if anything, you’re missing.

Take blogging. Please, somebody, take blogging. There’s an argument that says that the only people actually sitting down and reading blogs are the people who also happen to write them. A little community of mutual back-patting and reassurance. There’s a counter argument that looks at the number of blogs that show up in the top 10 search results and concludes that bloggers, with their series of niche contributions to human knowledge are actually making us better informed as a race. I no longer have to look for the website of some guy who spends his life installing Linux for help on installing Linux. I just find a blog post of somebody who happens once to have done that and maintains a blog.

So here am I. Looking for the next big thing, avoiding my own obsolescence and, where possible, contributing to human knowledge by reporting my niche experiences. I’m pretty sure there’ll be an initial flurry of posts and it’ll die, but you can’t say I didn’t try. Well, I mean, you could. You’d be wrong. It’d me more accurate to say I should have tried harder…

Recent Posts
Recent Comments
About Us
jp: works like a charm! thanks!...
Blake: Check this out: http://bugs.adobe.com/jira/browse/SDK-28016...
Boydell: Wow. That was it. You are the only one that had it figured out, and I looked at many...
mark van schaik: thanks! was using a beta SDK version for a production app, which stopped working over...
Sebastian: Steve, I find most asynchronous programming to be incredibly painful. Haskell's appro...

This is the personal blog of a professional software engineer. This site and the views expressed on it are in no way endorsed by the RIAA.