Watching Japanese TV in Dublin

ssh terebi mkfifo pipe ssh terebi cat pipe | mplayer -

then in another window

ssh terebi mencoder -oac lavc -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=100:acodec=ac3:abitrate=64 /dev/video0 -o pipe

gives me watchable live tv.

I think I should really be using some variety of vlc but that's another day's work

Useful things when you've wiped you partition table (conclusion)

This is the concluding part of my earlier post on recovering after wiping the partition table and the first 30k or so of my harddisk. It has a happy ending.

It didn't take much more effort in the end to rebuild my laptop's partition table. It turns out that gpart was not terribly useful as I had used extended partitions and nuked the first one. It found my 2 linux partitions but refused to rebuild the partition table. It also outputs the location of the partitions in MB which is no use at all. This patch

diff -ur gpart-0.1h/src/gpart.h ../gpart-0.1h/src/gpart.h --- gpart-0.1h/src/gpart.h 2001-01-30 23:07:29.000000000 +0000 +++ ../gpart-0.1h/src/gpart.h 2007-01-03 05:07:06.000000000 +0000 @@ -171,7 +171,7 @@ struct disk_geom *disk_geometry(disk_desc *); int reread_partition_table(int); -#define s2mb(d,s) { (s)*=(d)->d_ssize; (s)/=1024; (s)/=1024; } +#define s2mb(d,s) { (s)*=(d)->d_ssize; } #define align(b,s) (byte_t *)(((size_t)(b)+(s)-1)&~((s)-1)) #include "gmodules.h"

makes it output bytes instead which I could then divide by 512 to get sectors. I had to compile it on a Debian box (not sure what vintage), it wouldn't compile under Ubuntu.

Next I used sfdisk -d /dev/sda to dump a partition table from another machine as an example and then edit that to look like

# partition table of /dev/sda unit: sectors /dev/sda1 : start= 63, size= 39053952, Id=83 /dev/sda2 : start= 39070143, size= 3903728, Id=82 /dev/sda3 : start= 42973938, size= 72260370, Id=8e /dev/sda4 : start=115234308, size= 1975932, Id=83, bootable

I worked out the sizes by subtracting the start values from each other. Saving that in a file and doing sfdisk < my_partitions.txt gave me a new partition table which amazingly worked. Of course /dev/sda1 wasn't recovered properly because it had also been partially overwritten by the format but luckily I had nothing useful on that partition.

Get The Focaccia!

I spotted a focaccia (Italian bread with veggies and maybe bits of bacon) shop while in Courmayeur a few days ago and got a great idea for a franchise. Kinda along the lines of It's A Bagel. As with all franchises you need a snappy name. I think

GET THE FOCACCIA!

has a certain ring to it. The menu would explain all the different types of focaccia available and so would be titled

WHAT THE FOCACCIA?

The opening hours, the recipe book and the store locator would be named along similar lines.

Memoising a complete parser

Not sure if "complete" is the correct term but I mean a parser that finds all possible parses of the stirng by the grammar. Usually very expensive but by memoising the application of rule A at char N in the string it becomes cheaper. The key is the value that you store. For memoised recursive descent, you just store the parsed value or FAIL. For this you have to store an object that can generate parsed values in the correct order and which caches any previously generated ones. Not so bad really. There are some questions though about quantifiers. It might be worth trying to share data between X{4} and X{5} applied at the same char. But maybe not.

An alternative to a recovery floppy

Does anybody ever make a recovery disk these days after installing Linux. I certainly don't and now I'm regretting it! It's too inconvenient to root out a floppy or a blank CD and even if I had made one, I wouldn't have it with me here in Tokyo so I'd be just as badly off. Basically they are not terribly helpful.

The amount of non-redundant information stored on a recovery disk is quite small, basically it's your MBR and partition table and maybe some special boot params. Really all you need is a safe copyof your MBR and partition table. With that you can boot from a generic recovery CD, fix the partition table and MBR and then reboot as normal (or maybe fix the data on your partitions first now that you can access them).

So the world would be a better place if instead of offering to make a recovery CD, Linux distros offered to email you a copy of your MBR and partition table. If you use gmail then that's the end of it, it'll be there in your gmail account should the need arise. Download it and use dd to install it. Bingo. If you store your email on the computer itself then you need to copy that mail to somewhere else but that's still much easier that creating a floppy/CD that you're going to lose.

I'm off to file some bugs (just as soon as I get my data back!).

Useful things when you've wiped you partition table

Doh! While trying to get my slax distro onto a USB key, I fucked up big time and dos formatted (mkfs.vfat) my hardrive (my head is still in the era where hard disks are hdX, not sdX).

gpart can try to guess where your partitions were but it doesn't know LVM2

LVM2 partitions have a sector which begins with "LABELONE" as documented in these slides (thanks to agk on #lvm!). I invoked the following script with ./lvm.pl /dev/sda to get a list of all the sectors that contains that string.

#! /usr/bin/perl use strict; use warnings; my $buf; my $sector = 0; my $f = shift; open(F, $f) || die "couldn't open '$f', $!"; while(read(F, $buf, 512)) { if ($buf =~ /^LABELONE/) { print "sector = $sector\n"; } $sector++; } print "last sector was $sector\n";

Stuffing all those sector numbers into $ss and then doing

for s in $ss; do sudo perl -e 'open(F, "/dev/sda"); seek(F, shift()*512, 0); read(F, $buf, 512); print $buf' > sec.$s $s; done

gets me all the sectors in sec.XXX files. I can then

for s in $ss; do echo $s; echo; hexdump -c sec.$s; echo ;done|less

to take a look at them without trashing my console. The important thing to remember is that LABELONE will be followed by a secotr number. If it's 1 then this is sector 1 of the partition but really I'm looking for sector 0 so I need to subtract 1.

Making a secure bootable cd with sshd and my ssh keys

Update: ddclient doesn't seem to work. I don't have time to figure out why, the rest works fine though. Update: slax is currently "frozen", I've updated the links in this doc to point to the legacy hosting service. Also, it seems that they have messed with the sshd config and possibly broken it in the latest version.

I've built a headless box that's going to live in Japan when I go back to Ireland. There'll be no one in Japan to fix it if it goes bad. The best I can do is get someone to insert a CD and reboot but after some searching and some asking it seems none of the bootable CDs will boot the machine into a useful state for remote admin. slax was recommended to me on TLUG as a good customisable distro. It turned out to be pretty easy thanks to slax's rootcopy feature which allows you to make minor modifications without digging around (too much) in the compressed package files. I've broken the process down into a few steps, with a quick bash script for each one.

Create a directory to work in and cd into it

mkdir slax cd slax

Get a copy of the .iso

You need slax-frodo-x.y.z.iso, you can get it from the slax download page. The frodo edition is the minimal version on which all the others are based.

Make a copy of the files from the CD

This is necessary because we need to add some files. So I ran this as ./prep_image.sh ~/slax-frodo-5.1.8.iso

#! /bin/bash iso=$1 shift if [ "$iso" = '' ]; then echo 1>&2 Usage: echo 1>&2 "$0 " exit 1 fi mkdir slax.mnt mount -t iso9660 $iso slax.mnt -o loop cp -v -Tr --preserve=all slax.mnt slax umount slax.mnt

Enable SSH

It's disabled by default (rc.sshd is not executable), so I replace it with one that is executable. I also turn off password logins because everyone knows the root password for slax. Finally I include an authorized_keys file so that I can ssh in with my ssh key. I ran this script as ./enable_ssh.sh ~/.ssh/authorized_keys

#! /bin/bash auth_keys=$1 shift if [ "$auth_keys" = '' ]; then echo 1>&2 Usage: echo 1>&2 "$0 <path/to/auth_keys_file<" exit 1 fi # make rc.sshd executable and stop root logins using a password # # extract rc.sshd from the module and tweak the permissions mkdir 02_core.mnt mount -t squashfs slax/base/02_core.mo 02_core.mnt -o loop mkdir -p rootcopy/etc/rc.d cp 02_core.mnt/etc/rc.d/rc.sshd rootcopy/etc/rc.d chmod 700 rootcopy/etc/rc.d/rc.sshd # disable password logins mkdir -p rootcopy/etc/ssh cp 02_core.mnt/etc/ssh/sshd_config rootcopy/etc/ssh cat >> rootcopy/etc/ssh/sshd_config <<EOM # since we're starting sshd by default, don't allow root logins with # passwd, must use ssh keys PasswordAuthentication no EOM # install keys file mkdir -p rootcopy/root/.ssh chmod 700 rootcopy/root/.ssh cp $auth_keys rootcopy/root/.ssh chmod 600 rootcopy/root/.ssh/authorized_keys cp -r --preserve=all rootcopy/* slax/rootcopy umount 02_core.mnt

Update a dyndns entry

The machine is on a cable modem and doesn't have a fixed IP, so I need to know where it is. I downloaded the ddclient module and the following script puts it in place, puts your ddclient.conf file into /etc and makes sure that it gets started on boot (I assumed it would start by itself actually, I'm not sure if that's a bug or not). I invoked this script as ./ddclient.sh ddclient.conf

#! /bin/bash conf=$1 shift if [ "$conf" = '' ]; then echo 1>&2 Usage: echo 1>&2 "$0 " exit 1 fi cp ddclient_3_7_0.mo slax/base mkdir -p rootcopy/etc/ cp $conf rootcopy/etc/ddclient.conf mkdir -p rootcopy/etc/rc.d cat > rootcopy/etc/rc.d/rc.local <<EOM #! /bin/bash /etc/rc.d/rc.ddclient start EOM chmod 755 rootcopy/etc/rc.d/rc.local

Add any other modules

There doesn't appear to be an editor in the slax-frodo image so I grabbed joe and copied the .mo into slax/base/

Move files into place and create a new .iso

So far everything we've done has been in a rootcopy/ directory in the current directory. Now it's time to move that into place in slax/ and invoke the iso building script with the following script which outputs it to slax-ssh.iso in the current directory.

#! /bin/bash rm -rf slax/rootcopy cp -r --preserve=all rootcopy slax cd slax ./make_iso.sh ../slax-ssh.iso

Test it in qemu

qemu -cdrom slax-ssh.iso should boot up. I never figured out how to get qemu bring up a real IP interface - the virtual machine can connect to the outside world but I can't connect to it. I tested it by setting up an ssh tunnel connecting port 22 on the VM to port 2022 on the host machine with ssh -R2022:localhost:22 fergal@my.host.machine.ip. Then I did ssh -p 2022 root@localhost with my ssh keys loaded and in I went. When my keys aren't loaded, I don't get a password prompt so I can't get in. Perfect.

deep 0.3

Finally got deep onto code.google.com/hosting/ and onto the python package repo CPAN-alike thingy, which means that it's just an easy_install deep away whenever I want it.

Using tailor to move from hg to svn

First question, why would I want to do that? Actually I don't really want to do that but code.google.com uses SVN as its VCS and I want to host a project there. I originally used Mercurial (hg) for that project and so I need to get it out of hg and into svn. The ultimate goal is to use either Bazaar-NG (bzr) with it's svn plugin or svk, this is a step along that path.

Second question, isn't that straight forward? No, tailor's hg->svn support is busted. hg doesn't care about directories, svn does and this makes it impossible to move from hg to svn if you have any directories (I filed a bug). So instead I must move everything to bzr first and then to svn as the bzr->svn code seems to handle directories correctly. It seems that one of hg->bzr or bzr->svn makes the directories problem go away.

The following little bash script takes 3 arguments:

• the path to the hg repo that you want to put into svn
• the path to the svn repository
• the name of the svn module that the code will live in

It uses these to output 2 tailor project files, hg2bzr.tailor and bzr2svn.tailor and then it runs tailor twice. #! /bin/bash source=$1 shift svn_repo=$1 shift svn_module=$1 shift # create 2 project files cat <<eof> hg2bzr.tailor [DEFAULT] verbose = True [project] target = bzr:temp start-revision = INITIAL root-directory = $PWD/working-bzr state-file = tailor.state source = hg:source subdir = . [hg:source] repository = $source [bzr:temp] # no details needed because eveything just ends up in working-bzr EOF cat <<eof> bzr2svn.tailor [DEFAULT] verbose = True [project] target = svn:target start-revision = INITIAL root-directory = $PWD/working-svn state-file = tailor.state source = bzr:temp subdir = . [bzr:temp] repository = $PWD/working-bzr [svn:target] module = $svn_module repository = $svn_repo EOF # convert from hg to bzr tailor --configfile hg2bzr.tailor # and then from bzr to svn tailor --configfile bzr2svn.tailor

I invoked it as

./hg_to_svn deep-hg https://python-deep.googlecode.com/svn/ trunk/

and the result is in the SVN repository. Yay!

It takes quite a while to push all the changes and I had to clear out the repo once as the svn command stalled and timed out (server problem I think). Then I made a mess of restarting the whole process (I think it also wasn't helped by me reusing a directory from my test run!). Speaking of the test run, I played around with svk and it seemed quite awkward in comparison to hg and bzr. Not having used any of them for heavy duty work I can't really speak with authority. It definitely seems that svk's branch and merge process (as described in a rather good tutorial) reuires typing more commands and typing more awkward paths than with bzr or hg.

Omiyamairi

We brought Ríona for omiyamairi on Wednesday. It's the shinto equivalent of christening but without the water. She and Midori were wearing a beautiful kimono. There are pictures here. We did it at the shrine in West Ogikubo. The sun was shining and the trees were beautiful.

One amusing thing is the girl in the booth who took our details. She is a miko - a servant of God. She is supposed to be a virgin. Miko are involved in/help out with various festivals etc. These days nobody really does it as their job. It's usually a school girl or maybe college student (they need lots of miko around new year) and they're just doing it part-time. What exactly a part-time virgin means, I'm not sure!

When is it art?

Sometimes artists use materials that are difficult for them to control. So, for example, using a very broad brush means your painting will (can not) capture small details but maybe that's OK or even desirable. So is it still art when the artist is just unskilled with the tools. For example, one day, some artist may discover a technique to capture fine detail with a broad brush. Now anyone using a broad brush can capture fine details and must choose not to (or choose not to learn how to). Does that change the quality of their art? Does it make a difference that they could have conveyed their intent more precisely? I don't think so.

Why am I suddenly thinking about art? I saw a poster for a band in Tokyo called "Bump of Chicken". I have no clue what was supposed to mean but that's not the point. Just because I and other native speakers know how to use English to create "fine detail", does that mean that someone else who only knows how to use it as a "broad brush" is wrong? If this was a public sign then I think the answer would be yes but it's not. A band's name is a (small) piece of art, not a means of conveying practical information and if the artist chooses a medium in which he is unskilled or which is difficult for him then I suppose the result is still legitimate even if it makes no sense to someone skilled in the medium.

So from now on, I'll look on dodgy English usage as abstract art.

Irish people everywhere

We arrived in Tokyo after a rather gruelling flight involving unexpected overnight in Paris and Ríona doing a fair amount of screaming (actually she was pretty good for quite a lot of the trip, she just seemed to be bored for the last few hours). Didn't do much the first day and didn't realise it was Friday (not Thursday) otherwise I would have made a trip to the office to get some bits and bobs.

Anyway, so here we are in Tokyo on Saturday morning and we take a trip out, stopping for coffee. After we order another westerner comes up and orders. "American coffee please". Oh for f*ck's sake, he's Irish. I didn't try talking to him, I assumed he lived here and foreigners living in Japan can be a bit odd, wanting to keep away from tourists but after a while he came over to our table and introduced himself (he heard my accent). He's an accordian player named Jackie Daly (what are the chances!?) and he's playing a few concerts. What's more he's here with Paddy and Bridget, a Japanese couple who live in Ireland and are apparently quite well known and they live a few hundred metres from Midori's parents house!

That's just ridiculous

Cluster bombs in Lebannon

Intersting analysis on the cluster bombs dropped in Lebannon after the cease fire had been agreed. They seem to have used about the worst type of cluster bomb available (in terms of long term civilain casualties) despite having had other options.

Israeli Massacre

This is the second account of the Marwahin massacre I've read. It's pretty shocking. Posting it so I'll remember.

Sayonara baby!

Midori has mostly avoided maternity blues but it occurred to her a few days ago that some day, Ríona will leave us. She had a dream that Ríona was packing a suitcase with all her worldly good and saying "Sayonara". Of course in the dream Ríona was still a baby, was mostly packing nappies and apparently said "Sayonara" in a very cute way!