Yesterday I gave a presentation about DragonFlyBSD’s HAMMER filesystem at the end of the system architecture lecture here at Karlsruhe Institute of Technology (also well known as University of Karlsruhe). It was quite interesting and exciting to stand in our hugest lecture hall and talk to maybe 150 students about an highly “innovative” topic, at least that’s how our Professor announced my presentation.
Automatic PFS creation for DragonFly’s HAMMER file system has now been committed (read here).
This article briefly describes how to install DragonFlyBSD on a Hammer filesystem, i.e. using Hammer for the root filesystem. Note that I make the assumption that the DragonFly installation will use the entire disk (in my case this is disk ad4). Also make sure that the DragonFly version you are using includes my RootOnHammer patches which should be the case for any release after 2.0 and every snapshot ISO image from now on. Btw, the whole approach is similar to ZFSOnRoot, except that I will not use the installer.
Start the live-cd installer and enter the root prompt.
Partition ad4 creating one slice covering the entire disk.
fdisk -B -I ad4
Generate a disklabel in your favourite editor that looks like the following one and save it under /tmp/disklabel.ad4s1.
# size offset fstype a: 262144 0 4.2BSD # 128.000MB b: 6291456 262144 swap # 3072.000MB c: 234441585 0 unused # 114473.430MB d: 227887984 6553600 4.2BSD # 111273.430MB
I’m using 128 MB for the boot partition (a), which should be enough to hold several kernels and it’s modules. Note that I am using 3 GB for the swap partition (b). The remaining space is left for the Hammer partition (d).
Then store the disklabel onto your disk:
disklabel -R -r ad4s1 /tmp/disklabel.ad4s1
Create the boot filesystem (UFS) and the Hammer filesystem:
newfs /dev/ad4s1a newfs_hammer -L HammerGeil /dev/ad4s1d
At first we set up the boot filesystem by mounting it and copying the /boot directory, the kernel and it’s modules onto it:
mount /dev/ad4s1a /mnt cpdup -vvv /boot /mnt/boot cpdup -vvv /kernel /mnt/kernel cpdup -vvv /modules /mnt/modules
Then we adjust /mnt/boot/loader.conf to use ad4s1d (our Hammer partition) as root filesystem:
touch /mnt/boot/loader.conf echo 'vfs.root.mountfrom="hammer:ad4s1d"' >> /mnt/boot/loader.conf
That’s all we need to do for the boot partition, so we unmount it.
Next we continue with the Hammer partition:
mount_hammer /dev/ad4s1d /mnt
onto which we will install all remaining parts of DragonFlyBSD. It’s as easy as copying the files from the live-cd to the Hammer partition:
cpdup -vvv /bin /mnt/bin cpdup -vvv /sbin /mnt/sbin cpdup -vvv /root /mnt/root cpdup -vvv /usr /mnt/usr cpdup -vvv /var /mnt/var cpdup -vvv /dev /mnt/dev cpdup -vvv /etc.hdd /mnt/etc cp /.cshrc /.profile /mnt mkdir /mnt/mnt /mnt/proc /mnt/tmp
We further want later on to mount the boot partition into /bootdir, and use /boot as short-cut (symlink) to /bootdir/boot:
mkdir /mnt/bootdir cd mnt ln -s bootdir/boot boot cd ..
We also want to edit /mnt/etc/fstab to contain the following:
# Device Mountpoint FStype Options Dump Pass# /dev/ad4s1a /bootdir ufs rw 1 1 /dev/ad4s1b none swap sw 0 0 /dev/ad4s1d / hammer rw 1 1 /dev/cd0 /cdrom cd9660 ro,noauto 0 0 proc /proc procfs rw 0 0
You might want to edit /mnt/etc/rc.conf now or later, after we have booted into the system.
That’s all, so we unmount the Hammer root partition and reboot (don’t forget to remove the live-cd):
umount /mnt reboot
Once we are up and running in our new Hammer-powered DragonFly system, we want to set up the timezone edit /etc/rc.conf and set the root password. You can easily set your timezone via:
You also might want to make /usr/obj non-history aware:
chflags nohistory /usr/obj chflags noshistory /usr/obj # do we need this?
Checkout the pkgsrc sources:
cd /usr make pkgsrc-checkout
What I also found to be very useful is to extend /usr/pkg/etc/mk.conf for the following two lines:
This tells pkgsrc to not create a work directory within each pkgsrc package where it extracts the sources and compiles the application. Instead it will use /usr/obj.
As we don’t want to backup and mirror all the contents of /usr/obj (which should be considered temporary data of no value), we can create it’s own Hammer pseudo-filesystem for it, which allows to treat it separately for pruning and mirroring operations.
rmdir /usr/obj cd /usr hammer pfs-master obj
As Hammer’s pseudo-filesystems are represented by symlinks, it isn’t possible to assign access rights to them or flags like nohistory, but there should be a workaround using null mounts:
# We rename the PFS to obj.pfs mv /usr/obj /usr/obj.pfs # We create a directory /usr/obj, which holds the permissions mkdir /usr/obj # Finally we null mount /usr/obj.pfs on top of /usr/obj mount_null /usr/obj.pfs /usr/obj
We’d probably like to do the same for /tmp:
rm -rf /tmp mkdir /tmp chmod 1777 /tmp # sticky flags for /tmp cd / hammer pfs-master tmp.pfs mount_null /tmp.pfs /tmp
Enjoy DragonFly and Hammer!
Last week I installed FreeBSD 7.0-RC2. Now that FreeBSD 7.0-RELEASE is available, I tried to update using freebsd-update:
freebsd-update -r 7.0-RELEASE upgrade freebsd-update install reboot
But I always got error message during “install” like:
chflags: ///etc/mail/freebsd.cf: Operation not supported
So maybe it only occurs if you’ve installed FreeBSD on a ZFS filesystem, which doesn’t possibly support those file flags like noschg. The simple solution to overcome this was:
mv /bin/chflags /bin/chflags.old cat > /bin/chflags #!/bin/sh /bin/chflags.old $@ exit 0 ^D chmod +x /bin/chflags
After that I retried:
freebsd-update install reboot
And it works!
A few seconds ago I tried out DragonFly’s HAMMER filesystem myself. All you need is to compile a new kernel with “options HAMMER” and the userland programs hammer, newfs_hammer and mount_hammer, plus a free partition (/dev/ad0s1d in my case). Here we go:
# newfs_hammer -L Home /dev/ad0s1d Volume 0 DEVICE /dev/ad0s1d size 4.00GB --------------------------------------------- 1 volume total size 4.00GB cluster-size: 16.00MB max-volume-size: 0.50TB max-filesystem-size: 16384.00TB boot-area-size: 16.00MB memory-log-size: 16.00MB allocate cluster id=10a9b3f3f83e5fd0 0@02040000 cluster 0 has 1024 buffers # mount_hammer /dev/ad0s1d /mnt # echo "hallo" > /mnt/abc # hammer now 0x47910e84 # echo "hallo leute" > /mnt/abc # cat /mnt/abc@@0x47910e73 hallo # cat /mnt/abc hallo leute
As you can see, at first I create a new HAMMER filesystem on partition ad0s1d using newfs_hammer. In the next step I mount that filesystem onto /mnt using mount_hammer. Then I create a file on it which contains “hallo”. After I acquire the current timestamp (hammer now) I overwrite the file with content “hallo leute”. But the old content is not gone. I can still access it using the @@timestamp notation. Nice ;-)
ZFS (Zeta filesystem), according to Sun the "last word in filesystems", is definitively a great piece of software. It frees us sysadmins from the burden to plan upfront how to partition a disk, makes backups so much easier due to snapshots, allows space efficient storage thanks to optional compression support and is very safe in terms of data corruption (at least it detects corruption) and it’s quickly online again after a crash. But is it really the "last word in filesystems"?
Well, it will be successful, no question. FreeBSD has it, Solaris of course, Apple too, and it’s planned for NetBSD. Don’t know about Linux (licensing issues) and Windows.
DragonFly’s HammerFS is progressing well and will be available in an alpha version with the next release of DragonFlyBSD which is planned to be out around by mid of February. First interest was shown to port it over to NetBSD (in userland using puffs), which is great news because NetBSD is still more widely used than DragonFly and just runs on more hardware (at least DragonFly still doesn’t run on my new laptop).
So what’s so great about HammerFS?
You don’t need to manually take snapshots. The system is able keep a history of every change according to a retention policy that you can specify.
You can mount a filesystem or access a file as-of a specific point in time (in the past). What was the content of this file yesterday? Last week? Even atime and mtime is tracked.
The major goal of HammerFS is clustering. A HammerFS filesystem can be shared by a bunch of machines. No need of a single master. Based on a Chorum protocol. (this is currently not implemented)
Backups made easy
Three things make backups easy. First, the as-of feature (eases taking tarballs). Second, journaling. And third, clustering. Do you need to backup at all if three or more machines have an identical copy of your filesystem including history? Think about the hurdles when restoring a traditional backup. With HammerFS you simply replace the harddisk and reconnect the machine to the network. Done! Think about a malicious or stupid (like me sometimes) user deleting files accidentially, right before the next backup is taken. You will loose your work, unless you use something like HammerFS. ZFS doesn’t help here because it doesn’t has the infinite snapshot feature. While I trust harddisks and RAIDs, I don’t trust users!
HammerFS will allow you to open a file in DB mode. I don’t know how exactly this works out, but you will be able to use a file as a simple key/value database. No need for special libraries and probably quite fast.
The last couple of days I played with DragonFlyBSD. Except some minor issues (like fdisk completely hangs my BIOS :) it works pretty well. I was mostly interested in it's journaling provided by jscan. You can read more here (or follow the direct link to my mail).
I like DragonFly due to it's simplicity. Currently I am running FreeBSD 7.0-CURRENT. It has tons of features, but many of them are not well thought out. For example disk or file-system labeling through glabel confuses me a lot and doesn't work as expected. Normally it should show up the labeled disks/file systems in /dev/ufs/name or /dev/label/name, so that you can easily mount them without knowing the exact device number or slice. The concept is nice, but I think the implementation is way too complex (or too generic). Or I am just to stupid to understand. But I think the whole geom stuff (this framework is called geom) is not yet 100% stable.
Matthew Dillon (DragonFly) in now working on getting user land file systems working. Well, as I know him, this will be a comparatively easy task, as all the former work he did on DragonFly will help him a lot! So for example, the whole kernel can already run in user land. Or the work he did on syslink, which is a message passing interface.
Most of the features FreeBSD provides I don't need at all. What I really like is if the basic kernel infrastructure is very clean and understandable. This leads to a stable kernel and will make it easy to extend it.