Building a new Lustre Filesystem

Here are the quick and dirty steps to create a new Lustre filesystem for testing purposes. I use this at times to test out commands and test benchmarking tools, not to test performance, but to ensure they operate correctly.

This is a simple test environment on a single system with a single physical disk. Lustre is designed for scalability so these commands can be run on multiple machines and across many disks to ensure that a bottleneck does not occur in larger environments. The purposes of this is to generate a working Lustre filesystem for testing and sandbox work.

This set of directions assumes you have compiled and installed both the Lustre kernel and the Lustre userspace bits. Check my previous blog posting for how to complete those items if necessary. This also assumes that you have a spare physical disk that can be partitioned to create the various components of the filesystem. In the example case below I created the filesystem within a Xen virtual machine.

1) Create a script to partition the disk that will be used for testing (using /dev/xvdb for example purposes)
#!/bin/sh
sfdisk /dev/xvdb << EOF
,1ooo,L
,1000,L
,2000,L
,2000,L
EOF

2) Format the MGS Partition
- mkfs.lustre --mgs --reformat /dev/xvdb1
- mkdir -p /mnt/mgs
- mount -t lustre /dev/xvdb1 /mnt/mgs

3) Format the MDT Partition
- mkfs.lustre --mdt --reformat --mgsnid=127.0.0.1 --fsname=lusfs01 /dev/xvdb2
- mkdir -p /mnt/lusfs01/mdt
- mount -t lustre /dev/xvdb2 /mnt/lusfs01/mdt

4) Format the First OST Partition
- mkfs.lustre --ost --reformat --mgsnid=127.0.0.1 --fsname=lusfs01 /dev/xvdb3
- mkdir -p /mnt/lusfs01/ost00
- mount -t lustre /dev/xvdb3 /mnt/lusfs01/ost00

5) Format the Second OST Partition
- mkfs.lustre --ost --reformat --mgsnid=127.0.0.1 --fsname=lusfs01 /dev/xvdb4
- mkdir -p /mnt/lusfs01/ost01
- mount -t lustre /dev/xvdb4 /mnt/lusfs01/ost01

6) Mount the client view of the filesystem
- mkdir -p /mnt/lusfs01/client
- mount -t lustre 127.0.0.1@tcp0:/lusfs01 /mnt/lusfs01/client

At this point you should be able to do an ls, touch, rm or any other standard file manipulation command on files in /mnt/lusfs01/client.