In the previous section we presented a convenient way to deploy a shared storage using NFS. For better transactions performance, you could deploy MooseFS instead. MooseFS (Moose File System) is a shared file system ; it implements the same rough concepts of shared storage solutions - such as Ceph, Lustre or even GlusterFS. Main concepts A metadata server (MDS), also called master server, which manages the file repartition, their access and the namespace. A metalogger server (MLS) which backs up the MDS logs, including, objects, chunks, sessions and object metadata A chunk server (CSS) which store the data as chunks and replicate them across the chunkservers A client, which talks with the MDS and interact with the CSS. MooseFS clients manage MooseFS filesystem using FUSE For more informations, please see the Official project website Our setup will be made the following way: Two compute nodes running both MooseFS chunkserver and client services. One MooseFS master server, running the metadata service. One MooseFS slave server, running the metalogger service. For that particular walkthrough, we will use the following network schema: 10.0.10.15 for the MooseFS metadata server admin IP 10.0.10.16 for the MooseFS metadata server main IP 10.0.10.17 for the MooseFS metalogger server admin IP 10.0.10.18 for the MooseFS metalogger server main IP 10.0.10.19 for the MooseFS first chunkserver IP 10.0.10.20 for the MooseFS second chunkserver IP

Both components could be run anywhere , as long as the MooseFS chunkservers can reach the MooseFS master server. In our deployment, both MooseFS master and slave run their services inside a virtual machine ; you just need to make sure to allocate enough memory to the MooseFS metadata server, all the metadata being stored in RAM when the service runs. Hosts entry configuration In the /etc/hosts add the following entry : 10.0.10.16 mfsmaster Required packages Install the required packages by running the following commands : $ apt-get install zlib1g-dev python pkg-config $ yum install make automake gcc gcc-c++ kernel-devel python26 pkg-config User and group creation Create the adequate user and group : $ groupadd mfs && useradd -g mfs mfs Download the sources Go to the MooseFS download page and fill the download form in order to obtain your URL for the package. Extract and configure the sources Extract the package and compile it : $ tar -zxvf mfs-1.6.25.tar.gz && cd mfs-1.6.25 For the MooseFS master server installation, we disable from the compilation the mfschunkserver and mfsmount components : $ ./configure --prefix=/usr --sysconfdir=/etc/moosefs --localstatedir=/var/lib --with-default-user=mfs --with-default-group=mfs --disable-mfschunkserver --disable-mfsmount$ make && make install Create configuration files We will keep the default settings, for tuning performance, you can read the MooseFS official FAQ $ cd /etc/moosefs $ cp mfsmaster.cfg.dist mfsmaster.cfg $ cp mfsmetalogger.cfg.dist mfsmetalogger.cfg $ cp mfsexports.cfg.dist mfsexports.cfg In /etc/moosefs/mfsexports.cfg edit the second line in order to restrict the access to our private network: 10.0.10.0/24 / rw,alldirs,maproot=0 Create the metadata file : $ cd /var/lib/mfs && cp metadata.mfs.empty metadata.mfs Power up the MooseFS mfsmaster service You can now start the mfsmaster and mfscgiserv deamons on the MooseFS metadataserver (The mfscgiserv is a webserver which allows you to see via a web interface the MooseFS status realtime) : $ /usr/sbin/mfsmaster start && /usr/sbin/mfscgiserv startOpen the following url in your browser : http://10.0.10.16:9425 to see the MooseFS status page Power up the MooseFS metalogger service $ /usr/sbin/mfsmetalogger start

In the first part, we will install the last version of FUSE, and proceed to the installation of the MooseFS chunk and client in the second part. Installing FUSE Required package $ apt-get install util-linux $ yum install util-linux Download the sources and configure them For that setup we will retrieve the last version of fuse to make sure every function will be available : $ wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9.1/fuse-2.9.1.tar.gz && tar -zxvf fuse-2.9.1.tar.gz && cd fuse-2.9.1$ ./configure && make && make install Installing the MooseFS chunk and client services For installing both services, you can follow the same steps that were presented before (Steps 1 to 4) : Hosts entry configuration Required packages User and group creation Download the sources Extract and configure the sources Extract the package and compile it : $ tar -zxvf mfs-1.6.25.tar.gz && cd mfs-1.6.25 For the MooseFS chunk server installation, we only disable from the compilation the mfsmaster component : $ ./configure --prefix=/usr --sysconfdir=/etc/moosefs --localstatedir=/var/lib --with-default-user=mfs --with-default-group=mfs --disable-mfsmaster$ make && make install Create configuration files The chunk servers configuration is relatively easy to setup. You only need to create on every server directories that will be used for storing the datas of your cluster. $ cd /etc/moosefs $ cp mfschunkserver.cfg.dist mfschunkserver.cfg $ cp mfshdd.cfg.dist mfshdd.cfg $ mkdir /mnt/mfschunks{1,2} && chown -R mfs:mfs /mnt/mfschunks{1,2} Edit /etc/moosefs/mfhdd.cfg and add the directories you created to make them part of the cluster: # mount points of HDD drives # #/mnt/hd1 #/mnt/hd2 #etc. /mnt/mfschunks1 /mnt/mfschunks2 Power up the MooseFS mfschunkserver service $ /usr/sbin/mfschunkserver start

You can now access your cluster space from the compute node, (both acting as chunkservers) : $ mfsmount /var/lib/nova/instances -H mfsmaster mfsmaster accepted connection with parameters: read-write,restricted_ip ; root mapped to root:root $ mount/dev/cciss/c0d0p1 on / type ext4 (rw,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) none on /sys type sysfs (rw,noexec,nosuid,nodev) fusectl on /sys/fs/fuse/connections type fusectl (rw) none on /sys/kernel/debug type debugfs (rw) none on /sys/kernel/security type securityfs (rw) none on /dev type devtmpfs (rw,mode=0755) none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) none on /dev/shm type tmpfs (rw,nosuid,nodev) none on /var/run type tmpfs (rw,nosuid,mode=0755) none on /var/lock type tmpfs (rw,noexec,nosuid,nodev) none on /var/lib/ureadahead/debugfs type debugfs (rw,relatime) mfsmaster:9421 on /var/lib/nova/instances type fuse.mfs (rw,allow_other,default_permissions) You can interact with it the way you would interact with a classical mount, using build-in linux commands (cp, rm, etc...). The MooseFS client has several tools for managing the objects within the cluster (set replication goals, etc..). You can see the list of the available tools by running $ mfs <TAB> <TAB> mfsappendchunks mfschunkserver mfsfileinfo mfsgetgoal mfsmount mfsrsetgoal mfssetgoal mfstools mfscgiserv mfsdeleattr mfsfilerepair mfsgettrashtime mfsrgetgoal mfsrsettrashtime mfssettrashtime mfscheckfile mfsdirinfo mfsgeteattr mfsmakesnapshot mfsrgettrashtime mfsseteattr mfssnapshot You can read the manual for every command. You can also see the online help Add an entry into the fstab file In order to make sure to have the storage mounted, you can add an entry into the /etc/fstab on both compute nodes : mfsmount /var/lib/nova/instances fuse mfsmaster=mfsmaster,_netdev 0 0