GFS distributed File system Cluster Project

GFS distributed file system cluster project cluster environment

Volume type volume name volume type space size Brickdis-volume distributed volume 40Gnode1 (/ b1), node2 (/ b1) stripe-volume stripe volume 40Gnode1 (/ C1), node2 (/ C1) rep-volume replication volume 20Gnode3 (/ b1), node4 (/ b1) dis-stripe distributed stripe volume 40Gnode1 (/ D1), node2 (/ D1), node3 (/ D1), node4 (/ D1) dis-rep distributed replication volume 20Gnode1 (/ E1), node2 (/ E1), node3 (/ E1), Node4 (/ E1) experiment preparation 1. Add 4 disks to each of the four servers

2. Modify the name of the server

Modified to node1, node2, node3, node4 respectively

[root@localhost ~] # hostnamectl set-hostname node1 [root@localhost ~] # su3, format and mount the disks on four servers

Here we use scripts to perform mounts

# enter the opt directory [root@node1 ~] # cd / opt# disk formatting and mounting script [root@node1 opt] # vim a.shallows! / bin/bashecho "the disks exist list:" fdisk-l | grep 'disk / dev/sd [a murz]' echo "= =" PS3= "chose which disk you want to create:" select VAR in `disk / dev/sd* | grep-o'sd [BMZ]'| Uniq`quitdo case $VAR in sda) fdisk-l / dev/sda break Sd [bmerz]) # create partitions echo "n p w" | fdisk / dev/$VAR # make filesystem mkfs.xfs-I size=512 / dev/$ {VAR} "1" & > / dev/null # mount the system mkdir-p / data/$ {VAR} "1" & > / dev/null echo-e "/ dev/$ { VAR} "1" / data/$ {VAR} "1" xfs defaults 0\ n "> > / etc/fstab mount-a & > / dev/null break ; quit) break;; *) echo "wrong disk,please check again";; esacdone# gives script execution permission [root@node1 opt] # chmod + x a.sh

Push the script to the other three servers via scp

Scp a.sh root@192.168.45.134:/optscp a.sh root@192.168.45.130:/optscp a.sh root@192.168.45.136:/opt executes the script on four servers and completes the

This is just a sample.

[root@node1 opt] #. / a.shthe disks exist list:===1) sdb2) sdd4) sde5) quitchose which disk you want to create:1 / / Select the disk Welcome to fdisk (util-linux 2.23.2) .Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition tableBuilding a new DOS disklabel with disk identifier 0x37029e96.Command (m for help): Partition type: P primary (0 primary, 0 extended) 4 free) e extendedSelect (default p): Partition number (1-4, default 1): First sector (2048-41943039, default 2048): Using default value 2048Last sector, + sectors or + size {K Magi M ·G} (2048-41943039, default 41943039): Using default value 41943039Partition 1 of type Linux and of size 20 GiB is setCommand (m for help): The partition table has been altered calling ioctl () to re-read partition table.Syncing disks. Check the mount on the four servers

4. Set up the hosts file

Modify it on the first node1

# add vim / etc/hosts192.168.45.133 node1192.168.45.130 node2192.168.45.134 node3192.168.45.136 node4 at the end of the file

Push hosts files to other servers and clients through scp

# push the / etc/hosts file to another host [root@node1 opt] # scp / etc/hostsroot@192.168.45.130: / etc/hostsroot@192.168.45.130's password: hosts 100% 242 23.6KB/s 00:00 [root@node1 opt] # scp / etc/hostsroot@192.16 8.45.134:/etc/hostsroot@192.168.45 .134s password: hosts 100242 146.0KB/s 00:00 [root@node1 opt] # scp / etc/hostsroot@192.168.45.136: / etc/hostsroot@192.168.45.136's password: hosts

Check the push status on other servers

Turn off the firewalls of all servers and clients [root@node1 ~] # systemctl stop firewalld.service [root@node1 ~] # setenforce 0 build yum warehouses on clients and servers # enter the yum file path [root@node1 ~] # cd / etc/yum.repos.d/# create an empty folder [root@node1 yum.repos.d] # mkdir abc# and move all CentOS- files to abc [root@node1 yum.repos.d] # mv CentOS-* abc# create private yum source [root@node1 yum.repos.d] # vim GLFS.repo [demo] name=demobaseurl= http://123.56.134.27/demogpgcheck=0enable=1[gfsrepo]name=gfsrepobaseurl=http://123.56.134.27/gfsrepogpgcheck=0enable=1# reload yum source [root@node1 yum.repos.d] # yum list install the necessary software package [root@node1 yum.repos.d] # yum-y install glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma

Do the same thing on the other three stations

Start glusterd on four servers and set it to self-boot [root@node1 yum.repos.d] # systemctl start glusterd.service [root@node1 yum.repos.d] # systemctl enable glusterd.service to add node information [root@node1 yum.repos.d] # gluster peer probe node2peer probe: success. [root@node1 yum.repos.d] # gluster peer probe node3peer probe: success. [root@node1 yum.repos.d] # gluster peer probe node4peer probe: success.

View node information on other servers

[root@node1 yum.repos.d] # gluster peer status

Create distributed Volume # create distributed Volume [root@node1 yum.repos.d] # gluster volume create dis-vol node1:/data/sdb1 node2:/data/sdb1 force# check Information [root@node1 yum.repos.d] # gluster volume info dis-vol# View distributed existing Volume [root@node1 yum.repos.d] # gluster volume list# Boot Volume [root@node1 yum.repos.d] # gluster volume start dis-vol

Mount on the client # Recursive create mount point [root@manager yum.repos.d] # mkdir-p / text/dis# mount the volume you just created to the mount point you just created [root@manager yum.repos.d] # mount.glusterfs node1:dis-vol / text/dis

Create stripe volume # create volume [root@node1 yum.repos.d] # gluster volume create stripe-vol stripe 2 node1:/data/sdc1 node2:/data/sdc1 force# view existing volume [root@node1 yum.repos.d] # gluster volume listdis-volstripe-vol# boot stripe volume [root@node1 yum.repos.d] # gluster volume start stripe-vol volume start: stripe-vol: success mount on client # create mount point [root@manager yum.repos.d] # mkdir / text/strip# mount stripe volume [root@manager yum.repos.d] # mount.glusterfs node1:/stripe-vol / text/strip/

Check the mount status

Create replication volume # create replication volume [root@node1 yum.repos.d] # gluster volume create rep-vol replica 2 node3:/data/sdb1 node4:/data/sdb1 forcevolume create: rep-vol: success: please start the volume to access data# enable replication volume [root@node1 yum.repos.d] # gluster volume start rep-vol volume start: rep-vol: success

Prevent the client from replicating the volume

[root@manager yum.repos.d] # mkdir / text/rep [root@manager yum.repos.d] # mount.glusterfs node3:rep-vol / text/rep

View mount

Create distributed stripe # create distributed stripe volume [root@node1 yum.repos.d] # gluster volume create dis-stripe stripe 2 node1:/data/sdd1 node2:/data/sdd1 node3:/data/sdd1 node4:/data/sdd1 forcevolume create: dis-stripe: success: please start the volume to access data# launch distributed stripe volume [root@node1 yum.repos.d] # gluster volume start dis-stripe volume start: dis-stripe: success

Mount on the client

[root@manager yum.repos.d] # mkdir / text/dis-strip [root@manager yum.repos.d] # mount.glusterfs node4:dis-stripe / text/dis-strip/ create distributed replication volume # create distributed replication volume [root@node2 yum.repos.d] # gluster volume create dis-rep replica 2 node1:/data/sde1 node2:/data/sde1 node3:/data/sde1 node4:/data/sde1 forcevolume create: dis-rep: success: please start the volume to access data # enable replication volume [root@node2 yum.repos.d] # gluster volume start dis-rep volume start: dis-rep: success# View existing volume [root@node2 yum.repos.d] # gluster volume listdis-repdis-stripedis-volrep-volstripe-vol mount [root@manager yum.repos.d] # mkdir / text/dis-rep [root@manager yum.repos.d] # mount.glusterfs node3:dis-rep / text/dis-rep/ on the client

View mount

-above we have finished creating and mounting the volume-

Now let's test the volume and first create five 40m files [root@manager yum.repos.d] # dd if=/dev/zero of=/demo1.log bs=1M count=4040+0 records in40+0 records out41943040 bytes (42 MB) copied, 0.0175819 s, 0.0175819 GB/s [root@manager yum.repos.d] # dd if=/dev/zero of=/demo2.log bs=1M count=4040+0 records in40+0 records out41943040 bytes (42 MB) copied, 0.269746 s on the client. 155 MB/s [root@manager yum.repos.d] # dd if=/dev/zero of=/demo3.log bs=1M count=4040+0 records in40+0 records out41943040 bytes (42 MB) copied, 0.34134 s, 123 MB/s [root@manager yum.repos.d] # dd if=/dev/zero of=/demo4.log bs=1M count=4040+0 records in40+0 records out41943040 bytes (42 MB) copied, 1.55335 s, 27.0 MB/s [root@manager yum.repos.d] # dd if=/dev/zero of=/demo5.log bs=1M count=4040+0 records in40+0 records out41943040 bytes (42 MB) copied 1.47974 s 28.3 MB/s then copy five files to different volumes [root@manager yum.repos.d] # cp / demo* / text/dis [root@manager yum.repos.d] # cp / demo* / text/strip [root@manager yum.repos.d] # cp / demo* / text/rep [root@manager yum.repos.d] # cp / demo* / text/dis-strip [root@manager yum.repos.d] # cp / demo* / text / dis-rep View Volume contents View distributed Volume

View stripe volum

View replication Volum

View distributed stripe volum

View distributed replication volum

Failure testing shuts down node2 server observation result [root@manager yum.repos.d] # ls / text/dis dis-rep dis-strip rep strip [root@manager yum.repos.d] # ls / text/disdemo1.log demo2.log demo3.log demo4.log [root@manager yum.repos.d] # ls / text/dis-repdemo1.log demo2.log demo3.log demo4.log demo5.log [root@manager yum.repos.d] # ls / text/dis-strip/demo5.log [root@manager yum.repos.d] # ls / text/rep/demo1.log demo2.log demo3.log demo4.log demo5.log [root@manager yum.repos.d] # ls / text/strip/ [root@manager yum.repos.d] #

The results show that:

Distribution volume missing demo5.log file stripe volume unable to access replication volume normal access distributed stripe volume missing file distributed replication volume normal access delete volume

To delete a volume, you need to stop the volume. When you delete a volume, the volume group must be on.

# stop volume [root@manager yum.repos.d] # gluster volume delete dis-vol# delete volume [root@manager yum.repos.d] # gluster volume delete dis-vol access control # deny only [root@manager yum.repos.d] # gluster volume set dis-vol auth.reject 192.168.45.13 only allow [root@manager yum.repos.d] # gluster volume set dis-vol auth.allow 192.168.45.133