實驗拓撲圖:
實驗原理:
通過RHCS集群套件搭建GFS2集群文件系統,保證不同節點能夠同時對GFS2集群文件系統進行讀取和寫入,其次通過multipath實現node和FC,FC和Share Storage之間的多路冗余,最後實現存儲的mirror復制達到高可用。
GFS2:全局文件系統第二版,GFS2是應用最廣泛的集群文件系統。它是由紅帽公司開發出來的,允許所有集群節點並行訪問。元數據通常會保存在共享存儲設備或復制存儲設備的一個分區裡或邏輯卷中。
實驗環境:
[root@storage1 ~]# uname -r 2.6.32-279.el6.x86_64 [root@storage1 ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.3 (Santiago) [root@storage1 ~]# /etc/rc.d/init.d/iptables status iptables: Firewall is not running. [root@storage1 ~]# getenforce Disabled
[root@manager ~]# ssh-keygen \\生成公鑰和私鑰 Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): …… [root@manager ~]# for i in {1..6}; do ssh-copy-id -i 192.168.100.17$i; done \\將公鑰傳輸到各節點/root/.ssh/目錄下 [email protected]'s password: Now try logging into the machine, with "ssh '192.168.100.171'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting ..…… [root@manager ~]# ssh node1 \\測試登錄 Last login: Sat Jun 8 17:58:51 2013 from 192.168.100.31
[root@storage1 ~]# ifconfig eth0 | grep "inet addr" | awk -F[:" "]+ '{ print $4 }' 192.168.100.171 [root@storage1 ~]# ifconfig eth1 | grep "inet addr" | awk -F[:" "]+ '{ print $4 }' 192.168.200.171
[root@manager ~]# cat /etc/hosts 127.0.0.1 localhost localhost.rsyslog.org 192.168.100.102 manager manager.rsyslog.org 192.168.100.171 storage1 storage1.rsyslog.org 192.168.200.171 storage1 storage1.rsyslog.org 192.168.100.172 storage2 storage2.rsyslog.org 192.168.200.172 storage2 storage2.rsyslog.org 192.168.100.173 node1 node1.rsyslog.org 192.168.200.173 node1 node1.rsyslog.org 192.168.100.174 node2 node2.rsyslog.org 192.168.200.174 node2 node2.rsyslog.org 192.168.100.175 node3 node3.rsyslog.org 192.168.200.175 node3 node3.rsyslog.org 192.168.100.176 node4 node4.rsyslog.org 192.168.200.176 node4 node4.rsyslog.org [root@manager ~]# for i in {1..6}; do scp /etc/hosts 192.168.100.17$i:/etc/ ; done hosts 100% 591 0.6KB/s 00:00 hosts 100% 591 0.6KB/s 00:00 hosts 100% 591 0.6KB/s 00:00 hosts 100% 591 0.6KB/s 00:00 hosts 100% 591 0.6KB/s 00:00 hosts 100% 591 0.6KB/s 00:00
[root@manager ~]# cat /etc/yum.repos.d/rhel-gfs2.repo [rhel-cdrom] name=RHEL6U3-cdrom baseurl=file:///media/cdrom enabled=1 gpgcheck=0 [rhel-cdrom-HighAvailability] name=RHEL6U3-HighAvailability baseurl=file:///media/cdrom/HighAvailability enabled=1 gpgcheck=0 [rhel-cdrom-ResilientStorage] name=RHEL6U3-ResilientStorage baseurl=file:///media/cdrom/ResilientStorage enabled=1 gpgcheck=0 [rhel-cdrom-LoadBalancer] name=RHEL6U3-LoadBalancer baseurl=file:///media/cdrom/LoadBalancer enabled=1 gpgcheck=0 [rhel-cdrom-ScalableFileSystem] name=RHEL6U3-ScalableFileSystem baseurl=file:///media/cdrom/ScalableFileSystem enabled=1 gpgcheck=0 [root@manager ~]# for i in {1..6}; do scp /etc/yum.repos.d/rhel-gfs2.repo 192.168.100.17$i:/etc/yum.repos.d ; done rhel-gfs2.repo 100% 588 0.6KB/s 00:00 rhel-gfs2.repo 100% 588 0.6KB/s 00:00 rhel-gfs2.repo 100% 588 0.6KB/s 00:00 rhel-gfs2.repo 100% 588 0.6KB/s 00:00 rhel-gfs2.repo 100% 588 0.6KB/s 00:00 rhel-gfs2.repo 100% 588 0.6KB/s 00:00 [root@manager ~]# for i in {1..6}; do ssh 192.168.100.17$i "yum clean all && yum makecache"; done Loaded plugins: product-id, security, subscription-manager Updating certificate-based repositories. Unable to read consumer identity
[root@manager ~]# yum install luci –y [root@manager ~]# /etc/rc.d/init.d/luci start \\生成以下信息,說明配置成功,注意:安裝luci會安裝很多python包,python包盡量采用光盤自帶的包,否則啟動luci會出現報錯現象。 Adding following auto-detected host IDs (IP addresses/domain names), corresponding to `manager.rsyslog.org' address, to the configuration of self-managed certificate `/var/lib/luci/etc/cacert.config' (you can change them by editing `/var/lib/luci/etc/cacert.config', removing the generated certificate `/var/lib/luci/certs/host.pem' and restarting luci): (none suitable found, you can still do it manually as mentioned above) Generating a 2048 bit RSA private key writing new private key to '/var/lib/luci/certs/host.pem' 正在啟動 saslauthd: [確定] Start luci... [確定] Point your web browser to https://manager.rsyslog.org:8084 (or equivalent) to access luci [root@manager ~]# for i in {1..4}; do ssh node$i "yum install ricci -y"; done [root@manager ~]# for i in {1..4}; do ssh node$i "chkconfig ricci on && /etc/rc.d/init.d/ricci start"; done [root@manager ~]# for i in {1..4}; do ssh node$i "echo '123.com' | passwd ricci --stdin"; done \\ricci設置密碼,在Conga web頁面添加節點的時候需要輸入ricci密碼。 更改用戶 ricci 的密碼 。 passwd: 所有的身份驗證令牌已經成功更新。
安裝過程如下:
以下為安裝完成之後,所有節點狀態
點開一個節點,可以看到這個節點上所有相關服務都處於運行狀態。
登錄任意一個節點查看各服務的開機啟動情況,為2-5級別自動啟動。
[root@manager ~]# ssh node1 "chkconfig --list | grep cman" cman 0:關閉 1:關閉 2:啟用 3:啟用 4:啟用 5:啟用 6:關閉 [root@manager ~]# ssh node1 "chkconfig --list | grep rgmanager" rgmanager 0:關閉 1:關閉 2:啟用 3:啟用 4:啟用 5:啟用 6:關閉 [root@manager ~]# ssh node1 "chkconfig --list | grep clvmd" clvmd 0:關閉 1:關閉 2:啟用 3:啟用 4:啟用 5:啟用 6:關閉 [root@node2 ~]# cat /etc/cluster/cluster.conf \\查看各節點集群配置信息,各節點這部分必須一樣。 <?xml version="1.0"?> <cluster config_version="1" name="rsyslog"> <clusternodes> <clusternode name="node1" nodeid="1"/> <clusternode name="node2" nodeid="2"/> </clusternodes> <cman expected_votes="1" two_node="1"/> <fencedevices/> <rm/> </cluster> [root@node2 ~]# clustat \\查看集群節點狀態(可以通過 cluster -i 1 動態查看變化狀態) Cluster Status for rsyslog @ Sat Jun 8 00:03:40 2013 Member Status: Quorate Member Name ID Status ------ ---- ---- ------ node1 1 Online node2 2 Online, Local node3 3 Online