Linux部署NFS+Keepalived高可用集群

NFS简介

NFS（Network File System）是一个网络文件系统，是Linux系统直接支持文件共享的一直文件系统，它允许网络中的计算机之间通过TCP/IP网络共享资源。在NFS的应用中，本地NFS的客户端应用可以透明地读写位于远端NFS服务器上的文件，就像访问本地文件一样。一般NFS为单击部署，而NFS服务器主要用于存放企业重要数据，此时为了能保证数据的安全可靠，需要对NFS服务器实现同步+keepalived高可用满足企业业务需求，如下为NFS+keepalived高可用架构实现步骤：

系统环境：CentOS 6.9

架构：LAP+Discuz+MYSQL+NFS+Keepalived

LAP：192.168.8.2

LAP1：192.168.8.3

MYSQL：192.168.8.5

NFS+Keepalived（主）：192.168.8.9

NFS+Keepalived（备）：192.168.8.10

LAP端跟LAP1端，执行以下命令

[root@localhost ~]# yum -y install httpd httpd-devel php php-mysql php-devel nfs-utils

MYSQL端，执行以下命令

[root@localhost ~]# yum -y install mysql mysql-devel mysql-server

NFS（主）端和NFS（备）端，执行以下命令

[root@localhost ~]# yum -y install gcc gcc-c++ wget nfs* unzip

NFS（MASTER）端

1）下载Discuz论坛包

[root@localhost ~]# wget -c http://mirrors.yangxingzhen.com/discuz/Discuz_X3.4_SC_UTF8.zip

2）解压论坛包

[root@localhost ~]# unzip Discuz_X3.4_SC_UTF8.zip

3）创建共享目录，拷贝论坛代码到共享目录

[root@localhost ~]# mkdir -p /data/www

[root@localhost ~]# cd upload/

[root@localhost upload]# cp -a * /data/www

4）创建NFS配置文件，启动NFS服务

[root@localhost upload]# echo "/data/www *(rw,sync,no_root_squash)" >/etc/exports

[root@localhost upload]# cat /etc/exports

[root@localhost upload]# service rpcbind start

[root@localhost upload]# service nfs start

MySQL端

1）启动MYSQL服务

[root@localhost ~]# service mysqld start

2）进入MYSQL命令行终端

[root@localhost ~]# mysql

3）创建Discuz库并授权

mysql> create database discuz charset=utf8;

mysql> grant all on discuz.* to bbs@'192.168.8.2' identified by '123456';

mysql> grant all on discuz.* to bbs@'192.168.8.3' identified by '123456';

mysql> flush privileges;

LAP端

1）查看NFS服务器挂载情况

[root@localhost ~]# showmount -e 192.168.8.9

2）挂载NFS服务器共享目录到Apache发布目录（/var/www/html）

[root@localhost ~]# mount -t nfs 192.168.8.9:/data/www /var/www/html

[root@localhost ~]# df -h

3）启动Apache服务

[root@localhost ~]# service httpd start

4）打开浏览器访问本机IP，配置Discuz论坛

5）点击我同意，出现如下图

6）授权

[root@localhost ~]# cd /var/www/html/

[root@localhost html]# chmod o+w config/ data/ uc_* -R

7）点击下一步

8）输入数据库服务器地址、数据库名、数据库用户名、数据库密码，点击下一步

9）等待安装

至此，论坛搭建成功。

LAP1端

只需使用mount命令即可

1）挂载NFS服务器共享目录到Apache发布目录（/var/www/html）

[root@localhost ~]# mount -t nfs 192.168.8.9:/data/www /var/www/html

[root@localhost ~]# df –h

2）启动Apache服务

[root@localhost ~]# service httpd start

3）打开浏览器访问本机IP，如下图

NFS+Keepalived（主）部署

1）安装keepalived所需要的依赖包

[root@localhost ~]# yum -y install openssl openssl-devel popt-devel kernel-devel gcc gcc-c++ libnl libnl-devel libnfnetlink-devel popt wget

2）下载keepalived软件包

[root@localhost ~]# wget -c http://mirrors.yangxingzhen.com/keepalived/keepalived-1.2.15.tar.gz

3）解压

[root@localhost ~]# tar zxf keepalived-1.2.15.tar.gz

[root@localhost ~]# cd keepalived-1.2.15

4）预编译keepalived

[root@localhost ~]# ./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-696.16.1.el6.x86_64

5）编译、安装keepalived

[root@localhost ~]# make && make install

6）创建软连接，拷贝相应的文件

[root@localhost ~]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin

[root@localhost ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig

[root@localhost ~]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d

[root@localhost ~]# chkconfig --add keepalived

[root@localhost ~]# chkconfig keepalived on

[root@localhost ~]# mkdir /etc/keepalived

[root@localhost ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

7）编辑keepalived（主）配置文件

# 编辑vim /etc/keepalived/keepalived.conf

[root@localhost ~]# vim /etc/keepalived/keepalived.conf

# 配置内容如下

! Configuration File for keepalived
global_defs {
notification_email {
675583110@qq.com
}
notification_email_from 675583110@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nfs {
script "/data/shell/check_nfs.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
mcast_src_ip 192.168.8.9
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.8.99
}
track_script {
chk_nfs
}
}

[root@localhost ~]# service keepalived start

NFS+Keepalived（备）部署

1）创建共享目录

[root@localhost ~]# mkdir -p /data/www

2）拷贝NFS服务器（主）论坛代码

[root@localhost ~]# scp -r root@192.168.8.9:/data/www/* /data/www

3）创建NFS配置文件

[root@localhost ~]# echo "/data/www *(rw,sync,no_root_squash)" >/etc/exports

4）安装keepalived所需要的依赖包

[root@localhost ~]# yum -y install openssl openssl-devel popt-devel kernel-devel gcc gcc-c++ libnl libnl-devel libnfnetlink-devel popt wget

5）下载keepalived软件包

[root@localhost ~]# wget -c http://mirrors.yangxingzhen.com/keepalived/keepalived-1.2.15.tar.gz

6）解压

[root@localhost ~]# tar zxf keepalived-1.2.15.tar.gz

[root@localhost ~]# cd keepalived-1.2.15

7）预编译keepalived

[root@localhost ~]# ./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-696.16.1.el6.x86_64

8）编译、安装keepalived

[root@localhost ~]# make && make install

9）创建软连接，拷贝相应的文件

[root@localhost ~]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin

[root@localhost ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig

[root@localhost ~]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d

[root@localhost ~]# chkconfig --add keepalived

[root@localhost ~]# chkconfig keepalived on

[root@localhost ~]# mkdir /etc/keepalived

[root@localhost ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

10）编辑keepalived（主）配置文件

# 编辑vim /etc/keepalived/keepalived.conf

[root@localhost ~]# vim /etc/keepalived/keepalived.conf

# 配置内容如下

! Configuration File for keepalived
global_defs {
notification_email {
675583110@qq.com
}
notification_email_from 675583110@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nfs {
script "/data/shell/check_nfs.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
mcast_src_ip 192.168.8.10
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.8.99
}
track_script {
chk_nfs
}
}

11）启动keepalived服务、NFS服务

[root@localhost ~]# service keepalived start

[root@localhost ~]# service rpcbind start

[root@localhost ~]# service nfs start

12）LAP端跟LAP1端分别重新挂载NFS共享目录（VIP），执行以下命令

[root@localhost ~]# umount /var/www/html

[root@localhost ~]# mount -t nfs 192.168.8.99:/data/www /var/www/html

13）分别访问LAP和LAP1

14）分别在NFS主和备机器上编写主备切换脚本，实现当主（备）的NFS服务宕机自动切换到备（主）机器上

[root@localhost ~]# mkdir -p /data/shell

[root@localhost ~]# vim /data/shell/check_nfs.sh

#脚本内容如下

#!/bin/bash
#2018-1-16 13:19:57
#auto check nfs process
NUM=`ps -ef |grep nfs|grep -v grep|grep -v check|wc -l`
if [[ $NUM -eq 0 ]];then
    /etc/init.d/keepalived stop
fi

[root@localhost ~]# chmod o+x /data/shell/check_nfs.sh

15）实验测试

正常运行，VIP在NFS主机器上，当NFS服务器宕机之后自动漂移到备机器上，如下图

16）但是打开论坛是出现403，如下图

17）需要umount /var/www/html，再执行mount -t nfs 192.168.8.99:/data/www /var/www/html，chmod o+w data/ config/ uc_* -R即可