海运的博客

Centos配置LVS Keepalived高可用负载均衡服务器

发布时间：July 18, 2012 // 分类：高可用 // No Comments

LVS IP信息：

主LVS：192.168.1.2
备LVS：192.168.1.3
虚拟IP：192.168.1.6
应用服务器1：192.168.1.12
应用服务器2：192.168.1.13

查看内核是否支持LVS模块：

modprobe -l |grep ipvs
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_dh.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_ftp.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_lblc.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_lblcr.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_lc.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_nq.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_rr.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_sed.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_sh.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_wlc.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_wrr.ko

根据内核版本下载安装相应的ipvsadm：

ln -s /usr/src/kernels/2.6.18-274.17.1.el5-x86_64/ /usr/src/linux
wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
tar zxvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
make
make install

也可使用yum直接安装：

yum install ipvsadm

安装Keepalived：

wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
tar zxvf keepalived-1.1.19.tar.gz 
cd keepalived-1.1.19
./configure --sysconf=/etc/ --with-kernel-dir=/usr/src/kernels/2.6.18-274.17.1.el5-x86_64/
make
make install
ln -s /usr/local/sbin/keepalived /sbin/keepalived

主LVS配置keepalived：

! Configuration File for keepalived

global_defs {
   notification_email {
     admin@www.haiyun.me
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.6
    }
}

virtual_server 192.168.1.6 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    nat_mask 255.255.255.0
    persistence_timeout 50
    protocol TCP

    real_server 192.168.1.12 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.1.13 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    } 
}

启动keepalived，同时会在主LVS绑定VIP：

/etc/init.d/keepalived start
ip add show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:e7:cc:3b brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.3/24 brd 192.168.1.255 scope global eth0
    inet 192.168.1.6/32 scope global eth0

查看当前LVS参数：

ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.6:http rr
  -> 192.168.1.12:http            Route   1      0          0         
  -> 192.168.1.13:http            Route   1      0          0

复制主keepalived到备LVS，修改以下参数：

state BACKUP
priority 90

应用服务器增加虚拟VIP：

#!/bin/bash
VIP=192.168.1.6
. /etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $SNS
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac

测试LVS故障切换，停止主LVS上keepalived服务：

/etc/init.d/keepalived stop

查看备LVS已经接管为主LVS：

tail -n 10 /var/log/message
Jul 14 20:30:28 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Jul 14 20:30:29 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Jul 14 20:30:29 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jul 14 20:30:29 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.6

应用服务器故障测试，停止其中之一httpd服务：

/etc/init.d/httpd stop

查看keepalved日志已将出错服务器移除并邮件报警：

tail -n 10 /var/log/message
Jul 14 21:36:18 centos5 Keepalived_healthcheckers: TCP connection to [192.168.1.12:80] failed !!!
Jul 14 21:36:18 centos5 Keepalived_healthcheckers: Removing service [192.168.1.12:80] from VS [192.168.1.6:80]
Jul 14 21:36:18 centos5 Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.6:http rr
  -> 192.168.1.13:http            Route   1      0          0

Nginx反向代理做负载均衡及缓存服务器

发布时间：May 2, 2012 // 分类：高可用 // No Comments

安装编译环境及相关组件：

yum -y install gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 \
libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel curl \
curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidn libidn-devel openssl openssl-devel openldap \
openldap-devel nss_ldap openldap-clients openldap-servers unzip

编译安装PCRE、Nginx及缓存清除模块ngx_cache_purge：

/usr/local/src/
wget http://sourceforge.net/projects/pcre/files/pcre/8.30/pcre-8.30.zip
unzip pcre-8.30.zip 
cd pcre-8.30
./configure 
make && make install  
cd ..
wget http://labs.frickle.com/files/ngx_cache_purge-1.2.tar.gz
tar zxvf ngx_cache_purge-1.2.tar.gz 
wget http://nginx.org/download/nginx-1.0.15.tar.gz
tar zxvf nginx-1.0.15.tar.gz 
cd nginx-1.0.15
./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module  \
--with-http_gzip_static_module --add-module=../ngx_cache_purge-1.2
make && make install

Nginx配置文件：

user  www www;

worker_processes 4;

error_log  /usr/local/nginx/logs/nginx_error.log  crit;

pid        /usr/local/nginx/logs/nginx.pid;

worker_rlimit_nofile 65535;

events
    {
        use epoll;
        worker_connections 65535;
    }

http
    {
        include       mime.types;
                default_type application/octet-stream; 

                charset utf-8;   

        server_names_hash_bucket_size 128;
        client_header_buffer_size 32k;
        large_client_header_buffers 4 32k;
        client_max_body_size 50m;

        sendfile on;
        tcp_nopush     on;

        keepalive_timeout 60;

        tcp_nodelay on;

                proxy_temp_path /cache_tmp;
                proxy_cache_path /cache levels=1:2 keys_zone=proxy_cache:256m inactive=10d max_size=10g;

                client_body_buffer_size  512k;
                proxy_connect_timeout 5;   
                proxy_read_timeout 60;   
                proxy_send_timeout 15; 
                proxy_buffer_size        16k;
                proxy_buffers            4 64k;
                proxy_busy_buffers_size 128k;
                proxy_temp_file_write_size 128k;

        gzip on;
        gzip_min_length  1k;
        gzip_buffers     4 16k;
        gzip_http_version 1.0;
        gzip_comp_level 2;
        gzip_types       text/plain application/x-javascript text/css application/xml;
        gzip_vary on;

                upstream proxy_server_pool 
                {
                server node1.www.haiyun.me;
                server node2.www.haiyun.me;
                }
                 

                 log_format   proxy
                 $remote_addr-$remote_user-$time_local-$request-$body_bytes_sent-$http_referer-
                 $http_user_agent-$upstream_addr-$upstream_cache_status-$upstream_status ;
server
        {
                 listen       80;
                 server_name cdn.www.haiyun.me;
                 location /
                 {
                 proxy_next_upstream http_502 http_504 error timeout invalid_header;   
                 proxy_cache proxy_cache;
                 proxy_cache_valid  200 304 1d;
                 proxy_cache_key $host$uri$is_args$args; #$http_cookie
                 add_header Nginx-Cache "$upstream_cache_status from $upstream_addr";
                 proxy_set_header Host $host;
                 proxy_set_header X-Forwarded-For $remote_addr;
                 proxy_pass http://proxy_server_pool;
                 }

                 location ~ .*\.(php|jsp|cgi)?$
                 {
                 proxy_set_header Host $host;
                 proxy_set_header X-Forwarded-For $remote_addr;
                 proxy_pass http://proxy_server_pool;
                 }
                 location ~ /purge(/.*)   
                 {    
                 allow 127.0.0.1;   
                 deny all;   
                 proxy_cache_purge proxy_cache $host$uri$is_args$args;   
                 }   
                 access_log  /usr/local/nginx/logs/cdn.www.haiyun.me.log proxy;
         }
}
分类
最新文章
最近回复
  • 海运: 恩山有很多。
  • swsend: 大佬可以分享一下固件吗，谢谢。
  • Jimmy: 方法一 nghtp3步骤需要改成如下才能编译成功： git clone https://git...
  • 海运: 地址格式和udpxy一样，udpxy和msd_lite能用这个就能用。
  • 1: 怎么用 编译后的程序在家里路由器内任意一台设备上运行就可以吗？比如笔记本电脑 m参数是笔记本的...
  • 孤狼: ups_status_set: seems that UPS [BK650M2-CH] is ...
  • 孤狼: 擦。。。。apcupsd会失联 nut在冲到到100的时候会ONBATT进入关机状态，我想想办...
  • 海运: 网络，找到相应的url编辑重发请求，firefox有此功能，其它未知。
  • knetxp: 用浏览器F12网络拦截或监听后编辑重发请求，修改url中的set为set_super，将POS...
  • Albert: 啊啊啊啊啊啊啊啊啊 我太激动了，终于好了英文区搜索了半天，翻遍了 pve 论坛没找到好方法，博...