海运的博客

Centos配置LVS Keepalived高可用负载均衡服务器

发布时间:July 18, 2012 // 分类:高可用 // No Comments

LVS IP信息:

主LVS:192.168.1.2
备LVS:192.168.1.3
虚拟IP:192.168.1.6
应用服务器1:192.168.1.12
应用服务器2:192.168.1.13

查看内核是否支持LVS模块:

modprobe -l |grep ipvs
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_dh.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_ftp.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_lblc.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_lblcr.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_lc.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_nq.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_rr.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_sed.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_sh.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_wlc.ko
/lib/modules/2.6.18-274.17.1.el5/kernel/net/ipv4/ipvs/ip_vs_wrr.ko

根据内核版本下载安装相应的ipvsadm:

ln -s /usr/src/kernels/2.6.18-274.17.1.el5-x86_64/ /usr/src/linux
wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
tar zxvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
make
make install

也可使用yum直接安装:

yum install ipvsadm

安装Keepalived:

wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
tar zxvf keepalived-1.1.19.tar.gz 
cd keepalived-1.1.19
./configure --sysconf=/etc/ --with-kernel-dir=/usr/src/kernels/2.6.18-274.17.1.el5-x86_64/
make
make install
ln -s /usr/local/sbin/keepalived /sbin/keepalived

主LVS配置keepalived:

! Configuration File for keepalived

global_defs {
   notification_email {
     admin@www.haiyun.me
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.6
    }
}

virtual_server 192.168.1.6 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    nat_mask 255.255.255.0
    persistence_timeout 50
    protocol TCP

    real_server 192.168.1.12 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.1.13 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    } 
}

启动keepalived,同时会在主LVS绑定VIP:

/etc/init.d/keepalived start
ip add show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:e7:cc:3b brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.3/24 brd 192.168.1.255 scope global eth0
    inet 192.168.1.6/32 scope global eth0

查看当前LVS参数:

ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.6:http rr
  -> 192.168.1.12:http            Route   1      0          0         
  -> 192.168.1.13:http            Route   1      0          0      

复制主keepalived到备LVS,修改以下参数:

state BACKUP
priority 90

应用服务器增加虚拟VIP:

#!/bin/bash
VIP=192.168.1.6
. /etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $SNS
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac

测试LVS故障切换,停止主LVS上keepalived服务:

/etc/init.d/keepalived stop

查看备LVS已经接管为主LVS:

tail -n 10 /var/log/message
Jul 14 20:30:28 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Jul 14 20:30:29 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Jul 14 20:30:29 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jul 14 20:30:29 centos5 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.6

应用服务器故障测试,停止其中之一httpd服务:

/etc/init.d/httpd stop

查看keepalved日志已将出错服务器移除并邮件报警:

tail -n 10 /var/log/message
Jul 14 21:36:18 centos5 Keepalived_healthcheckers: TCP connection to [192.168.1.12:80] failed !!!
Jul 14 21:36:18 centos5 Keepalived_healthcheckers: Removing service [192.168.1.12:80] from VS [192.168.1.6:80]
Jul 14 21:36:18 centos5 Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.6:http rr
  -> 192.168.1.13:http            Route   1      0          0    

Nginx反向代理做负载均衡及缓存服务器

发布时间:May 2, 2012 // 分类:高可用 // No Comments

安装编译环境及相关组件:

yum -y install gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 \
libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel curl \
curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidn libidn-devel openssl openssl-devel openldap \
openldap-devel nss_ldap openldap-clients openldap-servers unzip

编译安装PCRE、Nginx及缓存清除模块ngx_cache_purge:

/usr/local/src/
wget http://sourceforge.net/projects/pcre/files/pcre/8.30/pcre-8.30.zip
unzip pcre-8.30.zip 
cd pcre-8.30
./configure 
make && make install  
cd ..
wget http://labs.frickle.com/files/ngx_cache_purge-1.2.tar.gz
tar zxvf ngx_cache_purge-1.2.tar.gz 
wget http://nginx.org/download/nginx-1.0.15.tar.gz
tar zxvf nginx-1.0.15.tar.gz 
cd nginx-1.0.15
./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module  \
--with-http_gzip_static_module --add-module=../ngx_cache_purge-1.2
make && make install  

Nginx配置文件:

user  www www;

worker_processes 4;

error_log  /usr/local/nginx/logs/nginx_error.log  crit;

pid        /usr/local/nginx/logs/nginx.pid;

worker_rlimit_nofile 65535;

events
    {
        use epoll;
        worker_connections 65535;
    }

http
    {
        include       mime.types;
                default_type application/octet-stream; 

                charset utf-8;   

        server_names_hash_bucket_size 128;
        client_header_buffer_size 32k;
        large_client_header_buffers 4 32k;
        client_max_body_size 50m;

        sendfile on;
        tcp_nopush     on;

        keepalive_timeout 60;

        tcp_nodelay on;

                proxy_temp_path /cache_tmp;
                proxy_cache_path /cache levels=1:2 keys_zone=proxy_cache:256m inactive=10d max_size=10g;

                client_body_buffer_size  512k;
                proxy_connect_timeout 5;   
                proxy_read_timeout 60;   
                proxy_send_timeout 15; 
                proxy_buffer_size        16k;
                proxy_buffers            4 64k;
                proxy_busy_buffers_size 128k;
                proxy_temp_file_write_size 128k;

        gzip on;
        gzip_min_length  1k;
        gzip_buffers     4 16k;
        gzip_http_version 1.0;
        gzip_comp_level 2;
        gzip_types       text/plain application/x-javascript text/css application/xml;
        gzip_vary on;

                upstream proxy_server_pool 
                {
                server node1.www.haiyun.me;
                server node2.www.haiyun.me;
                }
                 

                 log_format   proxy
                 $remote_addr-$remote_user-$time_local-$request-$body_bytes_sent-$http_referer-
                 $http_user_agent-$upstream_addr-$upstream_cache_status-$upstream_status ;
server
        {
                 listen       80;
                 server_name cdn.www.haiyun.me;
                 location /
                 {
                 proxy_next_upstream http_502 http_504 error timeout invalid_header;   
                 proxy_cache proxy_cache;
                 proxy_cache_valid  200 304 1d;
                 proxy_cache_key $host$uri$is_args$args; #$http_cookie
                 add_header Nginx-Cache "$upstream_cache_status from $upstream_addr";
                 proxy_set_header Host $host;
                 proxy_set_header X-Forwarded-For $remote_addr;
                 proxy_pass http://proxy_server_pool;
                 }

                 location ~ .*\.(php|jsp|cgi)?$
                 {
                 proxy_set_header Host $host;
                 proxy_set_header X-Forwarded-For $remote_addr;
                 proxy_pass http://proxy_server_pool;
                 }
                 location ~ /purge(/.*)   
                 {    
                 allow 127.0.0.1;   
                 deny all;   
                 proxy_cache_purge proxy_cache $host$uri$is_args$args;   
                 }   
                 access_log  /usr/local/nginx/logs/cdn.www.haiyun.me.log proxy;
         }
}
分类
最新文章
最近回复
  • boscotsang: 请问下服务器端如何控制权限让通过权限认证的客户端才能连接?现在这样只要有自编译的客户端程序都可...
  • yck932: 大侠,请问哪里有 tunnels 详细参考资料,谢谢
  • bubble: 现在linux 4.9以上的内核有个bbr的参数,用这个bbr比用锐速效果更好,可参考我的博文...
  • wang: openwrt 只有 tmp/etc/miniupnpd.conf 修改后又重置了,其他路径找...
  • wbq: 可以了 只是https透明代理如何做呢?
  • wbq: 你的这个配置需要浏览器设置代理ip+port, 现在假设不设置,只配置客户端网关为Nginx那...
  • wbq: 您好,想用Nginx做个透明代理,客户端机器配置下网关(即Nginx那台机器),客户端dns配...
  • ccc: 可以提供易调用openssl常用函数吗?
  • 夜未央: 在centos7 下面 貌似转换不了。有人遇到过么????
  • brian: 博主你好,请教一个问题。我使用fprobe进行将交换机流量转换成netflow时是没有问题的,...