海运的博客

FastDFS配合FastDHT去重

发布时间:January 28, 2014 // 分类:Linux服务 // No Comments

FastDHT使用Berkeley DB为后端存储,安装:

wget https://fastdht.googlecode.com/files/FastDHT_v1.23.tar.gz
tar zxvf FastDHT_v1.23.tar.gz 
cd FastDHT
./make.sh C_INCLUDE_PATH=/usr/local/BerkeleyDB.6.0/include \
LIBRARY_PATH=/usr/local/BerkeleyDB.6.0/lib/
./make.sh install
mkdir -p /home/yuqing/fastdht
echo "/usr/local/BerkeleyDB.6.0/lib/" > /etc/ld.so.conf.d/db.conf
ldconfig

配置fastdht:

/etc/fdht/fdht_servers.conf
group_count = 2
group0 = 192.168.1.1:11411
group0 = 192.168.1.2:11411
group1 = 192.168.1.1:11411
group1 = 192.168.1.2:11411

配置fastdfs storage

/etc/fdfs/storage.conf 
check_file_duplicate=1
file_signature_method=hash
key_namespace=FastDFS
keep_alive=1
#include /etc/fdht/fdht_servers.conf

再次测试上传多次同个文件,重复上传的文件以链接方式保存。

rpmbuild遇到的问题

发布时间:January 28, 2014 // 分类:CentOS // No Comments

当打包文件包含软链接时遇到以下错误:error: Symlink points to BuildRoot,解决:

#源文件使用绝对路径,软链接使用%{buildroot}路径,打包时会自动处理
ln -sf /usr/local/lib/libfdfsclient.so.1 %{buildroot}/usr/local/lib/libfdfsclient.so

安装打包好的rpm时提示,而依赖的动态库本身位于要安装的软件包中。

error: Failed dependencies:libfastcommon.so()(64bit) is needed by fastdfs-4.0.6-1.el6.x86_64

原因是rpm build时会检查依赖,安装时如果不满足依赖不能安装,禁用之:

AutoReqProv: no

分布式存储FastDFS使用

发布时间:January 28, 2014 // 分类:Linux服务 // No Comments

安装:

yum install -y libevent-devel pcre-devel zlib-devel
wget http://fastdfs.googlecode.com/files/FastDFS_v4.06.tar.gz
tar zxvf FastDFS_v4.06.tar.gz
cd FastDFS
./make.sh && ./make.sh install

配置tracker:

/etc/fdfs/tracker.conf 
base_path=/home/fastdfs #工作目录,记录日志相差信息
bind_addr=0.0.0.0
port=22122
run_by_group=fastdfs
run_by_user=fastdfs

配置storage:

/etc/fdfs/storage.conf 
group_name=group1 #服务器所在组,组内服务器数据同步
base_path=/home//fastdfs
store_path0=/home/fastdfs #数据目录
tracker_server=192.168.1.1:22122 #tracker服务器

配置client:

/etc/fdfs/client.conf 
tracker_server=192.168.1.1:22122

测试上传:

fdfs_test /etc/fdfs/client.conf upload .bashrc 

查看group/storage信息及删除storage:

fdfs_monitor /etc/fdfs/client.conf
fdfs_monitor /etc/fdfs/client.conf delete group1 192.168.1.1

安装配置php客户端:

cd php_client/
./configure
make && make install
cp fastdfs_client.ini /etc/php.d/
#配置调用上面配置的/etc/fdfs/client.conf 

使用php上传文件:

<?php
$tracker = fastdfs_tracker_get_connection();
var_dump($tracker);
$storage = fastdfs_tracker_query_storage_store();
var_dump($storage);
$ret = fastdfs_storage_upload_by_filename('fastdfs.spec');
var_dump($ret);
$ret = fastdfs_storage_upload_by_filename1('fastdfs.spec');
var_dump($ret);
$file = file_get_contents("fastdfs.spec");
$ret = fastdfs_storage_upload_by_filebuff($file, 'spec');
var_dump($ret);
$ret = fastdfs_storage_upload_by_filebuff1($file, 'spec');
var_dump($ret);
?>

storage存储安装Nginx模块,用于提供文件下载,文件不存在时通过tracker获取同组内其它storage数据:

wget http://fastdfs.googlecode.com/files/fastdfs-nginx-module_v1.15.tar.gz
./configure --add-module=/root/fastdfs-nginx-module/src
cp mod_fastdfs.conf /etc/fdfs/ #配置文件

插件配置:

tracker_server=192.168.1.1:22122
group_name=group1
url_have_group_name = false #url是否包含组名
response_mode=proxy #向其它storage获取数据方式,代理或重定向
    #location /grout0/M00 { #url_have_group_name要设置为true
    location /M00 {
            root /home/yuqing/fastdfs/data;
            ngx_fastdfs_module;
    }

直接使用Nginx提供文件访问,不通过tracker:

  location /M00 {
    alias /home/yuqing/fastdfs/data;
    #ngx_fastdfs_module;
  }

自建yum源

发布时间:January 27, 2014 // 分类:CentOS // No Comments

安装制作软件包:

yum -y install createrepo

创建仓库目录:

mkdir -p /yum/centos/6/{i386,x86_64}
生成repodata索引
createrepo -d /yum/centos/6/x86_64/
#更新索引
createrepo --update /yum/centos/6/x86_64/

配置Nginx:

location / {
    root /yum;
    autoindex_exact_size off;
    autoindex_localtime on;
    autoindex  on;
}

配置yum使用源:

[testrepo]
name=test repo
baseurl=http://192.168.1.1/centos/6/$basearch/
gpgcheck=0
enabled=1

制作OpenResty RPM包

发布时间:January 27, 2014 // 分类:CentOS // No Comments

创建rpmbuild目录:

mkdir -p ~/rpmbuild/{SOURCES,SPECS,SRPMS}

下载OpenResty源码:

wget -P ~/rpmbuild/SOURCE http://openresty.org/download/ngx_openresty-1.4.3.4.tar.gz

创建spec文件:

%define nginx_home %{_localstatedir}/cache/nginx
%define nginx_user nginx
%define nginx_group nginx
%define filename ngx_openresty-1.4.3.4

Summary: High performance web server
Name: openresty
Version: 1.4.3
Release: 4%{?dist}
Vendor: nginx inc.
URL: http://openresty.org/

Source0: http://openresty.org/download/%{filename}.tar.gz
Source1: logrotate
Source2: nginx.init
Source3: nginx.sysconf
Source4: nginx.conf
Source5: nginx.vh.default.conf
Source6: nginx.vh.example_ssl.conf

License: 2-clause BSD-like license
Group: System Environment/Daemons

BuildRequires: zlib-devel
BuildRequires: pcre-devel
BuildRequires: perl
BuildRequires: openssl-devel
BuildRequires: readline-devel
Requires: initscripts >= 8.36
Requires(pre): shadow-utils
Requires(post): chkconfig
Provides: webserver

%description
nginx [engine x] is an HTTP and reverse proxy server, as well as
a mail proxy server.

%package debug
Summary: debug version of nginx
Group: System Environment/Daemons
Requires: openresty
%description debug
Not stripped version of nginx built with the debugging log support.

%prep
#解压文件到build目录,默认目录名名称+版本 -n进入到指定的目录,用于文件名和解压目录名不同
%setup -q -n %{filename}

%build
./configure \
        --sbin-path=%{_sbindir}/nginx \
        --conf-path=%{_sysconfdir}/nginx/nginx.conf \
        --error-log-path=%{_localstatedir}/log/nginx/error.log \
        --http-log-path=%{_localstatedir}/log/nginx/access.log \
        --pid-path=%{_localstatedir}/run/nginx.pid \
        --lock-path=%{_localstatedir}/run/nginx.lock \
        --http-client-body-temp-path=%{_localstatedir}/cache/nginx/client_temp \
        --http-proxy-temp-path=%{_localstatedir}/cache/nginx/proxy_temp \
        --http-fastcgi-temp-path=%{_localstatedir}/cache/nginx/fastcgi_temp \
        --http-uwsgi-temp-path=%{_localstatedir}/cache/nginx/uwsgi_temp \
    --without-http_scgi_module \
        --user=%{nginx_user} \
        --group=%{nginx_group} \
    --with-luajit \
        --with-http_ssl_module \
        --with-http_gunzip_module \
        --with-http_gzip_static_module \
        --with-file-aio \
        --with-ipv6 \
        --with-debug \
        $*
make %{?_smp_mflags}
#$RPM_BUILD_DIR为rpmbuild/BUILD目录,编译软件所在的目录
%{__mv} %{_builddir}/%{filename}/build/nginx-1.4.3/objs/nginx %{_builddir}/nginx.debug
./configure \
        --sbin-path=%{_sbindir}/nginx \
        --conf-path=%{_sysconfdir}/nginx/nginx.conf \
        --error-log-path=%{_localstatedir}/log/nginx/error.log \
        --http-log-path=%{_localstatedir}/log/nginx/access.log \
        --pid-path=%{_localstatedir}/run/nginx.pid \
        --lock-path=%{_localstatedir}/run/nginx.lock \
        --http-client-body-temp-path=%{_localstatedir}/cache/nginx/client_temp \
        --http-proxy-temp-path=%{_localstatedir}/cache/nginx/proxy_temp \
        --http-fastcgi-temp-path=%{_localstatedir}/cache/nginx/fastcgi_temp \
        --http-uwsgi-temp-path=%{_localstatedir}/cache/nginx/uwsgi_temp \
    --without-http_scgi_module \
        --user=%{nginx_user} \
        --group=%{nginx_group} \
    --with-luajit \
        --with-http_ssl_module \
        --with-http_gunzip_module \
        --with-http_gzip_static_module \
        --with-file-aio \
        --with-ipv6 \
        $*
make %{?_smp_mflags}

%install
#$RPM_BUILD_ROOT 虚拟安装的根目录,位于rpmbuild/BUILDROOT/名称+版本
%{__rm} -rf $RPM_BUILD_ROOT
%{__make} DESTDIR=$RPM_BUILD_ROOT install

%{__rm} -f $RPM_BUILD_ROOT%{_sysconfdir}/nginx/*.default
%{__rm} -f $RPM_BUILD_ROOT%{_sysconfdir}/nginx/scgi_params

%{__mkdir} -p $RPM_BUILD_ROOT%{_localstatedir}/log/nginx
%{__mkdir} -p $RPM_BUILD_ROOT%{_localstatedir}/run/nginx
%{__mkdir} -p $RPM_BUILD_ROOT%{_localstatedir}/cache/nginx
%{__mkdir} -p $RPM_BUILD_ROOT%{_localstatedir}/www

%{__mv} $RPM_BUILD_ROOT/usr/local/openresty/nginx/html $RPM_BUILD_ROOT%{_localstatedir}/www
%{__rm} -rf $RPM_BUILD_ROOT/usr/local/openresty/nginx/

%{__mkdir} -p $RPM_BUILD_ROOT%{_sysconfdir}/nginx/conf.d
%{__rm} $RPM_BUILD_ROOT%{_sysconfdir}/nginx/nginx.conf
%{__install} -m 644 -p %{SOURCE4} $RPM_BUILD_ROOT%{_sysconfdir}/nginx/nginx.conf
%{__install} -m 644 -p %{SOURCE5} $RPM_BUILD_ROOT%{_sysconfdir}/nginx/conf.d/default.conf
%{__install} -m 644 -p %{SOURCE6} $RPM_BUILD_ROOT%{_sysconfdir}/nginx/conf.d/example_ssl.conf

%{__mkdir} -p $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig
%{__install} -m 644 -p %{SOURCE3} $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig/nginx

# install init
%{__mkdir} -p $RPM_BUILD_ROOT%{_initrddir}
%{__install} -m755 %{SOURCE2} $RPM_BUILD_ROOT%{_initrddir}/nginx

# install log rotation stuff
%{__mkdir} -p $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d
%{__install} -m 644 -p %{SOURCE1} $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/nginx

# install debug
%{__install} -m644 %{_builddir}/nginx.debug $RPM_BUILD_ROOT%{_sbindir}/nginx.debug

#rpm build完成后删除虚拟安装目录
%clean
%{__rm} -rf $RPM_BUILD_ROOT

%files
#默认权限目录755,文件644
%defattr(-,root,root)

%{_sbindir}/nginx

#仅包含目录
%dir %{_sysconfdir}/nginx
%dir %{_sysconfdir}/nginx/conf.d
%{_localstatedir}/www/html/
#包含目录内所有文件
/usr/local/openresty/
%attr(0777,root,root) /usr/local/openresty/luajit/bin/luajit

#配置文件升级时不覆盖
%config(noreplace) %{_sysconfdir}/nginx/nginx.conf
%config(noreplace) %{_sysconfdir}/nginx/conf.d/default.conf
%config(noreplace) %{_sysconfdir}/nginx/conf.d/example_ssl.conf
%config(noreplace) %{_sysconfdir}/nginx/mime.types
%config(noreplace) %{_sysconfdir}/nginx/uwsgi_params
%config(noreplace) %{_sysconfdir}/nginx/fastcgi_params
%config(noreplace) %{_sysconfdir}/nginx/fastcgi.conf
%config(noreplace) %{_sysconfdir}/nginx/koi-utf
%config(noreplace) %{_sysconfdir}/nginx/koi-win
%config(noreplace) %{_sysconfdir}/nginx/win-utf

%config(noreplace) %{_sysconfdir}/logrotate.d/nginx
%config(noreplace) %{_sysconfdir}/sysconfig/nginx
%{_initrddir}/nginx

%attr(0755,root,root) %dir %{_localstatedir}/cache/nginx
%attr(0755,root,root) %dir %{_localstatedir}/log/nginx

%files debug
%attr(0755,root,root) %{_sbindir}/nginx.debug

%pre
#如安装httpd删除
rpm -q httpd > /dev/null && /sbin/service httpd stop > /dev/null 2>&1 && rpm -e httpd
#安装前添加运行nginx用户
getent group %{nginx_group} >/dev/null || groupadd -r %{nginx_group}
getent passwd %{nginx_user} >/dev/null || \
    useradd -r -g %{nginx_group} -s /sbin/nologin \
    -d %{nginx_home} -c "nginx user"  %{nginx_user}
exit 0

%post
# Register the nginx service
if [ $1 -eq 1 ]; then
    /sbin/chkconfig --add nginx
    # print site info
    cat <<BANNER
----------------------------------------------------------------------

Thanks for using nginx!

Please find the official documentation for nginx here:
* http://nginx.org/en/docs/

Commercial subscriptions for nginx are available on:
* http://nginx.com/products/

----------------------------------------------------------------------
BANNER

    # Touch and set permisions on default log files on installation

    if [ -d %{_localstatedir}/log/nginx ]; then
        if [ ! -e %{_localstatedir}/log/nginx/access.log ]; then
            touch %{_localstatedir}/log/nginx/access.log
            %{__chmod} 640 %{_localstatedir}/log/nginx/access.log
            %{__chown} nginx:adm %{_localstatedir}/log/nginx/access.log
        fi

        if [ ! -e %{_localstatedir}/log/nginx/error.log ]; then
            touch %{_localstatedir}/log/nginx/error.log
            %{__chmod} 640 %{_localstatedir}/log/nginx/error.log
            %{__chown} nginx:adm %{_localstatedir}/log/nginx/error.log
        fi
    fi
fi

%preun
if [ $1 -eq 0 ]; then
    /sbin/service nginx stop > /dev/null 2>&1
    /sbin/chkconfig --del nginx
fi

%postun
if [ $1 -ge 1 ]; then
    /sbin/service nginx upgrade &>/dev/null || :
fi

build:

yum install readline-devel pcre-devel openssl-devel rpm-build
rpmbuild -ba openresty.spec 

安装打包的rpm程序:

rpm -ivh ~/rpmbuild/RPMS/x86_64/openresty-1.4.3-4.el6.x86_64.rpm 
分类
最新文章
最近回复
  • 常易北: 满分顶楼主!!!我也是,现在网上瞎几把扫别人云主机的智障太多了!
  • track: NAT 部分没处理,这块感觉有问题。
  • yu: 查到有用的资料很幸福!
  • boscotsang: 请问下服务器端如何控制权限让通过权限认证的客户端才能连接?现在这样只要有自编译的客户端程序都可...
  • yck932: 大侠,请问哪里有 tunnels 详细参考资料,谢谢
  • bubble: 现在linux 4.9以上的内核有个bbr的参数,用这个bbr比用锐速效果更好,可参考我的博文...
  • wang: openwrt 只有 tmp/etc/miniupnpd.conf 修改后又重置了,其他路径找...
  • wbq: 可以了 只是https透明代理如何做呢?
  • wbq: 你的这个配置需要浏览器设置代理ip+port, 现在假设不设置,只配置客户端网关为Nginx那...
  • wbq: 您好,想用Nginx做个透明代理,客户端机器配置下网关(即Nginx那台机器),客户端dns配...