海运的博客

编译curl http3 quic支持

发布时间:December 27, 2021 // 分类: // 2 Comments

方法一,使用openssl ngtcp2 nghttp3编译curl支持http3 quic:

apt install build-essential autoconf libtool pkg-config 
git clone --depth 1 -b OpenSSL_1_1_1m+quic https://github.com/quictls/openssl
cd openssl/
./config enable-tls1_3 --prefix=/usr/local/openssl
 make && make install
cd ../
git clone https://github.com/ngtcp2/nghttp3
cd nghttp3/
autoreconf -fi
./configure --prefix=/usr/local/nghttp3 --enable-lib-only
make && make install
cd ../
git clone https://github.com/ngtcp2/ngtcp2
cd ngtcp2/
autoreconf -fi
./configure PKG_CONFIG_PATH=/usr/local/openssl/lib/pkgconfig:/usr/local/nghttp3/lib/pkgconfig LDFLAGS="-Wl,-rpath,/usr/local/openssl/lib" --prefix=/usr/local/ngtcp2 --enable-lib-only 
 make && make install
cd ../
wget https://github.com/curl/curl/releases/download/curl-7_80_0/curl-7.80.0.tar.gz
tar zxf curl-7.80.0.tar.gz 
cd curl-7.80.0/
LDFLAGS="-Wl,-rpath,/usr/local/openssl/lib64" ./configure --with-openssl=/usr/local/openssl/ --with-nghttp3=/usr/local/nghttp3 --with-ngtcp2=/usr/local/ngtcp2 --prefix=/usr/local/curl
make && make install
LD_LIBRARY_PATH="/usr/local/curl/lib/:/usr/local/openssl/lib/" /usr/local/curl/bin/curl -V

方法二,通过quiche编译支持http3 quic:

apt install build-essential cmake pkg-config
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
git clone --recursive https://github.com/cloudflare/quiche
cd quiche/
cargo build --package quiche --release --features ffi,pkg-config-meta,qlog
mkdir quiche/deps/boringssl/src/lib
ln -vnf $(find target/release -name libcrypto.a -o -name libssl.a) quiche/deps/boringssl/src/lib/
cd ../
wget https://github.com/curl/curl/releases/download/curl-7_80_0/curl-7.80.0.tar.gz
tar zxf curl-7.80.0.tar.gz 
cd curl-7.80.0/
./configure LDFLAGS="-Wl,-rpath,$PWD/../quiche/target/release" --with-openssl=$PWD/../quiche/quiche/deps/boringssl/src --with-quiche=$PWD/../quiche/target/release --prefix=/usr/local/curl
make && make install
cp ../quiche/target/release/libquiche.so /usr/local/curl/lib/
LD_LIBRARY_PATH="/usr/local/curl/lib/"  /usr/local/curl/bin/curl -V

使用curl测试http3 quic:

LD_LIBRARY_PATH="/usr/local/curl/lib/:/usr/local/openssl/lib/"  /usr/local/curl/bin/curl --http3 https://www.haiyun.me  -I

使用中遇到的问题,quiche编译的curl下载一会后断流,openssl编译的curl下载速度很慢。
参考:
https://github.com/curl/curl/blob/master/docs/HTTP3.md

nginx配置webdav及使用curl测试

发布时间:February 2, 2021 // 分类:Nginx // No Comments

需安装http-dav-ext扩展,有时使用非标准webdav客户端删除目录时最后不带/,nginx删除目录失败返回409,alias不支持rewrite使用root。

  location /webdav/ {
    satisfy any;
    allow 127.0.0.1;
    deny all;
    #root /;
    #if (-d $request_filename) { rewrite ^(.*[^/])$ $1/ break; }
    alias /webdav/;
    dav_methods PUT DELETE MKCOL COPY MOVE;
    dav_ext_methods PROPFIND OPTIONS;
    dav_access user:rw group:r all:r;
    create_full_put_path  on;
    port_in_redirect off;
    autoindex on;
    autoindex_localtime on;
    charset utf-8;
    auth_basic "Login";
    auth_basic_user_file htpasswd;
  }

配置nginx webdav不同用户使用不同的家目录:

map $remote_user $home {
  default      $remote_user;
  ''           guest;
}
server {
  listen 80 default_server;
  listen [::]:80 default_server;
  server_name _;

  root /var/www/html;
  index index.html index.htm index.nginx-debian.html;

  try_files $uri /;

  location / {
    #if (-d $request_filename) { rewrite ^(.*[^/])$ $1/ break; }
    alias /data/webdav/$home/;
    #dav_methods PUT DELETE MKCOL COPY MOVE;
    dav_ext_methods PROPFIND OPTIONS;
    dav_access user:r group:r all:r;
    create_full_put_path on;
    port_in_redirect off;
    autoindex on;
    autoindex_localtime on;
    charset utf-8;
    auth_basic "Login";
    auth_basic_user_file htpasswd;
  }

  #access_log off;
  access_log /var/log/nginx/webdav.log;
  error_log  /var/log/nginx/webdav_error.log;
}

使用curl测试webdav:
新建目录,注意最后一定要带/,不然返回409,MKCOL can create a collection only

curl -X MKCOL https://www.haiyun.me/webdav/test/

上传文件:

curl -T filename https://www.haiyun.me/webdav/

重命名文件:

curl -X MOVE --header 'Destination:https://www.haiyun.me/webdav/newname' https://www.haiyun.me/webdav/filename

删除文件:

curl -X DELETE https://www.haiyun.me/webdav/filename

php协程异步扩展swoole使用

发布时间:January 17, 2021 // 分类:PHP // No Comments

ubuntu 20.04 php7.4编译安装swoole:

apt install php-cli php-dev libcurl4-openssl-dev
wget https://github.com/swoole/swoole-src/archive/v4.8.13.tar.gz
phpize
./configure --enable-openssl --enable-http2 --enable-swoole-curl --enable-cares
make && make install
echo 'extension=swoole.so' > /etc/php/7.4/cli/conf.d/20-swoole.ini
php --ri swoole
#减小swoole.so文件的大小
strip -s /usr/lib/php/20190902/swoole.so

swoole原生协程http客户端:

<?php
//Co::set(['hook_flags' => SWOOLE_HOOK_ALL]);
$count = 10;
Co\run(function () use (&$result, $count) {
  $wg = new \Swoole\Coroutine\WaitGroup();
  $result = [];
  for ($i = 1; $i <= $count; $i++) {
    $wg->add();
    go(function () use ($i, $wg, &$result) {
      $cli = new Swoole\Coroutine\Http\Client('www.baidu.com', 80);
      $cli->set(['timeout' => 10]);
      $cli->setHeaders([
        'Host' => 'www.baidu.com',
        'User-Agent' => 'Mozilla/5.0 Firefox/78.0',
      ]);
      $cli->get('/');
      $result[$i] = $cli->getStatusCode();
      $cli->close();
      $wg->done();
    });
  }
  $wg->wait();
});
var_dump($result);

以hook方式协程运行php curl:

<?php
//Co::set(['hook_flags' => SWOOLE_HOOK_ALL]);
Co\run(function () {
  $chan = new Swoole\Coroutine\Channel(10);
  for ($i = 1; $i <= 10; $i++) {
    go(function () use ($i, $chan) {
      $header = array(
        'User-Agent: Mozilla/5.0 Firefox/78.0'
      );
      $ch = curl_init();
      curl_setopt($ch, CURLOPT_URL, "http://www.baidu.com");
      curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
      curl_exec($ch);
      $code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
      curl_close($ch);
      $chan->push(['index' => $i, 'code' => $code]);
    });
  }
  for ($i = 1; $i <= 10; $i++) {
    $res = $chan->pop();
    var_dump($res);
  }
});

高性能libcurl配合epoll的curl_multi_socket_action方法使用

发布时间:February 3, 2015 // 分类: // No Comments

libcurl对大量请求连接提供了管理socket的方法,用户可使用select/poll/epoll事件管理器监控socket事件,可读写时通知libcurl读写数据,libcurl读写完成后再通知用户程序改变监听socket状态。
两个重要的设置:
curl_multi_setopt(g.multi, CURLMOPT_SOCKETFUNCTION, sock_cb);
curl_multi_setopt(multi, CURLMOPT_TIMERFUNCTION, multi_timer_cb);
1.当使用curl_multi_add_handle(g->multi, conn->easy)添加请求时会回调multi_timer_cb,然后调用curl_multi_socket_action(g->multi, CURL_SOCKET_TIMEOUT, 0, &g->still_running)初始化请求并得到一个socket(fd)。
2.调用sock_cb回调函数,传入新建的sockfd,根据传入的what状态添加到相应的事件管理器,如封装epoll的libev或libevent。
3.当事件管理器发现socket状态改变时通过curl_multi_socket_action(g->multi, fd, action, &g->still_running)通知libcurl读写数据,然后再调用sock_cb通知事件管理器,如此反复。

libcurl官网提供的基于libevlibevent事件管理示例:
http://curl.haxx.se/libcurl/c/hiperfifo.html
http://curl.haxx.se/libcurl/c/evhiperfifo.html

PHP重新动态编译Curl扩展添加异步DNS支持c-ares

发布时间:January 30, 2015 // 分类: // No Comments

pycurl支持异步DNS支持c-ares类似,请先确定Libcurl是否已支持异步DNS解析c-ares,如不支持可升级libcurl支持异步DNS解析c-ares
理论上的libcurl更新添加支持异步DNS解析后,将库文件通过ldconfig添加到系统动态库,如果大版本号和之前版本相同,可以不用重新编译Php curl扩展已支持异步DNS,因为PHP curl依赖libcurl会自动选择版本较高的lib。
如查看系统共享库中的Libcurl:

ldconfig -p|grep curl
        libcurl.so.4 (libc6,x86-64) => /usr/local/curl/lib/libcurl.so.4
        libcurl.so.4 (libc6,x86-64) => /usr/lib64/libcurl.so.4
        libcurl.so (libc6,x86-64) => /usr/local/curl/lib/libcurl.so
        libcurl.so (libc6,x86-64) => /usr/lib64/libcurl.so

然后查看php curl扩展的共享库依赖,可见已自动选择新编译的libcurl.so.4:

ldd /usr/lib64/php/modules/curl.so|grep curl
        libcurl.so.4 => /usr/local/curl/lib/libcurl.so.4 (0x00007f406f0f9000)

不过为了稳定可以重新编译下php curl扩展,本文PHPyum安装,如果是源码安装的可参考:PHP动态编译添加IMAP模块支持
查看已安装的PHP版本,并下

php -v
PHP 5.5.21 (cli) (built: Jan 21 2015 15:35:14) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies

下载相应版本的源码并重新编译curl扩展:

yum install php-devel
wget https://github.com/php/php-src/archive/php-5.5.21.tar.gz
tar zxvf php-5.5.21.tar.gz 
cd php-src-php-5.5.21/ext/curl/
phpize 
./configure --with-curl=/usr/local/curl

查看当前PHP CURL是否已支持异步DNS支持:

php -i|grep AsynchDNS
AsynchDNS => Yes
分类
最新文章
最近回复
  • 海运: 恩山有很多。
  • swsend: 大佬可以分享一下固件吗,谢谢。
  • Jimmy: 方法一 nghtp3步骤需要改成如下才能编译成功: git clone https://git...
  • 海运: 地址格式和udpxy一样,udpxy和msd_lite能用这个就能用。
  • 1: 怎么用 编译后的程序在家里路由器内任意一台设备上运行就可以吗?比如笔记本电脑 m参数是笔记本的...
  • 孤狼: ups_status_set: seems that UPS [BK650M2-CH] is ...
  • 孤狼: 擦。。。。apcupsd会失联 nut在冲到到100的时候会ONBATT进入关机状态,我想想办...
  • 海运: 网络,找到相应的url编辑重发请求,firefox有此功能,其它未知。
  • knetxp: 用浏览器F12网络拦截或监听后编辑重发请求,修改url中的set为set_super,将POS...
  • Albert: 啊啊啊啊啊啊啊啊啊 我太激动了,终于好了英文区搜索了半天,翻遍了 pve 论坛没找到好方法,博...