Varnish やってみる

Varnish

やったこと

クライアント => Varnish => Nginx （Varnish にまるごとキャッシュさせてしまう）

$ curl --head local.vagrant
HTTP/1.1 200 OK
Server: nginx/1.10.3 (Ubuntu)
Date: Sat, 14 Oct 2017 14:13:11 GMT
Content-Type: text/html
Last-Modified: Sat, 14 Oct 2017 12:39:07 GMT
ETag: W/"59e2056b-264"
Vary: Accept-Encoding
X-Varnish: 10 8
Age: 44
Via: 1.1 varnish (Varnish/5.2)
Accept-Ranges: bytes
Connection: keep-alive

f:id:whitech0c0late:20171014231144p:plain

実際試したのは、

ブラウザ => Vagrant => Varnish => Nginx

前準備

これで、Vagrant の hostname に設定したやつでアクセスできる

$ vagrant plugin install vagrant-share
$ vagrant plugin install vagrant-hostsupdater

Vagrantfile

Vagrant.configure("2") do |config|

  config.vm.box = "bento/ubuntu-16.04"

  config.vm.define :ubuntu do |ubuntu|
    ubuntu.vm.network :forwarded_port, guest: 80, host: 80
    ubuntu.vm.network :forwarded_port, guest: 443, host: 443
    ubuntu.vm.hostname = "local.vagrant"
    ubuntu.vm.network :private_network, ip: "192.168.33.10"
  end

  config.vm.provision "shell", inline: <<-SHELL
    sudo sh -c 'echo 127.0.1.1 $(hostname) >> /etc/hosts'
    sudo timedatectl set-timezone Asia/Tokyo

    # nginx
    wget -qO - https://nginx.org/keys/nginx_signing.key | sudo apt-key add -
    sudo sh -c 'deb http://nginx.org/packages/mainline/ubuntu/ xenial nginx' >> /etc/apt/sources.list.d/nginx.list
    sudo sh -c 'deb-src http://nginx.org/packages/mainline/ubuntu/ xenial nginx' >> /etc/apt/sources.list.d/nginx.list

    # varnish
    curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/script.deb.sh | sudo bash

    sudo apt-get update -y && sudo apt-get upgrade -y
    sudo apt-get install -y nginx varnish
  SHELL
end

それぞれ、最新のやつをインスコできたのを確認

vagrant@ubuntu:~$ varnishd -V
varnishd (varnish-5.2.0 revision 4c4875cbf)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS

vagrant@ubuntu:~$ nginx -v
nginx version: nginx/1.10.3 (Ubuntu)
  • 80 で受ける (vcl の指定とか、メモリの割当とかもよしなにカエル)
  • nginx は 8080 で受け付けるようにしておく
$ sudo cp /etc/varnish/default.vcl /etc/varnish/default.vcl.bak
$ sudo systemctl stop varnish

conf

$ sudo vim /lib/systemd/system/varnish.service
$ sudo systemctl daemon-reload

VCL

  • VCL(Varnish Configuration Language) にいろいろ設定かく
  • 基本的なのみてみるやってみる
backend default {
  .host = "127.0.0.1";
  .port = "8080";
}

TTL の設定

vcl_backend_responseサブルーチンを上書く。

sub vcl_backend_response {
  set beresp.ttl = 5m;
}
  • デフォルトでは、2分間キャッシュ。
  • 理想的なTTLは、サイトのコンテンツの更新頻度や処理する必要のあるトラフィックの量によって適切に設定する

キャッシュしない

  • 以下の例では、hoge.com かつ、/admin の条件のときキャッシュしない
  • (Varnish がリクエストを受けるたびに、実行される vcl_recv を上書く)
sub vcl_recv {
  if (req.http.host == "hoge.com" && req.url ~ "^/admin") {
    return (pass);
  }
}

Cookieの設定を解除する

  • Cookie を用いて、動的なコンテンツを表示している場合はキャッシュ動作を無効にする必要がある
sub vcl_recv {
  if ((req.http.host == "hoge.com" && req.url ~ "^/admin") ||
       req.http.Cookie == "logged_in") {
    eturn (pass);
  }
  
  # Cookie 設定解除
  unset req.http.Cookie;
}

POST のリクエスト

  • すべての POSTリクエストをスキップする
sub vcl_recv {
    if ((req.http.host == "hoge.com" && req.url ~ "^/admin") ||
         req.http.Cookie == "logged_in" ||
         req.method == "POST") {
      return (pass);
    }
    
    # Cookie 設定解除
    unset req.http.Cookie;
}

高可用性

  • バックエンドのサーバをチェックし、障害時はキャッシュコンテンツを返し続ける
  • (例：1秒ごとのチェックで、直近の10回のうち8回の応答に50msかかる場合)
backend default {
  .host = '127.0.0.1';
  .port = '8080';
  .probe = {
    .url = "/healthcheck";
    .timeout = 50ms;
    .interval = 1s;
    .window = 10;
    .threshold = 8;
  }
}
set beresp.grace = 1h;

を指定すると、バックエンドのサーバへ1h リクエストを送ることを停止できる

ログ

生ログ（１リクエスト分）

$ sudo varnishlog

*   << BeReq    >> 32773
-   Begin          bereq 32772 fetch
-   Timestamp      Start: 1507990846.416902 0.000000 0.000000
-   BereqMethod    GET
-   BereqURL       /
-   BereqProtocol  HTTP/1.1
-   BereqHeader    Host: local.vagrant
-   BereqHeader    Pragma: no-cache
-   BereqHeader    Upgrade-Insecure-Requests: 1
-   BereqHeader    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
-   BereqHeader    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
-   BereqHeader    Accept-Language: ja,en-US;q=0.8,en;q=0.6
-   BereqHeader    X-Forwarded-For: 192.168.33.1
-   BereqHeader    Accept-Encoding: gzip
-   BereqHeader    X-Varnish: 32773
-   VCL_call       BACKEND_FETCH
-   VCL_return     fetch
-   BackendOpen    27 boot.default 127.0.0.1 8080 127.0.0.1 52048
-   BackendStart   127.0.0.1 8080
-   Timestamp      Bereq: 1507990846.417464 0.000562 0.000562
-   Timestamp      Beresp: 1507990846.417485 0.000583 0.000021
-   BerespProtocol HTTP/1.1
-   BerespStatus   200
-   BerespReason   OK
-   BerespHeader   Server: nginx/1.10.3 (Ubuntu)
-   BerespHeader   Date: Sat, 14 Oct 2017 14:20:46 GMT
-   BerespHeader   Content-Type: text/html
-   BerespHeader   Last-Modified: Sat, 14 Oct 2017 12:39:07 GMT
-   BerespHeader   Transfer-Encoding: chunked
-   BerespHeader   Connection: keep-alive
-   BerespHeader   ETag: W/"59e2056b-264"
-   BerespHeader   Content-Encoding: gzip
-   TTL            RFC 120 10 0 1507990846 1507990846 1507990846 0 0
-   VCL_call       BACKEND_RESPONSE
-   VCL_return     deliver
-   BerespHeader   Vary: Accept-Encoding
-   Storage        malloc s0
-   ObjProtocol    HTTP/1.1
-   ObjStatus      200
-   ObjReason      OK
-   ObjHeader      Server: nginx/1.10.3 (Ubuntu)
-   ObjHeader      Date: Sat, 14 Oct 2017 14:20:46 GMT
-   ObjHeader      Content-Type: text/html
-   ObjHeader      Last-Modified: Sat, 14 Oct 2017 12:39:07 GMT
-   ObjHeader      ETag: W/"59e2056b-264"
-   ObjHeader      Content-Encoding: gzip
-   ObjHeader      Vary: Accept-Encoding
-   Fetch_Body     2 chunked stream
-   Gzip           u F - 384 612 80 80 3004
-   BackendReuse   27 boot.default
-   Timestamp      BerespBody: 1507990846.417679 0.000777 0.000194
-   Length         384
-   BereqAcct      431 0 431 258 384 642
-   End

*   << Request  >> 32772
-   Begin          req 32771 rxreq
-   Timestamp      Start: 1507990846.416653 0.000000 0.000000
-   Timestamp      Req: 1507990846.416653 0.000000 0.000000
-   ReqStart       192.168.33.1 49361
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: local.vagrant
-   ReqHeader      Connection: keep-alive
-   ReqHeader      Pragma: no-cache
-   ReqHeader      Cache-Control: no-cache
-   ReqHeader      Upgrade-Insecure-Requests: 1
-   ReqHeader      User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
-   ReqHeader      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
-   ReqHeader      Accept-Encoding: gzip, deflate
-   ReqHeader      Accept-Language: ja,en-US;q=0.8,en;q=0.6
-   ReqHeader      X-Forwarded-For: 192.168.33.1
-   VCL_call       RECV
-   VCL_return     hash
-   ReqUnset       Accept-Encoding: gzip, deflate
-   ReqHeader      Accept-Encoding: gzip
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 32773 fetch
-   Timestamp      Fetch: 1507990846.417731 0.001078 0.001078
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Server: nginx/1.10.3 (Ubuntu)
-   RespHeader     Date: Sat, 14 Oct 2017 14:20:46 GMT
-   RespHeader     Content-Type: text/html
-   RespHeader     Last-Modified: Sat, 14 Oct 2017 12:39:07 GMT
-   RespHeader     ETag: W/"59e2056b-264"
-   RespHeader     Content-Encoding: gzip
-   RespHeader     Vary: Accept-Encoding
-   RespHeader     X-Varnish: 32772
-   RespHeader     Age: 0
-   RespHeader     Via: 1.1 varnish (Varnish/5.2)
-   VCL_call       DELIVER
-   VCL_return     deliver
-   Timestamp      Process: 1507990846.417753 0.001100 0.000022
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Content-Length: 384
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1507990846.417885 0.001232 0.000132
-   ReqAcct        440 0 440 354 384 738
-   End

*   << Session  >> 32771
-   Begin          sess 0 HTTP/1
-   SessOpen       192.168.33.1 49361 a0 192.168.33.10 80 1507990846.416404 24
-   Link           req 32772 rxreq
-   SessClose      RX_TIMEOUT 5.007
-   End

TODO

  • purge, ban
  • いろいろ conf いじる
  • 部分キャッシュ、動的キャッシュ
  • Hitch
  • Fastly (Varnish as a service ?)

すごい

Varnishによる一貫性を考慮した積極的キャッシュ戦略実験 - でこてっくろぐ ねお : http://dekotech.dekokun.info/entry/2016/12/09/170024

REF