in his famous speech at the University of Texas in 2014, Admiral William H. McRaven said that if you want to change the world, start off by making your bed. Sometimes small things can have a big impact – whether it’s making your bed in the morning or making few changes to your website’s HTTP server configuration.
Does that seem like an overstatement? The first months of 2020 have flushed down the drain all definitions of what’s normal and reasonable in our world. With almost half of the Earth’s population locked down in their homes due to the COVID‑19 pandemic, the Internet has become their only mode of communication, entertainment, buying food, working, and education. And each week the Internet is seeing higher network traffic and server load than ever before. According to a report published by BroadbandNow on March 25, “Eighty eight (44%) of the 200 cities we analyzed have experienced some degree of network degradation over the past week compared to the 10 weeks prior”.
Major media platforms like Netflix and YouTube are limiting the quality of their transmissions in order to protect network links, making more bandwidth available for people to work, communicate with their families, or attend virtual lessons at their school. But still this is not enough, as network quality gradually worsens and many servers become overloaded.
If you own a website and can manage its HTTP server configuration, you can help. A few small changes can reduce the network bandwidth generated by your users and the load on servers. It’s a win‑win situation: if your site is currently under heavy load, you can reduce it, enabling you to serve more users and possibly lowering your costs. If it’s not under high load, faster loading improves your users’ experience (and sometimes positively affects your position in Google search results).
It doesn’t really matter if you have an application with millions of users each month or a small blog with baking recipes – every kilobyte of network traffic you eliminate frees capacity for someone who desperately needs to check medical testing results online or create a parcel label to send something important to relatives.
In this blog we present a few simple but powerful changes you can make to your NGINX configuration. As a real‑world example, we use the e‑commerce site of our friends at Rogalove, an ecological cosmetics manufacturer here in Poland where I live. The site is a fairly standard WooCommerce installation running NGINX 1.15.9 as its web server. For the sake of our calculations, we assume the site gets 100 unique users per day, 30% of users are recurring visitors, and each user accesses an average of 4 pages during a session.
Nginx官方博客最近发布了一篇文章《Help the World by Healing Your NGINX Configuration》,文章介绍了几种提高性能和减少网络带宽的非常实用的Nginx优化配置方法,值得参考。
Nginx官方博客最近发布了一篇文章:Help the World by Healing Your NGINX Configuration,文章介绍了几种提高性能和减少网络带宽的非常实用的Nginx优化配置方法。如果你的网站也是和easeapi一样用户量相对较小,可以直接拿来使用。
As you may know, the HTML, CSS, and JavaScript files used to build pages on modern websites can be really huge. In most situations, web servers can compress these and other text files on the fly to conserve network bandwidth.
就像你所知网站上构建页面的HTML,CSS和JavaScript文件可能非常庞大。在大多数情况下,Web服务器可以动态的压缩这些以及其他文本文件,以节省网络带宽。
One way to see if a web server is compressing files is with the browser’s developer tools. For many browsers, you access the tools with the F12 key . Here’s an example:
查看Web服务器是否正在压缩文件的一种方法是使用浏览器的开发人员工具。对于许多浏览器,您可以使用F12键访问这些工具,这是一个例子:
As you see at the bottom left, there is no compression: the text files are 1.15 MB in size and that much data was transferred.
如你在左下方所见,请求的文件没有压缩:文本文件的大小为1.15 MB,并且传输了很多数据。
By default, compression is disabled in NGINX but depending on your installation or Linux distribution, some settings might be enabled in the default nginx.conf file. Here we enable gzip compression in the NGINX configuration file:
默认情况下,NGINX当中禁用压缩,但是根据您的安装或Linux发行版,某些设置可能会在默认的nginx.conf文件中启用。在这里,我们在NGINX配置文件中启用gzip压缩:
gzip on; gzip_types application/xml application/json text/css text/javascript application/javascript; gzip_vary on; gzip_comp_level 6; gzip_min_length 1000; #压缩级别1-9,级别越高文件越小越费CPU,推荐6 gzip_comp_level 6 #gzip_min_length 1000 当返回内容大于此值时才会使用gzip进行压缩,以K为单位,当值为0时,所有页面都进行压缩 #gzip_buffers 16 8k 设置用于处理请求压缩的缓冲区数量和大小。比如32 4K表示按照内存页(one memory page)大小以4K为单位(即一个系统中内存页为4K),申请32倍的内存空间。建议此项不设置,使用默认值。As you see in the following screenshot, with compression the data transfer goes down to only 260 KB – a reduction of about 80%! For each new user on your page, you save about 917 KB of data transfer.
如你在以下屏幕截图中所见,压缩后的数据传输量仅为260 KB ,减少了约80%!对于页面上的每个新用户,您可以节省大约917 KB的数据传输。
When a browser retrieves a file for a web page, it keeps a copy in a local on‑disk cache so that it doesn’t have to refetch the file from the server when you visit the page again. Each browser uses its own logic to decide when to use a local copy of a file and when to fetch it again in case it has changed on the server. But as the website owner, you can set cache control and expiration headers in the HTTP responses you send, to make the browser’s caching behavior more efficient. In the long term you get many fewer unnecessary HTTP requests.
当浏览器检索网页的文件时,它会将副本保留在本地磁盘缓存中,这样,当你再次访问该页面时,它不必从服务器重新获取文件。每个浏览器都使用自己的逻辑来决定何时使用本地的副本文件,以及何时在服务器上更改了文件时再次获取它。但是,作为网站所有者,你可以在发送的HTTP响应中设置cache control 和 expiration 以提高浏览器的缓存行为的效率。从长远来看,您会收到很多不必要的HTTP请求。
对于资源文件来说,短期内一般不会改变,因此可以针对性的设置较大的缓存时间。当浏览器首次下载到该文件之后,在缓存期间内再次访问会从本地缓存中读取,不会再次向服务器发起请求。
For a good start, you can set a long cache expiration time for fonts and images, which probably do not change often (and even if they change, they usually get a new file name). In the following example we instruct the client browser to keep fonts and images in the local cache for a month:
首先,您可以为字体和图像设置较长的缓存过期时间,这些字体和图像可能不会经常更改(即使更改,它们通常也会获得新的文件名)。在以下示例中,我们指示客户端浏览器将字体和图像在本地缓存中保留一个月
location ~* \.(?:jpg|jpeg|gif|png|ico|woff2)$ { expires 1M; add_header Cache-Control "public"; }此项配置生效后,会在请求的头部增加下面两个字段:
//缓存时间(秒) cache-control: max-age=2592000 //浏览器和缓存服务器都可以缓存 cache-control: public Syntax: expires [modified] time; expires epoch | max | off; Default: expires off; Context: http, server, location, if in locationCache-Control”字段的内容取决于指定时间的符号:
时间为负-“Cache-Control: no-cache”。time为正或为零—“ Cache-Control:max-age = t”,其中t是指令中指定的时间,以秒为单位。该epoch参数将“ Expires”设置为值“ Thu, 01 Jan 1970 00:00:01 GMT”,将“ Cache-Control”设置为“ no-cache”。
该max参数将“ Expires”设置为值“ Thu, 31 Dec 2037 23:55:55 GMT”,并将“ Cache-Control”设置为10年。
该off参数禁用添加或修改“ Expires”和“ Cache-Control”响应头字段。
HTTP/2 is a next‑generation protocol for serving web pages, designed for better network and host‑server utilization. According to the Google documentation, it enables much faster page loading:
HTTP/2是一种下一代服务网页的协议,旨在更好地利用网络和主机服务器。根据谷歌的文档,它可以更快地加载页面:
The resulting protocol is more friendly to the network, because fewer TCP connections are used in comparison to HTTP/1.x. This means less competition with other flows, and longer‑lived connections, which in turn leads to better utilization of available network capacity.
产生的协议对网络更友好,因为与HTTP/1.x相比,使用的TCP连接更少。这意味着与其他流的竞争更少,连接寿命更长,这反过来会更好地利用可用的网络容量。
NGINX 1.9.5 and later (and NGINX Plus R7 and later) supports the HTTP/2 protocol, and all you need to do is to enable it 😀. To do so, include the http2 parameter on the listen directives in your NGINX configuration files:
listen 443 ssl http2;Note that in most cases, you also need to enable TLS to use HTTP/2.
You can verify that your (or any) site supports HTTP/2 with the HTTP2.Pro service:
HTTP/2
HTTP/2是下一代HTTP协议,也是HTTP/1.1后首次更新的大版本。2015年5月HTTP/2标准正式发布,目前大多数终端系统均已经支持HTTP/2,是时候将服务升级至HTTP/2了。相对HTTP/1.1,HTTP/2具有明显的优势。HTTP/2主要新增了如下特性:
头部压缩HTTP/1.1的头部信息有很多字段都是可以预知的,在HTTP/2中对头部信息进行了压缩,可节省网络流量。
二进制分帧HTTP/2采用二进制格式传输数据(HTTP/1.1采用的是文本格式),HTTP/2将请求和响应数据分割为更小的帧,均采用二进制编码。
多路复用HTTP/1.1中,如果想并发多个请求,必须使用多个TCP连接,且浏览器会对单个域名有6-8个的TCP请求的限制。在HTTP/2中,有了二进制分帧之后,一个域名只需要占用一个TCP连接,单个连接可以互不干扰的进行请求和响应。
服务器推送在HTTP/1.1中,终端浏览器需要先下载HTML,解析后再去加载js/css等资源。而在HTTP2中,服务器可直接向浏览器推送资源而不需要等待浏览器解析并请求。
基于以上重要的改进,HTTP/2可大幅提升性能和效率,更重要的是HTTP/2完全兼容HTTP/1.1,可以平滑过渡。Ngnix从1.9.5开始支持HTTP/2,简单配置即可:
listen 443 ssl http2
ALPN
特别注意的是,即使服务器启用HTTP/2,并不意味着在和客户端进行通信时就会使用HTTP/2,也有可能会降级到HTTP/1.1。ALPN协议允许服务端和客户端进行应用层协议的协商(类似的协议还有NPN,只不过目前Google仅支持ALPN),需要确认客户端是否支持HTTP/2。启用ALPN的要求:
Make yourself a cup of your favorite beverage, sit comfortably, and think: when was the last time you looked at your access log file? Last week, last month, never? Even if you use it for day-to-day monitoring of your site, you probably focus only on errors (400 and 500 status codes, and so on), not successful requests.
自己喝一杯自己喜欢的饮料,舒适地坐着,然后思考:上一次查看访问日志文件是什么时候?上周,上个月,从来没有?即使将其用于站点的日常监视,也可能只关注错误(400和500状态代码等),而不关注成功的请求。
By reducing or eliminating unnecessary logging, you save disk storage, CPU, and I/O operations on your server. This not only makes your server a little faster – if you’re deployed in a cloud environment, the freed‑up I/O throughput and CPU cycles might be a life saver for another virtual machine or application residing on the same physical machine.
通过减少或消除不必要的日志记录,可以节省服务器上的磁盘存储,CPU和I / O操作。这不仅使您的服务器更快一点-如果你部署在云环境中,则释放的I / O吞吐量和CPU周期可能为同一虚拟机上的其他虚拟机或应用程序节省生命。
There are several different ways to reduce and optimize logging. Here we highlight three.
有几种减少和优化日志记录的方法。在这里,我们重点介绍三个。
Method 1: Disable Logging of Requests for Page Resources(禁用资源文件的请求log)
This is a quick and easy solution if you don’t need to log requests that retrieve ordinary page resources such as images, JavaScript files, and CSS files. All you need to do is to create a new location block that matches those file types, and disable logging inside it. (You can also add this access_log directive to the location block above where we set the Cache-Control header .)
如果你不需要记录检索普通页面资源(例如图像,JavaScript文件和CSS文件)的请求,则这是一种快速简便的解决方案。你需要做的就是创建一个location块与这些文件类型匹配,并禁用其中的日志记录。(您也可以将此access_log指令添加到我们设置标头的上方的location块中。Cache-Control)
location ~* \.(jpg|jpeg|gif|png|ico|woff2|js|css)$ { access_log off; }
Method 2: Disable Logging of Successful Requests(禁用成功请求的log)
This is a more powerful method because it discards queries with a 2xx or 3xx response code, logging only errors. It is slightly more complicated than Method 1 because it depends on how your NGINX logging is configured. In our example we use the standard nginx.conf included in Ubuntu Server distributions, so that regardless of the virtual host all requests are logged to /var/log/nginx/access.log.
这是一个更强大的方法,因为它会丢弃带有2xx或3xx响应代码的查询,只记录错误。它比方法1稍微复杂一些,因为它取决于NGINX日志的配置方式。在我们的示例中,我们使用标准nginx.conf公司包括在Ubuntu服务器发行版中,因此不管虚拟主机是什么,所有请求都会记录到/var/log/nginx/access.log.
Using an example from the official NGINX documentation let’s turn on conditional logging. Create a variable $loggable and set it to 0 for requests with 2xx and 3xx response codes, and otherwise to 1. Then reference this variable as a condition in the access_log directive.
Here’s the original directive in the http context in /etc/nginx/nginx.conf:
access_log /var/log/nginx/access.log;Add a map block and reference it from the access_log directive:
禁用成功请求的log如果仅需要关心那些访问出现错误的情况,可以禁用请求成功时的log。示例配置
map $status $loggable { ~^[23] 0; default 1; } access_log /var/log/nginx/access.log combined if=$loggable;Note that although combined is the default log format, you need to specify it explicitly when including the if parameter.
Method 3: Minimizing I/O Operations with Buffering
这个也是我推荐大家使用的。大多数情况下我们还是希望能记录所有的请求日志,包括成功和失败的,以便进行流量分析。这个时候,兼顾到性能,可以考虑使用缓冲区来减小IO操作频次。
Even if you want to log all requests you can minimize I/O operations by turning on access log buffering. With this directive NGINX waits to write log data to disk until a 512-KB buffer is filled or 1 minute has passed since the last flush, whichever occurs first.
即使您希望记录所有请求,也可以通过打开访问日志缓冲来最大程度地减少I / O操作。使用此指令,NGINX将等待将日志数据写入磁盘,直到填满512 KB缓冲区或自上次刷新以来经过1分钟以后。
access_log /var/log/nginx/access.log combined buffer=512k flush=1m;
If your server provides larger files (or smaller but extremely popular files, like forms or reports), it can be useful to set the maximum speed at which clients can download them. If your site is already experiencing a high network load, limiting download speed leaves more bandwidth to keep critical parts of your application responsive. This is a very popular solution used by hardware manufacturers – you may wait longer to download a 3-GB driver for your printer, but with thousands of other people downloading at the same time you’ll still able to get your download. 😉
如果服务器提供大文件(或者体积小但非常流行的文件,如表单或报表),那么设置客户端下载这些文件的最大速度是很有用的。如果您的站点已经经历了高网络负载,那么限制下载速度会留下更多带宽来保持应用程序的关键部分响应。这是硬件制造商使用的一种非常流行的解决方案-您可能需要更长的时间来下载您的打印机的3GB驱动程序,但是在成千上万的其他人同时下载的情况下,您仍然可以获得您的下载。
Use the limit_rate directive to limit bandwidth for a particular URL. Here we’re limiting the transfer rate for each file under /download to 50 KB per second.
使用limit_rate指令限制特定URL的带宽。在这里,我们将下载下的每个文件的传输速率限制为每秒50kb。
如果网站提供有较大文件的下载,为防止一个请求占用过多的带宽而影响其他人的访问,可以对其进行限速。
location /download/ { limit_rate 50k; }You might also want to rate‑limit only larger files, which you can do with the limit_rate_after directive. In this example the first 500 KB of every file (from any directory) is transferred without speed restrictions, with everything after that capped at 50 KB/s. This enables faster delivery of critical parts of the website while slowing down others.
可能你还希望只对较大的文件进行速率限制,这可以通过使用limit_rate_after指令来实现。在本例中,每个文件(来自任何目录)500kb的传输没有速度限制,之后的所有内容都以50kb/s为上限。这样可以更快地传递网站的关键部分,同时降低其他部分的速度。
location / { limit_rate_after 500k; limit_rate 50k; }Note that rate limits apply to individual HTTP connections between a browser and NGINX, and so don’t prevent users from getting around rate limits by using download managers.
注意,速率限制适用于浏览器和NGINX之间的单个HTTP连接,因此不会阻止用户使用下载管理器绕过速率限制。
Lastly, you can also limit the number of concurrent connections to your server or the rate of request.
最后,还可以限制到服务器的并发连接数或请求速率。