{"id":1185,"date":"2013-01-25T10:47:18","date_gmt":"2013-01-25T02:47:18","guid":{"rendered":"http:\/\/www.strongd.net\/?p=1185"},"modified":"2013-06-18T17:08:36","modified_gmt":"2013-06-18T09:08:36","slug":"nginx-1-2-0-versus-resin-4-0-29-performance-tests","status":"publish","type":"post","link":"https:\/\/www.strongd.net\/?p=1185","title":{"rendered":"NginX 1.2.0 versus Resin 4.0.29 performance tests"},"content":{"rendered":"<p>Recently, we decided to spend some extra time improving Resin&#8217;s performance and scalability. Since we like challenges, we set a goal of meeting or beating nginx, a fast C-based web server. When working on performance, we use benchmarks to see which proposed changes improve Resin&#8217;s performance, and which proposed changes do not. The autobench\/httperf benchmark is particularly interesting because it simulates high load rates and exposes scalability issues better than some other benchmarks like ab. After completing the Resin performance work, it turned out that we exceeded our goal and were actually able to beat nginx, and thought we&#8217;d share our results.<\/p>\n<p>We have recently run some performance benchmarks comparing Resin 4.0.29 versus NginX 1.2.0. These benchmarks show that Resin Pro matches or exceeds NginX&#8217;s throughput.<\/p>\n<p>We tested 0k (empty HTML page), 1K, 8K and 64K byte files. At every level Resin matched or exceeded NginX performance.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<div dir=\"ltr\" lang=\"en\">\n<table id=\"toc\">\n<tbody>\n<tr>\n<td>\n<div id=\"toctitle\">\n<h2>Contents<\/h2>\n<\/div>\n<ul>\n<li><a href=\"#Benchmark_tools\">1\u00a0Benchmark tools<\/a>\n<ul>\n<li><a href=\"#httperf\">1.1\u00a0httperf<\/a><\/li>\n<li><a href=\"#AutoBench\">1.2\u00a0AutoBench<\/a>\n<ul>\n<li><a href=\"#Setup_Overview\">1.2.1\u00a0Setup Overview<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#Configuration\">1.3\u00a0Configuration<\/a><\/li>\n<li><a href=\"#Hardware_Software_Specifications\">1.4\u00a0Hardware Software Specifications<\/a>\n<ul>\n<li><a href=\"#Client_HW.2FOS_specs:\">1.4.1\u00a0Client HW\/OS specs:<\/a><\/li>\n<li><a href=\"#Server_HW_specs:\">1.4.2\u00a0Server HW specs:<\/a><\/li>\n<li><a href=\"#Test_software:\">1.4.3\u00a0Test software:<\/a><\/li>\n<li><a href=\"#Software_under_test:\">1.4.4\u00a0Software under test:<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#0k_test\">2\u00a00k test<\/a>\n<ul>\n<li><a href=\"#Command_Line_Arguments\">2.1\u00a0Command Line Arguments<\/a>\n<ul>\n<li><a href=\"#0k.sh\">2.1.1\u00a00k.sh<\/a><\/li>\n<li><a href=\"#admin.sh\">2.1.2\u00a0admin.sh<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#0k_html_file_file_0k.html\">2.2\u00a00k html file file_0k.html<\/a><\/li>\n<li><a href=\"#0k_full_Results_for_0K_test\">2.3\u00a00k full Results for 0K test<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#1K_test\">3\u00a01K test<\/a>\n<ul>\n<li><a href=\"#Command_line\">3.1\u00a0Command line<\/a>\n<ul>\n<li><a href=\"#1k.sh\">3.1.1\u00a01k.sh<\/a><\/li>\n<li><a href=\"#admin.sh_2\">3.1.2\u00a0admin.sh<\/a><\/li>\n<li><a href=\"#1k.html\">3.1.3\u00a01k.html<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#1k_full_Results_for_1K_test\">3.2\u00a01k full Results for 1K test<\/a><\/li>\n<li><a href=\"#64_K_test_results\">3.3\u00a064 K test results<\/a>\n<ul>\n<li><a href=\"#Result\">3.3.1\u00a0Result<\/a><\/li>\n<li><a href=\"#64k.sh\">3.3.2\u00a064k.sh<\/a><\/li>\n<li><a href=\"#64_test_file\">3.3.3\u00a064 test file<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2>Benchmark tools<\/h2>\n<p>The benchmark tests used the following tools:<\/p>\n<ul>\n<li>httperf<\/li>\n<li>AutoBench<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h3>httperf<\/h3>\n<p><a href=\"http:\/\/www.hpl.hp.com\/research\/linux\/httperf\/\">httperf<\/a>\u00a0is tool produced by HP for measuring web server performance. The httperf tool supports HTTP\/1.1 keepalives and SSL protocols.<\/p>\n<p>&nbsp;<\/p>\n<h3>AutoBench<\/h3>\n<p><a href=\"http:\/\/www.xenoclast.org\/autobench\/\">Autobench<\/a>\u00a0is a tool for automating the process of performing a comparative benchmark test against two a web servers.\u00a0<b>Autobench<\/b>\u00a0uses\u00a0<b>httperf<\/b>. Autobench runs httperf against each host. AutoBench increases the number of requests per seconds on each iteration. AutoBench delivers output in a format that can be easily consumed by spreadsheet tools. AutoBench has a mode where it can drive multiple clients against a set of servers to minimize the possibility of testing your client throughput instead of server throughput. The command\u00a0<b>autobenchd<\/b>\u00a0is used to run a daemon on client machines. The\u00a0<b>autobench_admin<\/b>command drives many clients to run test at same time by communicating with\u00a0<b>autobenchd<\/b>.<\/p>\n<h4>Setup Overview<\/h4>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Setup_benchmark_diagram.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Setup benchmark diagram.png\" src=\"http:\/\/wiki4.caucho.com\/images\/d\/d9\/Setup_benchmark_diagram.png\" width=\"641\" height=\"334\" \/><\/a><\/p>\n<h3>Configuration<\/h3>\n<p>The only change that was made was the\u00a0<b>worker_processes<\/b>\u00a0were set to 8 for NginX to improve throughput.<\/p>\n<h3>Hardware Software Specifications<\/h3>\n<h4>Client HW\/OS specs:<\/h4>\n<ul>\n<li>i7 4 core \/ 8 HT, 2.8 GHZ, 8Meg Cache, 8 GB RAM.<\/li>\n<li>Ubuntu 12 \/ Linux Kernel 3.2.0-26-generic<\/li>\n<\/ul>\n<h4>Server HW specs:<\/h4>\n<ul>\n<li>i7 4 core \/ 8 HT, 2.8 GHZ, 8Meg Cache, 8 GB RAM.<\/li>\n<li>Ubuntu 12 \/ Linux Kernel 3.2.0-26-generic<\/li>\n<\/ul>\n<h4>Test software:<\/h4>\n<ul>\n<li>Autobench 2.1.1<\/li>\n<li>httperf 0.9.0<\/li>\n<\/ul>\n<h4>Software under test:<\/h4>\n<ul>\n<li>Resin Pro 4.0.29<\/li>\n<li>nginx 1.2.0<\/li>\n<\/ul>\n<h2>0k test<\/h2>\n<p>We want to make sure Resin can handle as many concurrent connections as possible without glitches or blocking. The tiny 0k file is a good test for high concurrency, because it spends less time on network overhead and more time stressing the threading and internal locks. Because we used an 8-core machine, we can be certain that we&#8217;re avoiding unnecessary locks or timing problems.<\/p>\n<p>For most sites, the small file stress test simulates heavy ajax use, and small file use. As sites become more interactive, this small file test becomes ever more important.<\/p>\n<p>As a comparison of a threaded Java web server with an event-based C web server, the 0k test is a good test of the threading and event manager dispatch. Because most of the time in the test is establishing a connection or switching between requests at the very highest rate, both the threading and the event management get a tough work-out.<\/p>\n<h3>Command Line Arguments<\/h3>\n<h4>0k.sh<\/h4>\n<pre>.\/admin.sh 300000 2000 20000 1000 0k<\/pre>\n<h4>admin.sh<\/h4>\n<pre>autobench_admin\r\n--clients xen:4600,lancre:4600\r\n--uri1 \/file_$5.html\r\n--host1 ch_resin --port1 8080\r\n--uri2 \/file_$5.html\r\n--host2 ch_nginx --port2 80\r\n--num_conn $1\r\n--num_call 10\r\n--low_rate $2\r\n--high_rate $3\r\n--rate_step $4\r\n--timeout 3\r\n--file out_con$1_start$2_end$3_step$4_$5.tsv<\/pre>\n<p>Above is used to setup 300,000 connections at a rate of 20,000 to 200,000 requests per second. Each iteration increases the rate by 10,000 from 20,000 to 200,000.<\/p>\n<h3>0k html file file_0k.html<\/h3>\n<pre> &lt;html&gt; \r\n &lt;body&gt;\r\n &lt;pre&gt;&lt;\/pre&gt;\r\n &lt;\/body&gt;\r\n &lt;\/html&gt;<\/pre>\n<h3>0k full Results for 0K test<\/h3>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Resin_nginx_0k_full2.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Resin nginx 0k full2.png\" src=\"http:\/\/wiki4.caucho.com\/images\/5\/5c\/Resin_nginx_0k_full2.png\" width=\"640\" height=\"354\" \/><\/a><\/p>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Resin_nginx_response_time.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Resin nginx response time.png\" src=\"http:\/\/wiki4.caucho.com\/images\/f\/ff\/Resin_nginx_response_time.png\" width=\"640\" height=\"329\" \/><\/a><\/p>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Nginx_resin_0k_IO_throughput.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Nginx resin 0k IO throughput.png\" src=\"http:\/\/wiki4.caucho.com\/images\/1\/1d\/Nginx_resin_0k_IO_throughput.png\" width=\"640\" height=\"355\" \/><\/a><\/p>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Nginx_resin_errors_0k.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Nginx resin errors 0k.png\" src=\"http:\/\/wiki4.caucho.com\/images\/6\/63\/Nginx_resin_errors_0k.png\" width=\"640\" height=\"286\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h2>1K test<\/h2>\n<h3>Command line<\/h3>\n<h4>1k.sh<\/h4>\n<pre>.\/admin.sh 200000 1000 10000 250 1k<\/pre>\n<p>&nbsp;<\/p>\n<h4>admin.sh<\/h4>\n<pre>autobench_admin\r\n--clients xen.caucho.com:4600,lancre.caucho.com:4600 \r\n--uri1 \/file_$5.html\r\n--host1 ch_resin --port1 8080\r\n--uri2 \/file_$5.html\r\n--host2 ch_nginx --port2 80\r\n--num_conn $1\r\n--num_call 10\r\n--low_rate $2\r\n--high_rate $3\r\n--rate_step $4\r\n--timeout 3\r\n--file out_con$1_start$2_end$3_step$4_$5.tsv<\/pre>\n<p>&nbsp;<\/p>\n<h4>1k.html<\/h4>\n<pre>&lt;html&gt;\r\n&lt;body&gt;\r\n&lt;pre&gt;\r\n0 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n1 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n2 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n3 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n4 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n5 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n6 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n7 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n8 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n9 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 \r\n&lt;\/pre&gt;\r\n&lt;\/body&gt; \r\n&lt;\/html&gt;<\/pre>\n<h3>1k full Results for 1K test<\/h3>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Resin_nginx_1k_requests_per_second.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Resin nginx 1k requests per second.png\" src=\"http:\/\/wiki4.caucho.com\/images\/2\/2f\/Resin_nginx_1k_requests_per_second.png\" width=\"640\" height=\"303\" \/><\/a><\/p>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Resin_nginx_1k_response_time.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Resin nginx 1k response time.png\" src=\"http:\/\/wiki4.caucho.com\/images\/9\/92\/Resin_nginx_1k_response_time.png\" width=\"640\" height=\"307\" \/><\/a><\/p>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Resin_nginx_IO_1k.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Resin nginx IO 1k.png\" src=\"http:\/\/wiki4.caucho.com\/images\/7\/77\/Resin_nginx_IO_1k.png\" width=\"640\" height=\"303\" \/><\/a><\/p>\n<p><a href=\"http:\/\/wiki4.caucho.com\/File:Resin_nginx_1k_errors.png\"><img loading=\"lazy\" decoding=\"async\" alt=\"Resin nginx 1k errors.png\" src=\"http:\/\/wiki4.caucho.com\/images\/7\/78\/Resin_nginx_1k_errors.png\" width=\"640\" height=\"376\" \/><\/a><\/p>\n<h3>64 K test results<\/h3>\n<h4>Result<\/h4>\n<p>PUT IMAGES HERE<\/p>\n<h4>64k.sh<\/h4>\n<pre>.\/admin.sh 10 50 1500 50 64k<\/pre>\n<pre>====admin.sh====\r\n&lt;pre&gt;\r\nautobench_admin\r\n--clients xen.caucho.com:4600,lancre.caucho.com:4600 \r\n--uri1 \/file_$5.html\r\n--host1 ch_resin --port1 8080\r\n--uri2 \/file_$5.html\r\n--host2 ch_nginx --port2 80\r\n--num_conn $1\r\n--num_call 10\r\n--low_rate $2\r\n--high_rate $3\r\n--rate_step $4\r\n--timeout 3\r\n--file out_con$1_start$2_end$3_step$4_$5.tsv<\/pre>\n<h4>64 test file<\/h4>\n<pre>&lt;html&gt;\r\n&lt;body&gt;\r\n&lt;pre&gt;\r\n0\r\n0 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n1 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n2 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n3 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n4 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n5 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n6 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n7 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n8 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n9 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789\r\n\r\n[Repeats 63 more times, there are 0-7 blocks repeated 0-7 times]\r\n\r\n&lt;\/pre&gt;\r\n&lt;\/body&gt;\r\n&lt;\/html&gt;<\/pre>\n<\/div>\n<div id=\"catlinks\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Recently, we decided to spend some extra time improving Resin&#8217;s performance and scalability. Since we like challenges, we set a goal of meeting or beating nginx, a fast C-based web server. When working on performance, we use benchmarks to see which proposed changes improve Resin&#8217;s performance, and which proposed changes do not. The autobench\/httperf benchmark &hellip; <a href=\"https:\/\/www.strongd.net\/?p=1185\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">NginX 1.2.0 versus Resin 4.0.29 performance tests<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,35,20],"tags":[17],"class_list":["post-1185","post","type-post","status-publish","format-standard","hentry","category-resin","category-35","category-20","tag-resin-2"],"_links":{"self":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts\/1185","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1185"}],"version-history":[{"count":2,"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts\/1185\/revisions"}],"predecessor-version":[{"id":1234,"href":"https:\/\/www.strongd.net\/index.php?rest_route=\/wp\/v2\/posts\/1185\/revisions\/1234"}],"wp:attachment":[{"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1185"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1185"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.strongd.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1185"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}