<p>The IP and port to listen for requests with:</p>
<ulclass="simple">
<li>(insecure) HTTP/1.1 for <ttclass="docutils literal"><spanclass="pre">--bind</span></tt>, leveraging <ttclass="docutils literal">fasthttp</tt> library;</li>
<li>(secure) HTTP/1.1 over TLS for <ttclass="docutils literal"><spanclass="pre">--bind-tls</span></tt>, leveraging <ttclass="docutils literal">fasthttp</tt> library;</li>
<li>(insecure) HTTP/1.1 for <ttclass="docutils literal"><spanclass="pre">--bind-2</span></tt>, leveraging Go's <ttclass="docutils literal">net/http</tt> library; (not as performant as the <ttclass="docutils literal">fasthttp</tt> powered endpoint;)</li>
<li>(secure) H2 or HTTP/1.1 over TLS for <ttclass="docutils literal"><spanclass="pre">--bind-tls-2</span></tt>, leveraging Go's <ttclass="docutils literal">net/http</tt>; (not as performant as the <ttclass="docutils literal">fasthttp</tt> powered endpoint;)</li>
<li>(secure) H3 over QUIC for <ttclass="docutils literal"><spanclass="pre">--bind-quic</span></tt>, leveraging <ttclass="docutils literal"><spanclass="pre">github.com/lucas-clemente/quic-go</span></tt> library; (given that H3 is still a new protocol, this must be used with caution; also one should use the <ttclass="docutils literal"><spanclass="pre">--http3-alt-svc</span><ip:port></tt>;)</li>
<li>if one uses just <ttclass="docutils literal"><spanclass="pre">--bind-tls</span></tt> (without <ttclass="docutils literal"><spanclass="pre">--bind-tls-2</span></tt>, and without <ttclass="docutils literal"><spanclass="pre">--http2-disabled</span></tt>), then the TLS endpoint is split between <ttclass="docutils literal">fasthttp</tt> for HTTP/1.1 and Go's <ttclass="docutils literal">net/http</tt> for H2;</li>
</ul>
</blockquote>
<p><ttclass="docutils literal"><spanclass="pre">--tls-bundle</span><path></tt>, <ttclass="docutils literal"><spanclass="pre">--tls-public</span><path></tt>, and <ttclass="docutils literal"><spanclass="pre">--tls-private</span><path></tt> (optional)</p>
<blockquote>
<p>If TLS is enabled, these options allows one to specify the certificate to use, either as a single file (a bundle) or separate files (the actual public certificate and the private key).</p>
<p>If one doesn't specify any of these options, an embedded self-signed certificate will be used. In such case, one can choose between RSA (the <ttclass="docutils literal"><spanclass="pre">--tls-self-rsa</span></tt> flag) or Ed25519 (the <ttclass="docutils literal"><spanclass="pre">--tls-self-ed25519</span></tt> flag);</p>
(It can be used only with <ttclass="docutils literal"><spanclass="pre">--bind-tls-2</span></tt>, given that <ttclass="docutils literal">fasthttp</tt> only supports HTTP/1.)</blockquote>
<p><ttclass="docutils literal"><spanclass="pre">--processes</span><count></tt> and <ttclass="docutils literal"><spanclass="pre">--threads</span><count></tt></p>
<blockquote>
<p>The number of processes and threads per each process to start. (Given Go's concurrency model, the threads count is somewhat a soft limit, hinting to the runtime the desired parallelism level.)</p>
<p>It is highly recommended to use one process and as many threads as there are cores.</p>
<p>Depending on the use-case, one can use multiple processes each with a single thread; this would reduce goroutine contention if it causes problems.
(However note that if using <ttclass="docutils literal"><spanclass="pre">--archive-inmem</span></tt>, then each process will allocate its own copy of the database in RAM; in such cases it is highly recommended to use <ttclass="docutils literal"><spanclass="pre">--archive-mmap</span></tt>.)</p>
(<strong>recommended</strong>) The CDB file is <aclass="reference external"href="#mmap">memory mapped</a>, thus reading its data uses the kernel's file-system cache, as opposed to issuing <ttclass="docutils literal">read</tt> syscalls.</blockquote>
Before starting to serve requests, read the CDB file so that its data is buffered in the kernel's file-system cache. (This option can be used with or without <ttclass="docutils literal"><spanclass="pre">--archive-mmap</span></tt>.)</blockquote>
<p><ttclass="docutils literal"><spanclass="pre">--index-all</span></tt>, <ttclass="docutils literal"><spanclass="pre">--index-paths</span></tt>, <ttclass="docutils literal"><spanclass="pre">--index-data-meta</span></tt>, and <ttclass="docutils literal"><spanclass="pre">--index-data-content</span></tt></p>
<blockquote>
<p>In order to serve a request <ttclass="docutils literal">kawipiko</tt> does the following:</p>
<ulclass="simple">
<li>given the request's path, it is used to locate the corresponding resource's metadata (i.e. response headers) and data (i.e. response body) references;
by using <ttclass="docutils literal"><spanclass="pre">--index-paths</span></tt> a RAM-based lookup table is created to eliminate a CDB read operation for this purpose; (the memory impact is proportional to the size of all resource paths combined; given that the number of resources is acceptable, say up to a couple hundred thousand, one could safely use this option;)</li>
<li>based on the resource's metadata reference, the actual metadata (i.e. the response headers) is located;
by using <ttclass="docutils literal"><spanclass="pre">--index-data-meta</span></tt> a RAM-based lookup table is created to eliminate a CDB read operation for this purpose; (the memory impact is proportional to the size of all resource metadata blocks combined; given that the metadata blocks are deduplicated, one could safely use this option; if one also uses <ttclass="docutils literal"><spanclass="pre">--archive-mmap</span></tt> or <ttclass="docutils literal"><spanclass="pre">--archive-inmem</span></tt>, then the memory impact is only proportional to the number of resource metadata blocks;)</li>
<li>based on the resource's data reference, the actual data (i.e. the response body) is located;
by using <ttclass="docutils literal"><spanclass="pre">--index-data-content</span></tt> a RAM-based lookup table is created to eliminate a CDB operation operation for this purpose; (the memory impact is proportional to the size of all resource data blocks combined; one can use this option to obtain the best performance; if one also uses <ttclass="docutils literal"><spanclass="pre">--archive-mmap</span></tt> or <ttclass="docutils literal"><spanclass="pre">--archive-inmem</span></tt>, then the memory impact is only proportional to the number of resource data blocks;)</li>
<li><ttclass="docutils literal"><spanclass="pre">--index-all</span></tt> enables all the options above;</li>
<li>(depending on the use-case) it is recommended to use <ttclass="docutils literal"><spanclass="pre">--index-paths</span></tt>; if <ttclass="docutils literal"><spanclass="pre">--exclude-etag</span></tt> was used during archival, one can also use <ttclass="docutils literal"><spanclass="pre">--index-data-meta</span></tt>;</li>
<li>it is recommended to use either <ttclass="docutils literal"><spanclass="pre">--archive-mmap</span></tt> or <ttclass="docutils literal"><spanclass="pre">--archive-inmem</span></tt>, else (especially if data is indexed) the resulting effect is that of loading everything in RAM;</li>
<p>On Linux, and if supported, enable a strict <ttclass="docutils literal">seccomp</tt> filter to reduce the potential attack surface in case of a security issue.</p>
<p>The current filter is the minimal set of <ttclass="docutils literal">syscall</tt>'s required to have the server working (thus quite safe).
At each stage (opening the archive, indexing the archive, serving the archive) the non-required <ttclass="docutils literal">syscall</tt>'s are filtered.</p>
<p>(At the moment the filter is quite strict and determined by experimentation. If you enable <ttclass="docutils literal">seccomp</tt> and the server is <ttclass="docutils literal">kill</tt>-ed, check <ttclass="docutils literal">auditd</tt> logs for the problematic <ttclass="docutils literal">syscall</tt> and open an issue report.)</p>
</blockquote>
<p><ttclass="docutils literal"><spanclass="pre">--limit-descriptors</span></tt>, and <ttclass="docutils literal"><spanclass="pre">--limit-memory</span></tt></p>
<blockquote>
Constrains resource usage by configuring via <ttclass="docutils literal">setrlimit</tt> either <ttclass="docutils literal">RLIMIT_NOFILE</tt> (in case of descriptors) or both <ttclass="docutils literal">RLIMIT_DATA</tt> and <ttclass="docutils literal">RLIMIT_AS</tt> (in case of memory).</blockquote>
<p>It starts the server in a "dummy" mode, ignoring all archive related arguments and always responding with <ttclass="docutils literal">hello <spanclass="pre">world!\n</span></tt> (unless <ttclass="docutils literal"><spanclass="pre">--dummy-empty</span></tt> was used) and without additional headers except the HTTP status line and <ttclass="docutils literal"><spanclass="pre">Content-Length</span></tt>.</p>
<p>This argument can be used to benchmark the raw performance of the underlying <ttclass="docutils literal">fasthttp</tt>, Go's <ttclass="docutils literal">net/http</tt>, or QUIC performance; this is the upper limit of the achievable performance given the underlying technologies.
(From my own benchmarks <ttclass="docutils literal">kawipiko</tt>'s adds only about ~15% overhead when actually serving the <ttclass="docutils literal"><spanclass="pre">hello-world.cdb</span></tt> archive.)</p>
<p>Enables delaying each response with a certain amount (for example <ttclass="docutils literal">1s</tt>, <ttclass="docutils literal">1ms</tt>, etc.)</p>
<p>It can be used to simulate the real-world network latencies, perhaps to see how a site with many resources loads in various conditions.
(For example, see <aclass="reference external"href="https://notes.volution.ro/v1/2019/08/notes/e8700e9a/">an experiment</a> I made with an image made out of 1425 tiles.)</p>
</blockquote>
<p><ttclass="docutils literal"><spanclass="pre">--profile-cpu</span><path></tt>, and <ttclass="docutils literal"><spanclass="pre">--profile-mem</span><path></tt></p>
<blockquote>
Enables CPU and memory profiling using Go's profiling infrastructure.</blockquote>
