kawipiko/documentation/manuals/server.rst

242 lines
11 KiB
ReStructuredText
Raw Normal View History

#############################################
kawipiko -- blazingly fast static HTTP server
#############################################
``kawipiko-server``
-------------------
::
>> kawipiko-server --help
2021-12-23 18:39:06 +00:00
>> kawipiko-server --man
::
--archive <path>
--archive-inmem (memory-loaded archive file)
--archive-mmap (memory-mapped archive file)
--archive-preload (preload archive in OS cache)
--bind <ip>:<port> (HTTP, only HTTP/1.1, FastHTTP)
--bind-2 <ip>:<port> (HTTP, only HTTP/1.1, Go net/http)
--bind-tls <ip>:<port> (HTTPS, only HTTP/1.1, FastHTTP)
--bind-tls-2 <ip>:<port> (HTTPS, with HTTP/2, Go net/http)
--bind-quic <ip>:<port> (HTTPS, with HTTP/3)
--http1-disable
--http2-disable
--http3-alt-svc <ip>:<port>
--tls-bundle <path> (TLS certificate bundle)
--tls-public <path> (TLS certificate public)
--tls-private <path> (TLS certificate private)
--tls-self-rsa (use self-signed RSA)
--tls-self-ed25519 (use self-signed Ed25519)
--processes <count> (of slave processes)
--threads <count> (of threads per process)
--index-all
--index-paths
--index-data-meta
--index-data-content
--hosts-disable (ignore `Host` header)
--special-pages-disable
--security-headers-disable
--security-headers-tls
--seccomp-enable
--limit-descriptors <count>
--limit-memory <MiB>
--timeout-disable
2021-12-23 18:39:06 +00:00
--report --quiet --debug
--dummy
--dummy-empty
--dummy-delay <duration>
--profile-cpu <path>
--profile-mem <path>
--version
--help (show this short help)
--man (show the full manual)
--sources-md5 (dump an `md5sum` of the sources)
--sources-cpio (dump a `cpio.gz` of the sources)
--sbom --sbom-text --sbom-json
--------
Flags
.....
``--bind <ip:port>``, ``--bind-tls <ip:port>``, ``--bind-2 <ip:port>``, ``--bind-tls-2 <ip:port>``, and ``--bind-quic <ip:port>``
The IP and port to listen for requests with:
* (insecure) HTTP/1.1 for ``--bind``, leveraging ``fasthttp`` library;
* (secure) HTTP/1.1 over TLS for ``--bind-tls``, leveraging ``fasthttp`` library;
* (insecure) HTTP/1.1 for ``--bind-2``, leveraging Go's ``net/http`` library; (not as performant as the ``fasthttp`` powered endpoint;)
* (secure) H2 or HTTP/1.1 over TLS for ``--bind-tls-2``, leveraging Go's ``net/http``; (not as performant as the ``fasthttp`` powered endpoint;)
* (secure) H3 over QUIC for ``--bind-quic``, leveraging ``github.com/lucas-clemente/quic-go`` library; (given that H3 is still a new protocol, this must be used with caution; also one should use the ``--http3-alt-svc <ip:port>``;)
* if one uses just ``--bind-tls`` (without ``--bind-tls-2``, and without ``--http2-disabled``), then the TLS endpoint is split between ``fasthttp`` for HTTP/1.1 and Go's ``net/http`` for H2;
``--tls-bundle <path>``, ``--tls-public <path>``, and ``--tls-private <path>`` (optional)
If TLS is enabled, these options allows one to specify the certificate to use, either as a single file (a bundle) or separate files (the actual public certificate and the private key).
If one doesn't specify any of these options, an embedded self-signed certificate will be used. In such case, one can choose between RSA (the ``--tls-self-rsa`` flag) or Ed25519 (the ``--tls-self-ed25519`` flag);
``--http1-disable``, ``--http2-disable``
Disables that particular protocol.
(It can be used only with ``--bind-tls-2``, given that ``fasthttp`` only supports HTTP/1.)
``--processes <count>`` and ``--threads <count>``
The number of processes and threads per each process to start. (Given Go's concurrency model, the threads count is somewhat a soft limit, hinting to the runtime the desired parallelism level.)
It is highly recommended to use one process and as many threads as there are cores.
Depending on the use-case, one can use multiple processes each with a single thread; this would reduce goroutine contention if it causes problems.
(However note that if using ``--archive-inmem``, then each process will allocate its own copy of the database in RAM; in such cases it is highly recommended to use ``--archive-mmap``.)
``--archive <path>``
The path of the CDB file that contains the archived static content.
(It can be created with the ``kawipiko-archiver`` tool.)
``--archive-inmem``
Reads the CDB file in RAM, and thus all requests are served from RAM without touching the file-system.
(The memory impact is equal to the size of the CDB archive. This can be used if enough RAM is available to avoid swapping.)
``--archive-mmap``
(**recommended**) The CDB file is `memory mapped <#mmap>`__, thus reading its data uses the kernel's file-system cache, as opposed to issuing ``read`` syscalls.
``--archive-preload``
Before starting to serve requests, read the CDB file so that its data is buffered in the kernel's file-system cache. (This option can be used with or without ``--archive-mmap``.)
``--index-all``, ``--index-paths``, ``--index-data-meta``, and ``--index-data-content``
In order to serve a request ``kawipiko`` does the following:
* given the request's path, it is used to locate the corresponding resource's metadata (i.e. response headers) and data (i.e. response body) references;
by using ``--index-paths`` a RAM-based lookup table is created to eliminate a CDB read operation for this purpose; (the memory impact is proportional to the size of all resource paths combined; given that the number of resources is acceptable, say up to a couple hundred thousand, one could safely use this option;)
* based on the resource's metadata reference, the actual metadata (i.e. the response headers) is located;
by using ``--index-data-meta`` a RAM-based lookup table is created to eliminate a CDB read operation for this purpose; (the memory impact is proportional to the size of all resource metadata blocks combined; given that the metadata blocks are deduplicated, one could safely use this option; if one also uses ``--archive-mmap`` or ``--archive-inmem``, then the memory impact is only proportional to the number of resource metadata blocks;)
* based on the resource's data reference, the actual data (i.e. the response body) is located;
by using ``--index-data-content`` a RAM-based lookup table is created to eliminate a CDB operation operation for this purpose; (the memory impact is proportional to the size of all resource data blocks combined; one can use this option to obtain the best performance; if one also uses ``--archive-mmap`` or ``--archive-inmem``, then the memory impact is only proportional to the number of resource data blocks;)
* ``--index-all`` enables all the options above;
* (depending on the use-case) it is recommended to use ``--index-paths``; if ``--exclude-etag`` was used during archival, one can also use ``--index-data-meta``;
* it is recommended to use either ``--archive-mmap`` or ``--archive-inmem``, else (especially if data is indexed) the resulting effect is that of loading everything in RAM;
``--hosts-disable``
Disables the virtual-hosts feature by ignoring the `Host` header.
``--special-pages-disable``
Disables serving a few special pages internal to the server like: ::
/__/heartbeat
/__/kawipiko/about
/__/kawipiko/version
/__/kawipiko/manual.txt
/__/kawipiko/manual.html
/__/kawipiko/sbom.txt
/__/kawipiko/sbom.json
/__/kawipiko/sources.md5
/__/kawipiko/sources.cpio
/__/kawipiko/banners/errors/403
/__/kawipiko/banners/errors/...
``--security-headers-disable``
Disables adding a few security related headers: ::
Referrer-Policy: strict-origin-when-cross-origin
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: sameorigin
``--security-headers-tls``
Enables adding the following TLS related headers to the response: ::
Strict-Transport-Security: max-age=31536000
Content-Security-Policy: upgrade-insecure-requests
These instruct the browser to always use HTTPS for the served domain.
(Useful even without HTTPS, when used behind a TLS terminator, load-balancer or proxy that do support HTTPS.)
``--seccomp-enable``
On Linux, and if supported, enable a strict ``seccomp`` filter to reduce the potential attack surface in case of a security issue.
The current filter is the minimal set of ``syscall``'s required to have the server working (thus quite safe).
At each stage (opening the archive, indexing the archive, serving the archive) the non-required ``syscall``'s are filtered.
(At the moment the filter is quite strict and determined by experimentation. If you enable ``seccomp`` and the server is ``kill``-ed, check ``auditd`` logs for the problematic ``syscall`` and open an issue report.)
``--limit-descriptors``, and ``--limit-memory``
Constrains resource usage by configuring via ``setrlimit`` either ``RLIMIT_NOFILE`` (in case of descriptors) or both ``RLIMIT_DATA`` and ``RLIMIT_AS`` (in case of memory).
``--report``
Enables periodic reporting of various metrics.
Also enables reporting a selection of metrics if certain thresholds are matched (which most likely is a sign of high-load).
``--quiet``
Disables most logging messages.
``--debug``
Enables all logging messages.
``--dummy``, ``--dummy-empty``
It starts the server in a "dummy" mode, ignoring all archive related arguments and always responding with ``hello world!\n`` (unless ``--dummy-empty`` was used) and without additional headers except the HTTP status line and ``Content-Length``.
This argument can be used to benchmark the raw performance of the underlying ``fasthttp``, Go's ``net/http``, or QUIC performance; this is the upper limit of the achievable performance given the underlying technologies.
(From my own benchmarks ``kawipiko``'s adds only about ~15% overhead when actually serving the ``hello-world.cdb`` archive.)
2021-12-23 18:39:06 +00:00
``--dummy-delay <duration>``
Enables delaying each response with a certain amount (for example ``1s``, ``1ms``, etc.)
It can be used to simulate the real-world network latencies, perhaps to see how a site with many resources loads in various conditions.
(For example, see `an experiment <https://notes.volution.ro/v1/2019/08/notes/e8700e9a/>`__ I made with an image made out of 1425 tiles.)
2021-12-23 18:39:06 +00:00
``--profile-cpu <path>``, and ``--profile-mem <path>``
Enables CPU and memory profiling using Go's profiling infrastructure.