The path to the target CDB file that contains the archived static content.</blockquote>
<p><ttclass="docutils literal"><spanclass="pre">--compress</span></tt>, and <ttclass="docutils literal"><spanclass="pre">--compress-level</span></tt></p>
<blockquote>
<p>Each individual file (and consequently of the corresponding HTTP response body) is compressed with either <ttclass="docutils literal">gzip</tt>, <ttclass="docutils literal">zopfli</tt> or <ttclass="docutils literal">brotli</tt>; by default (or alternatively with <ttclass="docutils literal">identity</tt>) no compression is used.</p>
<p>Even if compression is explicitly requested, if the compression ratio is bellow a certain threshold (depending on the uncompressed size), the file is stored without any compression.
(It's senseless to force the client to spend time and decompress the response body if that time is not recovered during network transmission.)</p>
<p>The compression level can be chosen, the value depending on the algorithm:</p>
<ulclass="simple">
<li><ttclass="docutils literal">gzip</tt> -- <ttclass="docutils literal"><spanclass="pre">-1</span></tt> for algorithm default, <ttclass="docutils literal"><spanclass="pre">-2</span></tt> for Huffman only, <ttclass="docutils literal">0</tt> to <ttclass="docutils literal">9</tt> for fast to slow;</li>
<li><ttclass="docutils literal">zopfli</tt> -- <ttclass="docutils literal"><spanclass="pre">-1</span></tt> for algorithm default, <ttclass="docutils literal">0</tt> to <ttclass="docutils literal">30</tt> iterations for fast to slow;</li>
<li><ttclass="docutils literal">brotli</tt> -- <ttclass="docutils literal"><spanclass="pre">-1</span></tt> for algorithm default, <ttclass="docutils literal">0</tt> to <ttclass="docutils literal">9</tt> for fast to slow, <ttclass="docutils literal"><spanclass="pre">-2</span></tt> for extreme;</li>
<li>(by "algorithm default", it is meant "what that algorithm considers the recommended default compression level";)</li>
<li><ttclass="docutils literal">kawipiko</tt> by default uses the maximum compression level for each algorithm; (i.e. <ttclass="docutils literal">9</tt> for <ttclass="docutils literal">gzip</tt>, <ttclass="docutils literal">30</tt> for <ttclass="docutils literal">zopfli</tt>, and <ttclass="docutils literal"><spanclass="pre">-2</span></tt> for <ttclass="docutils literal">brotli</tt>;)</li>
<p><ttclass="docutils literal"><spanclass="pre">--compress-cache</span><path></tt>, and <ttclass="docutils literal"><spanclass="pre">--sources-cache</span><path></tt></p>
<p>At the given path a single file is created (that is an BBolt database), that will be used to cache the following information:</p>
<ulclass="simple">
<li>in case of <ttclass="docutils literal"><spanclass="pre">--sources-cache</span></tt>, the fingerprint of each file contents is stored, so that if the file was not changed, re-reading it shouldn't be attempted unless it is absolutely necessary; also if the file is small enough, its contents is stored in this database (deduplicated by its fingerprint);</li>
<li>in case of <ttclass="docutils literal"><spanclass="pre">--compress-cache</span></tt> the compression outcome of each file contents is stored (deduplicated by its fingerprint), so that compression is done only once over multiple runs;</li>
</ul>
<p>Each of these caches can be safely reused between multiple related archives, especially when they have many files in common.
Each of these caches can be independently used (or shared).</p>
<p>Using these caches allows one to very quickly rebuild an archive when only a couple of files have been changed, without even touching the file-system for the unchanged ones.</p>
Disables using <ttclass="docutils literal">_index.*</tt> and <ttclass="docutils literal">index.*</tt> files (where <ttclass="docutils literal">.*</tt> is one of <ttclass="docutils literal">.html</tt>, <ttclass="docutils literal">.htm</tt>, <ttclass="docutils literal">.xhtml</tt>, <ttclass="docutils literal">.xht</tt>, <ttclass="docutils literal">.txt</tt>, <ttclass="docutils literal">.json</tt>, and <ttclass="docutils literal">.xml</tt>) to respond to a request whose URL path ends in <ttclass="docutils literal">/</tt> (corresponding to the folder wherein <ttclass="docutils literal">_index.*</tt> or <ttclass="docutils literal">index.*</tt> file is located).
(This can be used to implement "slash" blog style URL's like <ttclass="docutils literal">/blog/whatever/</tt> which maps to <ttclass="docutils literal">/blog/whatever/index.html</tt>.)</blockquote>
Disables using a file with the suffix <ttclass="docutils literal">.html</tt>, <ttclass="docutils literal">.htm</tt>, <ttclass="docutils literal">.xhtml</tt>, <ttclass="docutils literal">.xht</tt>, and <ttclass="docutils literal">.txt</tt> to respond to a request whose URL does not exactly match an existing file.
(This can be used to implement "suffix-less" blog style URL's like <ttclass="docutils literal">/blog/whatever</tt> which maps to <ttclass="docutils literal">/blog/whatever.html</tt>.)</blockquote>
Disables adding an <ttclass="docutils literal"><spanclass="pre">Cache-Control:</span> public, immutable, <spanclass="pre">max-age=3600</span></tt> header that forces the browser (and other intermediary proxies) to cache the response for an hour (the <ttclass="docutils literal">public</tt> and <ttclass="docutils literal"><spanclass="pre">max-age=3600</span></tt> arguments), and furthermore not request it even on reloads (the <ttclass="docutils literal">immutable</tt> argument).</blockquote>
<p>Enables adding an <ttclass="docutils literal">ETag</tt> response header that contains the SHA256 of the response body.</p>
<p>By not including the <ttclass="docutils literal">ETag</tt> header (i.e. the default), and because identical headers are stored only one, if one has many files of the same type (that in turn without <ttclass="docutils literal">ETag</tt> generates the same headers), this can lead to significant reduction in stored headers blocks, including reducing RAM usage.
(At this moment it does not support HTTP conditional requests, i.e. the <ttclass="docutils literal"><spanclass="pre">If-None-Match</span></tt>, <ttclass="docutils literal"><spanclass="pre">If-Modified-Since</span></tt> and their counterparts; however this <ttclass="docutils literal">ETag</tt> header might be used in conjuction with <ttclass="docutils literal">HEAD</tt> requests to see if the resource has changed.)</p>
(For example, by default, if <cite>/file</cite> exists, then there is also a <cite>/file/</cite> redirect towards <cite>/file</cite>; and vice-versa from <cite>/folder</cite> towards <cite>/folder/</cite>.)</blockquote>
Disables the creation of an internal list of references that can be used in conjunction with the <ttclass="docutils literal"><spanclass="pre">--index-all</span></tt> flag of the <ttclass="docutils literal"><spanclass="pre">kawipiko-server</span></tt>.</blockquote>
It will log various information about the archived files (including compression statistics).</blockquote>
</div>
<divclass="section"id="ignored-files">
<h1>Ignored files</h1>
<ulclass="simple">
<li>any file with the following prefixes: <ttclass="docutils literal">.</tt>, <ttclass="docutils literal">#</tt>;</li>
<li>any file with the following suffixes: <ttclass="docutils literal">~</tt>, <ttclass="docutils literal">#</tt>, <ttclass="docutils literal">.log</tt>, <ttclass="docutils literal">.tmp</tt>, <ttclass="docutils literal">.temp</tt>, <ttclass="docutils literal">.lock</tt>;</li>
<li>any file that contains the following: <ttclass="docutils literal">#</tt>;</li>
<li>any file that exactly matches the following: <ttclass="docutils literal">Thumbs.db</tt>, <ttclass="docutils literal">.DS_Store</tt>;</li>
<li>(at the moment these rules are not configurable through flags;)</li>
</ul>
</div>
<divclass="section"id="wildcard-files">
<h1>Wildcard files</h1>
<p>By placing a file whose name matches <ttclass="docutils literal">_wildcard.*</tt> (i.e. with the prefix <ttclass="docutils literal">_wildcard.</tt> and any other suffix), it will be used to respond to any request whose URL fails to find a "better" match.</p>
<p>These wildcard files respect the folder hierarchy, in that wildcard files in (direct or transitive) subfolders override the wildcard file in their parents (direct or transitive).</p>
<p>In addition to <ttclass="docutils literal">_wildcard.*</tt>, there is also support for <ttclass="docutils literal">_200.html</tt> (or just <ttclass="docutils literal">200.html</tt>), plus <ttclass="docutils literal">_404.html</tt> (or just <ttclass="docutils literal">404.html</tt>).</p>
</div>
<divclass="section"id="redirect-files">
<h1>Redirect files</h1>
<p>By placing a file whose name is <ttclass="docutils literal">_redirects</tt> (or <ttclass="docutils literal">_redirects.txt</tt>), it instructs the archiver to create redirect responses.</p>
<p>The syntax is quite simple:</p>
<preclass="literal-block">
# This is a comment.
# NOTE: Absolute paths are allowed only at the top of the sources folder.
/some-path https://example.com/ 301
# NOTE: Relative paths are always, and are reinterpreted as relative to the containing folder.
./some-path https://example.com/ 302
# NOTE: Redirects only for a specific domain. (The protocol is irelevant.)
# (Allowed only at the top of the sources folder.)
<h1>Symlinks, hardlinks, loops, and duplicated files</h1>
<p>You freely use symlinks (including pointing outside of the content root) and they will be crawled during archival respecting the "logical" hierarchy they introduce.
(Any loop that you introduce into the hierarchy will be ignored and a warning will be issued.)</p>
<p>You can safely symlink or hardlink the same file (or folder) in multiple places (within the content hierarchy), and its data will be stored only once.
(The same applies to duplicated files that have exactly the same data.)</p>
