<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sisudoc-spine/src/sisudoc/ocda/abstraction/load.d, branch main</title>
<subtitle>SiSU Spine: document publishing and search (in D) 2015</subtitle>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/'/>
<entry>
<title>org files mostly headers</title>
<updated>2026-10-25T00:45:37+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-10-24T22:41:41+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=2a55a61d10739e782a82fcb438607e552148fbd9'/>
<id>2a55a61d10739e782a82fcb438607e552148fbd9</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>ocda.db: read a document language of a database</title>
<updated>2026-09-22T18:54:32+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T15:42:42+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=3f87aa600b5080135557d846ee1ba7daef9d3155'/>
<id>3f87aa600b5080135557d846ee1ba7daef9d3155</id>
<content type='text'>
A database now holds every language of its document, so reading one
means naming a language. Three small pieces:

- spineDbLanguages says what languages a file holds. Its own template
  rather than part of the reader (asking a file what it holds does not
  instantiate the object setter and the markup regexes with it).

- abstractionLoad and spineDocFromArtefact take a language and pass it
  to dbReadFile, which resolves it to a doc_id, takes that language
  document where there is one, and lists the languages and stops where
  there is a choice.

- spineArtefactLanguages turns an artefact into the documents to build:
  one for a .ssp, and for a database the languages it holds filtered by
  --lang. The artefact loop then has a language loop inside it, so a
  single .ocda.db argument builds every language, as the pod it came
  from would.

--db-round-trip takes --lang for the same reason, and  ocda-db looses
--parallel (would cause a race).

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A database now holds every language of its document, so reading one
means naming a language. Three small pieces:

- spineDbLanguages says what languages a file holds. Its own template
  rather than part of the reader (asking a file what it holds does not
  instantiate the object setter and the markup regexes with it).

- abstractionLoad and spineDocFromArtefact take a language and pass it
  to dbReadFile, which resolves it to a doc_id, takes that language
  document where there is one, and lists the languages and stops where
  there is a choice.

- spineArtefactLanguages turns an artefact into the documents to build:
  one for a .ssp, and for a database the languages it holds filtered by
  --lang. The artefact loop then has a language loop inside it, so a
  single .ocda.db argument builds every language, as the pod it came
  from would.

--db-round-trip takes --lang for the same reason, and  ocda-db looses
--parallel (would cause a race).

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>pod: read when zstd-wrapped</title>
<updated>2026-09-22T17:46:31+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T12:58:06+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=f89f0152a913cec14c6ee688a4f1b2ac7c4758ff'/>
<id>f89f0152a913cec14c6ee688a4f1b2ac7c4758ff</id>
<content type='text'>
sisupod using zstd compression recognised by first bytes.
&lt;doc&gt;.sisupod: spine reads one before it writes one, so when the name
changes (from .zip to .ssiupod) every pod already published stays
readable.

A .sisupod is one zstd frame wrapping the archive spine already builds.
The reader unwraps the bytes and hands the same archive to the same
parser, so every guard downstream is untouched: entry names, per-entry
and total size, path depth, escape and symlinks.

Which container it is comes from the first four bytes (rather than the
name). A pod published as a plain .zip reads as it always did, a
.sisupod reads, and either one renamed reads too. The suffix is still
recognised, both spellings, for the argument and for a url.

libzstd is declared rather than bound: provides the whole surface of
fifteen extern C prototypes (there is nothing to generate and no
upstream tree to track, which is the arrangement sqlite3 already has).
dub links it with "libs": [ "zstd" ]; nix needs zstd.out rather than
zstd, whose default output is the binaries and carries no library at
all.

A frame declares its uncompressed size in its own header, and for a pod
fetched over https that number is attacker controlled. The declared size
is checked against a ceiling before a buffer is asked for, a frame that
will not declare one is refused, and what comes out is checked against
what was promised. The ceiling is the extraction limit the archive
reader already applies, so the two bounds agree.

Measured: output built from a .sisupod is byte identical to output built
from the same pod's .zip, 359 files over text, html, epub, odt and .ssp
for three documents, live-manual's ten languages included. A truncated
frame is refused and the document skipped.

  free_culture           2863504 -&gt; 1099736   2.60x
  the_wealth_of_networks 4294022 -&gt; 1172649   3.66x
  live-manual            6972418 -&gt;  675970  10.31x

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
sisupod using zstd compression recognised by first bytes.
&lt;doc&gt;.sisupod: spine reads one before it writes one, so when the name
changes (from .zip to .ssiupod) every pod already published stays
readable.

A .sisupod is one zstd frame wrapping the archive spine already builds.
The reader unwraps the bytes and hands the same archive to the same
parser, so every guard downstream is untouched: entry names, per-entry
and total size, path depth, escape and symlinks.

Which container it is comes from the first four bytes (rather than the
name). A pod published as a plain .zip reads as it always did, a
.sisupod reads, and either one renamed reads too. The suffix is still
recognised, both spellings, for the argument and for a url.

libzstd is declared rather than bound: provides the whole surface of
fifteen extern C prototypes (there is nothing to generate and no
upstream tree to track, which is the arrangement sqlite3 already has).
dub links it with "libs": [ "zstd" ]; nix needs zstd.out rather than
zstd, whose default output is the binaries and carries no library at
all.

A frame declares its uncompressed size in its own header, and for a pod
fetched over https that number is attacker controlled. The declared size
is checked against a ceiling before a buffer is asked for, a frame that
will not declare one is refused, and what comes out is checked against
what was promised. The ceiling is the extraction limit the archive
reader already applies, so the two bounds agree.

Measured: output built from a .sisupod is byte identical to output built
from the same pod's .zip, 359 files over text, html, epub, odt and .ssp
for three documents, live-manual's ten languages included. A truncated
frame is refused and the document skipped.

  free_culture           2863504 -&gt; 1099736   2.60x
  the_wealth_of_networks 4294022 -&gt; 1172649   3.66x
  live-manual            6972418 -&gt;  675970  10.31x

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>spine: a .ocda.db can be fetched, as a pod zip can</title>
<updated>2026-09-12T16:14:40+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-11T18:26:09+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=25d3ab8e6524697ab3f2972f6dba1fbe26fcb0ec'/>
<id>25d3ab8e6524697ab3f2972f6dba1fbe26fcb0ec</id>
<content type='text'>
A URL ending in .ocda.db is now downloaded and processed in place,
through the path that already did it for .zip: the same curl call, the
same size and timeout limits, the same refusal of local and private
addresses, the same --allow-downloads guard, the same temp file cleanup.
Only the pattern had to widen.

  rgx_url_zip     ^https?://...[.]zip$
  rgx_url_source  ^https?://...([.]zip|[.]ocda[.]db)$

downloadZipUrl is no downloadSourceUrl and the download temp directory
is spine-download rather than spine-zip-pod; (the extraction directory
keeps its name, being still only for zips).

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A URL ending in .ocda.db is now downloaded and processed in place,
through the path that already did it for .zip: the same curl call, the
same size and timeout limits, the same refusal of local and private
addresses, the same --allow-downloads guard, the same temp file cleanup.
Only the pattern had to widen.

  rgx_url_zip     ^https?://...[.]zip$
  rgx_url_source  ^https?://...([.]zip|[.]ocda[.]db)$

downloadZipUrl is no downloadSourceUrl and the download temp directory
is spine-download rather than spine-zip-pod; (the extraction directory
keeps its name, being still only for zips).

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>html metadata: a link to the ocda.db</title>
<updated>2026-09-09T23:04:06+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-08T22:44:45+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=c6e822e3ce4b096163e0016e5735ef9c16e8997f'/>
<id>c6e822e3ce4b096163e0016e5735ef9c16e8997f</id>
<content type='text'>
The metadata page gains a line between the markup source and the
source digests.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The metadata page gains a line between the markup source and the
source digests.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>ocda loader: one way in whatever the source</title>
<updated>2026-09-09T22:22:47+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-08T17:52:43+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=c7049608de7d163137521495665753ae7070be08'/>
<id>c7049608de7d163137521495665753ae7070be08</id>
<content type='text'>
sisudoc.ocda.abstraction.load names the five things a document can
be read from, tells them apart, and loads the two that are
self-describing artefacts:

  .sst / .ssm  + images   the markup source
  pod (dir)    + images   the same, bundled
  pod .zip                the same, zipped
  .ssp         + images   the abstraction, as text
  .ocda.db                the abstraction, sqlite, images inside

abstractionSourceOf(path) is the detection, by name and for a
directory by whether it holds pod.manifest. abstractionLoad(path)
returns a LoadedAbstraction: the source kind, whether it was
loaded, why not when it was not, and the document itself.

The three source forms are deliberately not loaded here. Reading
them is the parser's job (sisudoc.ocda.meta.metadoc
spineAbstraction) and it needs the manifest, environment and
configuration that spine.d assembles, none of which belongs in a
loader. What this gives that case is the dispatch and a plain
statement of where it is handled, rather than a silent empty
result.

  spine --abstraction-source=&lt;path&gt;

says what a path is and, for an artefact, loads it and reports
what came back: the document, title and author, header block
sizes, object counts and the objects in each section. Exit 0 when
an abstraction was loaded, 1 when not.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
sisudoc.ocda.abstraction.load names the five things a document can
be read from, tells them apart, and loads the two that are
self-describing artefacts:

  .sst / .ssm  + images   the markup source
  pod (dir)    + images   the same, bundled
  pod .zip                the same, zipped
  .ssp         + images   the abstraction, as text
  .ocda.db                the abstraction, sqlite, images inside

abstractionSourceOf(path) is the detection, by name and for a
directory by whether it holds pod.manifest. abstractionLoad(path)
returns a LoadedAbstraction: the source kind, whether it was
loaded, why not when it was not, and the document itself.

The three source forms are deliberately not loaded here. Reading
them is the parser's job (sisudoc.ocda.meta.metadoc
spineAbstraction) and it needs the manifest, environment and
configuration that spine.d assembles, none of which belongs in a
loader. What this gives that case is the dispatch and a plain
statement of where it is handled, rather than a silent empty
result.

  spine --abstraction-source=&lt;path&gt;

says what a path is and, for an artefact, loads it and reports
what came back: the document, title and author, header block
sizes, object counts and the objects in each section. Exit 0 when
an abstraction was loaded, 1 when not.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
</feed>
