| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dr_document_make in three identifier spellings becomes document_make,
which is what the file has been called since the rename.
The mixin import audit, a template declares its imports at template
scope and they are then visible in every scope it is mixed into (which
is how a template-level split once hijacked a UFCS lookup in code that
had not been touched). Disabling htmlSnippet's imports and rebuilding
names the consumers that were relying on them rather than on their own:
across twelve mixin sites, exactly one. metadata.d now imports the `to`
it uses. The templates keep their imports, their own functions needing
them, and narrowing those is a change of its own rather than a cleanup.
And the standing FIX in source_pod.d: the insert digest line for a non
pod source recorded the insert's path, where every other line in
digests.txt is a filename and a digest naming a file that is not in
the pod under that name cannot be checked against it.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
| |
One file for the document (including all languages) with digest of the
document. digests.txt lists it once, beside the shared images, taken
when the last language has been written.
digests.txt carries all digests.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A pod now carries its catalogues, so a translation can be updated
rather than only retyped: providing the .pot and .po (the means of
maintaining it).
A directory walk, where every other thing the pod writer carries is
named. A catalogue set is whatever the translator has, and enumerating
.pot and .po files would mean the writer deriving the document's
languages a second time, from a second place, to say what it already
reads from the manifest.
Done once, on the last language, as the markup blocks are. The tree is
not language specific, so a pass for each language would re-read all of
it, 5.5 MB ten times over for live-manual, into an archive the next pass
overwrites.
Every name is checked before it is carried, by the rule the pod reader
applies to an archive it is handed. The writer is reading a directory
someone else may have filled, which is the reason the reader checks.
The cost is small, and it is the whole argument for compressing the
archive as a stream rather than per member:
uncompressed content 6972418 -> 13167489 +89%
the .sisupod 692942 -> 891388 +28.6%
5.5 MB of catalogues cost 198 KB, because the .po files largely repeat
markup already in the archive and whole-stream compression sees it.
Per-member deflate could not.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Both now take the number from one constant now set at 10,000, the writer
checks it before the archive is written.
Previously the number set for the pod reader was less than 500 members.
The writer had no limit, (so spine could write a pod it would then
refuse to read, reporting too many entries: a good file that looks
corrupt, and only for a document with enough parts).
500 was set when a pod held markup and images. A pod carrying
translation catalogues has a different arithmetic, and the count follows
from how many files a document is made of times how many languages it
has, not from how large the document is:
the_wealth_of_networks 213,405 words, 1 file per language
12 languages -> 49 entries
live-manual 24,724 words, 20 files per language
12 languages -> 506 entries
The big book is not what runs into this; the modular manual is.
live-manual stood two languages from an unreadable artefact.
10,000 covers a hundred-insert manual in thirty languages, about 6,200
entries, with room. It is deliberately well clear of any real document
rather than snug above the largest one known: the two errors are not
comparable, since too low refuses a legitimate document with a message
that reads as corruption, while too high defers to a size cap a moment
later. The count is the weakest of the three guards and is not what
bounds resource use; the per entry and total size caps do that, both
before a byte is written.
Refusing also removes any archive an earlier language left. The writer
runs once per language and only the last pass holds every language, so
it is the last that goes over, and the passes before it wrote smaller
archives that passed. Without this, refusal left a pod missing a
language: an artefact that reads perfectly well and is wrong. A missing
file is an error someone notices.
The arithmetic is recorded beside the constant so the next person can
re-derive the number rather than guess at it.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pod archive spine writes is now named .sisupod and carries the
same archive inside one zstd frame. The reader has accepted both
since the previous commit, so every pod already published stays
readable and this changes only what is written.
free_culture 2863504 -> 1100180
live-manual 6972418 -> 692942 10.06x
The wrap is at the pod call site and not in createZipFile, which
also writes every epub and every odt.
Best compression from whole-stream (rather than per-member compression).
(for live-manual per-member deflate manages about 3x compared to 10x for
whole pod content compression).
The suffix is named as pod_archive_suffix.
--pod-compression sets the level, 19 by default: a pod is written once
and fetched many times (692942 bytes at 19, 837703 at 9, 2054365 at 1).
(A value that is not a number warns and the default is used).
Output built from a written .sisupod is byte identical to output built
from the markup it came from.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
| |
Spine now looks for conf/document_make, and bundles that name.
(the dr_ prefix was a sisu-era name to distinguish it)
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes for three faults in the insert (.ssi) digests.
The key was the language captured from whichever path resolved the
insert. It did not change across the loop over languages, so
every language's inserts were filed under one language. The key is now
the language being digested, as the digest of the primary file above it
already did.
- digest is taken after the check that the file exists.
- digest is now taken inside the guard.
- non-pod path now correctly keyed on the document's own language.
(assisted by Claude-Code)
|
| | |
|
| |
|
|
|
|
|
|
|
| |
ocda (object centric document abstraction)
flag --ocda-db (or --abstraction-db)
replaces --show-abstraction-db
rename results in consequently large diff
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pod/ holds all document source representations:
pod/<doc>/ source tree
pod/<doc>/media/abstraction/<uid>.ssp abstraction, as text
pod/<doc>.zip tree, zipped, .ssp included
pod/<doc>.digests.txt sha256s of what is in them
pod/<uid>.ocda.db abstraction, as sqlite db
<lang>/abstraction/ is gone.
pod/<doc>/media/abstraction/<uid>.ssp preferred as having the
images (found within the pod tree) which .ssp needs to reproduce a
document but does not carry on its own.
The .ocda.db sits carries the images as well and (like the
pod.zip) can be used to reproduce a document directly.
digests.txt now covers the database as well as the zip, the source
and the .ssp; (as does the metadata html page).
Two ordering issues addressed:
- the pod builder clean-slates pod/<doc>/ before regenerating it,
and the .ssp is now written before that runs. The clean slate
now leaves media/abstraction/ alone, and the .ssp writer clears
that directory itself on the first language of a run, so a .ssp
for a language the document no longer is removed and cannot be
bundled.
- for a multi-language document the .ssp files accumulate one
language at a time and are bundled on the last, which is why the
directory cannot simply be emptied by whichever (language) gets
there first.
(assisted by Claude-Code)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Modules and imports rewritten to sisudoc.ocda.* and
sisudoc.outputs.*; dub.json excludedSourceFiles and the
spine:abstraction sub-package sourcePaths collapsed to
./src/sisudoc/ocda.
Verified: nix build .#spine-overlay-ldc clean.
(assisted by Claude-Code)
|
|
|
create new directories under ./src/sisudoc ocda & outputs in order to
separate the document abstraction library from downstream output
processing (stuff broken till paths & modules fixed)
|