aboutsummaryrefslogtreecommitdiffhomepage
path: root/org
diff options
context:
space:
mode:
authorRalph Amissah <ralph.amissah@gmail.com>2026-09-21 10:52:39 -0400
committerRalph Amissah <ralph.amissah@gmail.com>2026-09-22 14:23:53 -0400
commite1c9056903c2478066d5da8934d704f4d1935da4 (patch)
treef3fbfee319f3947e216f996d5097480bf82d65cb /org
parenttest (ref-abstr-ssp): live-manual mended headers (diff)
abstraction: format 2.0, source.digests
format 2.0, source.digest covers every markup file, the sha256 over one line per markup file, "<sha256> <filename>", sorted by filename. source.digest was the sha256 of the master file as read, before any insert. This is right for a .sst, which is the whole document. For a .ssm with inserts .ssi (containing the substantive part of a documents text) this is close to meaningless. Considered change of meaning (rather than an addition) and given the major part of the format version with it: 1.1 becomes 2.0, in the .ssp header line and in the database's schema.version row together. A 1.x reader refuses a 2.0 artefact, which is what that check is for. - Sorted, so the value does not depend on the order a filesystem hands back a directory. - By filename (rather than by path), so it does not depend on where the pod sits, which is the property the pod rebuild comparison rests on: a pod unzipped elsewhere must still give the same abstraction. Within one language every markup file lives in one directory, so a filename identifies it. Each line is checkable on its own against a digests.txt line or a files row, which the single value was not. The whole is reproducible with sha256sum and sort, and was verified that way for a one file document and for a twenty file one. An unreadable file is named in the digest rather than skipped. The parse has failed elsewhere by then, and a digest that quietly left a file out would claim the document is something it is not. The reference abstractions are regenerated. Two lines change in each and no others: the version, and the digest. (assisted by Claude-Code)
Diffstat (limited to 'org')
-rw-r--r--org/in_markup_source_files.org54
-rw-r--r--org/out_ocda_peg_ssp.org9
-rw-r--r--org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org4
3 files changed, 64 insertions, 3 deletions
diff --git a/org/in_markup_source_files.org b/org/in_markup_source_files.org
index 7aa49dc..8d8367e 100644
--- a/org/in_markup_source_files.org
+++ b/org/in_markup_source_files.org
@@ -343,6 +343,36 @@ template spineRawMarkupContent() {
= rawsrc.sourceContentSplitIntoHeaderAndBody(_opt_action, rawsrc.sourceContent(fn_src), fn_src);
return _0_header_1_body_content_2_insert_filelist_struct;
}
+ /+ ↓ one line per markup file, "<sha256> <filename>", sorted, then the
+ sha256 of that. The imports are local: this template is mixed in
+ beside others, and sort or join arriving in a mixing scope takes over
+ a UFCS lookup somewhere else.
+ +/
+ @trusted private ubyte[32] _markupFilesDigest(string[] _files) {
+ import std.algorithm : sort;
+ import std.array : join;
+ import std.conv : to;
+ import std.digest : toHexString;
+ import std.file : read;
+ import std.path : baseName;
+ string[] _lines;
+ foreach (_f; _files) {
+ if (_f.length == 0) { continue; }
+ try {
+ _lines ~= (cast(ubyte[]) _f.read).sha256Of.toHexString.to!string
+ ~ " " ~ _f.baseName;
+ } catch (Exception) {
+ /+ ↓ a file the scan named and the filesystem will not give. The
+ parse has failed elsewhere already, and a digest that quietly
+ left it out would claim the document is something it is not, so
+ it is named instead.
+ +/
+ _lines ~= "UNREADABLE " ~ _f.baseName;
+ }
+ }
+ _lines.sort;
+ return _lines.join("\n").sha256Of;
+ }
struct RawMarkupContent {
final sourceContent(in string fn_src) {
auto raw = MarkupRawUnit();
@@ -378,7 +408,29 @@ template spineRawMarkupContent() {
string header_type = "";
ST_doc_digest dig;
{
- dig.markup_doc = source_txt_str.sha256Of;
+ /+ ↓ the digest of the document's markup, over every file it is made
+ of and not over the master (.ssm) alone.
+ .
+ It was source_txt_str.sha256Of: the master file as read, before any
+ insert. For a .sst, which is the whole document, that is right. For
+ a .ssm it is close to meaningless (live-manual's master is 2,143
+ bytes of title and nineteen << lines, while the document is in the
+ nineteen files those lines name). Any chapter in any language could
+ be rewritten and the digest would not move.
+ .
+ It is now the sha256 over one line per markup file,
+ "<sha256> <filename>", sorted by filename. Sorted so that the
+ value does not depend on the order a filesystem hands back a
+ directory; by filename rather than by path so that it does not
+ depend on where the pod sits, which is the property the pod
+ rebuild comparison rests on. Within one language every markup
+ file lives in one directory, so a filename identifies it.
+ .
+ Each line is checkable on its own against a digests.txt line or
+ a files row, which the old single value was not.
+ +/
+ dig.markup_doc = _markupFilesDigest(
+ [fn_src.to!string] ~ insert_file_list_get);
dig.header = st.header.sha256Of;
dig.text = sourcefile_body_content.sha256Of;
}
diff --git a/org/out_ocda_peg_ssp.org b/org/out_ocda_peg_ssp.org
index 78e9a56..9fda543 100644
--- a/org/out_ocda_peg_ssp.org
+++ b/org/out_ocda_peg_ssp.org
@@ -41,6 +41,15 @@ module sisudoc.ocda.abstraction.ssp;
part when a property is added (a reader of the newer format still reads
the older one), the major part when one changes meaning or goes away.
+ 2.0 2026-09-18 source.digest changes meaning: it now covers every
+ markup file the document is made of, where it was the
+ sha256 of the master file alone. For a .sst that was
+ right; for a .ssm it was close to meaningless
+ (live-manual's digest was 2,143 bytes of master while
+ the document itself is in 19 inserts, and) any chapter
+ could be edited without the digest moving. Being a
+ change of meaning rather than an addition, it takes the
+ major part with it.
1.1 2026-09-10 @source block; the document metadata the output writers
read carried in full (14 meta and 3 make properties
added); .segment_lv4_is, the segment a cross
diff --git a/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org b/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
index e4eedf5..e57de3c 100644
--- a/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
+++ b/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
@@ -149,14 +149,14 @@ git+file:///home/ralph/grotto/repo/git.repo/projects/doc-reform/packages/nix-fla
#+NAME: ocda_ssp_format_version_part_major
#+BEGIN_SRC org
-1
+2
#+END_SRC
*** minor
#+NAME: ocda_ssp_format_version_part_minor
#+BEGIN_SRC org
-1
+0
#+END_SRC
* sqlite collection search db schema version