<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sisudoc-spine/src/sisudoc/ocda/abstraction/doc_has.d, branch main</title>
<subtitle>SiSU Spine: document publishing and search (in D) 2015</subtitle>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/'/>
<entry>
<title>org files mostly headers</title>
<updated>2026-10-25T00:45:37+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-10-24T22:41:41+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=2a55a61d10739e782a82fcb438607e552148fbd9'/>
<id>2a55a61d10739e782a82fcb438607e552148fbd9</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>abstraction 2.1: a poem states its object range</title>
<updated>2026-10-01T03:14:23+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-30T20:18:59+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=993a3f655359f98dc26d3d3222e07792e2ccddf7'/>
<id>993a3f655359f98dc26d3d3222e07792e2ccddf7</id>
<content type='text'>
- A poem is a container of verse, and stores its range of verse ocn,
  - its verse are the citable units with ocn.

- A note in the last verse of a poem previously was not gathered into
  the endnotes section, this now is fixed

Format 2.0 -&gt; 2.1: the property is an addition, the poem is not a
citable object (but contans a range of objects), its verse are
(individual citable objects), and every reader checks the major version.
A 2.0 database reads as having no ranges.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- A poem is a container of verse, and stores its range of verse ocn,
  - its verse are the citable units with ocn.

- A note in the last verse of a poem previously was not gathered into
  the endnotes section, this now is fixed

Format 2.0 -&gt; 2.1: the property is an addition, the poem is not a
citable object (but contans a range of objects), its verse are
(individual citable objects), and every reader checks the major version.
A 2.0 database reads as having no ranges.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>epub and html: no errors left over the sample set</title>
<updated>2026-09-14T16:18:51+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-14T00:53:26+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=6835275408e1629f450d163e0cc5221c977e87d2'/>
<id>6835275408e1629f450d163e0cc5221c977e87d2</id>
<content type='text'>
epubcheck 5.3.0 over the 35 sample epubs went from 1302 remaining errors
to none. The six causes for those remaining errors fixed here.

most of the count was one of them:

- Tables (1123 errors, 86%) where the writer emitted obsolete
  attributes, removed in 2014; table also now in div instead of a
  paragraph. also fixed:

- Navigation (115). Every level 4 heading's nav entry ended "#0", the
  ocn of an object that has no ocn and so no id.

- Duplicate ids (37). (from three unrelated sources).

- The publication identifier (27 warnings). dc:identifier was a
  hardcoded hex string, not a UUID, and the same one in all 35 files,
  so two documents in one library collided. It is now UUID v5 over the
  document's own uid: a real RFC 4122 identifier, stable across builds
  and distinct per document.

- Image manifest entries (15). media-type was "image/" plus the file
  extension, giving "image/jpg", which is not a media type; epubcheck
  reads it as a foreign resource and wants a fallback. There is now a
  mapping, which also covers svg and webp. Item ids came from the file
  basename, and "2bits_02_01-100.png" gave an id starting with a digit,
  which is not an XML name; ids are now prefixed and sanitised.

- An ocn resolving to no segment (9). A poem block and its first verse
  share one ocn; the verses are written into a segment and the block
  itself is not.

Also dropped from the package document: an xmlns:xsi that nothing
used, and a prefix declaration for the rendition vocabulary, which is
a reserved prefix and needed no declaring. The package now carries
xml:lang.

test-epub-validity.sh now allows no fatal and no error. Two NAV-011
warnings remain and are described in it. Three reference .ssp files
change, all of them the anchor tag disambiguation above.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
epubcheck 5.3.0 over the 35 sample epubs went from 1302 remaining errors
to none. The six causes for those remaining errors fixed here.

most of the count was one of them:

- Tables (1123 errors, 86%) where the writer emitted obsolete
  attributes, removed in 2014; table also now in div instead of a
  paragraph. also fixed:

- Navigation (115). Every level 4 heading's nav entry ended "#0", the
  ocn of an object that has no ocn and so no id.

- Duplicate ids (37). (from three unrelated sources).

- The publication identifier (27 warnings). dc:identifier was a
  hardcoded hex string, not a UUID, and the same one in all 35 files,
  so two documents in one library collided. It is now UUID v5 over the
  document's own uid: a real RFC 4122 identifier, stable across builds
  and distinct per document.

- Image manifest entries (15). media-type was "image/" plus the file
  extension, giving "image/jpg", which is not a media type; epubcheck
  reads it as a foreign resource and wants a fallback. There is now a
  mapping, which also covers svg and webp. Item ids came from the file
  basename, and "2bits_02_01-100.png" gave an id starting with a digit,
  which is not an XML name; ids are now prefixed and sanitised.

- An ocn resolving to no segment (9). A poem block and its first verse
  share one ocn; the verses are written into a segment and the block
  itself is not.

Also dropped from the package document: an xmlns:xsi that nothing
used, and a prefix declaration for the rendition vocabulary, which is
a reserved prefix and needed no declaring. The package now carries
xml:lang.

test-epub-validity.sh now allows no fatal and no error. Two NAV-011
warnings remain and are described in it. Three reference .ssp files
change, all of them the anchor tag disambiguation above.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>spine: process a document from .ocda.db or .ssp</title>
<updated>2026-09-12T16:14:39+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-11T17:57:47+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=f41bf1cc525c70cced764a5e4df70d499d82f9c9'/>
<id>f41bf1cc525c70cced764a5e4df70d499d82f9c9</id>
<content type='text'>
Process a document from its .ocda.db or .ssp abstraction. A .ssp or a
.ocda.db given as an argument is now a document like any other: read
rather than parsed, and handed to the output writers as the same doc
they already take.

The 35 document sample collection built from its .ssp files is byte
identical to the same collection built from markup: 1457 files, no
differences.

Three things had to be rebuilt rather than read, all of them a value
parsed rather than a structure derived:

- classify_topic_register_arr and its expanded twin. The split is not a
  plain one, so the rule moved to sisudoc.ocda.meta.topic_register and
  both the yaml reader and this one call it.
- creator_author_arr, which is creator.author split on the ", " it was
  joined with.
- title_sub, a copy of title_subtitle made where the header is read, and
  what epub3 puts in dc:title id="subtitle".

Fixed ordering bug found by the acceptance test. A heading's own anchor
can be a bare number taken from its text and this can collide with the
ocn of an unrelated object. Whole output comparison found that before
the fix there was one wrong link in one epub's table of contents.

From a .ocda.db, 1446 of 1457 files are identical. The 11 that are not
are the images: a database carries its own image blobs and nothing yet
extracts them, so the five sisu_markup images are not copied and the
epub that embeds them differs. That is the next step and is not a defect
in this one.

--source and --pod2 are refused with a warning rather than half
done, no artefact carrying the markup.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Process a document from its .ocda.db or .ssp abstraction. A .ssp or a
.ocda.db given as an argument is now a document like any other: read
rather than parsed, and handed to the output writers as the same doc
they already take.

The 35 document sample collection built from its .ssp files is byte
identical to the same collection built from markup: 1457 files, no
differences.

Three things had to be rebuilt rather than read, all of them a value
parsed rather than a structure derived:

- classify_topic_register_arr and its expanded twin. The split is not a
  plain one, so the rule moved to sisudoc.ocda.meta.topic_register and
  both the yaml reader and this one call it.
- creator_author_arr, which is creator.author split on the ", " it was
  joined with.
- title_sub, a copy of title_subtitle made where the header is read, and
  what epub3 puts in dc:title id="subtitle".

Fixed ordering bug found by the acceptance test. A heading's own anchor
can be a bare number taken from its text and this can collide with the
ocn of an unrelated object. Whole output comparison found that before
the fix there was one wrong link in one epub's table of contents.

From a .ocda.db, 1446 of 1457 files are identical. The 11 that are not
are the images: a database carries its own image blobs and nothing yet
extracts them, so the five sisu_markup images are not copied and the
epub that embeds them differs. That is the next step and is not a defect
in this one.

--source and --pod2 are refused with a warning rather than half
done, no artefact carrying the markup.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>ocda: heading cross reference linking</title>
<updated>2026-09-12T16:14:39+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-11T16:32:13+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=4e6774904ce811b074d8b3694b04dc5c1e8219e4'/>
<id>4e6774904ce811b074d8b3694b04dc5c1e8219e4</id>
<content type='text'>
The segment a cross reference to a heading lands on, gixed upstream in
ocda, where the field is created, rather than inferred by the loader.

A heading above level 4 opens no html segment of its own, so a link to
it has to land on the level 4 heading that follows. The build loop
cannot know that when it reads the heading, and says so in a comment of
its own: "for html segname need following lv4 not yet known". It
back-fills tag_assoc when the level 4 heading arrives (lv0to3_tags), and
the answer then lives only in that map.

tags.segment_lv4_is now holds it, resolved in a pass over the finished
head and body sections: walking backwards, the last level 4 heading seen
is the next one for everything above it, which is the same answer the
back-fill gives. Emitted in the .ssp only where it differs from
.segment_html_is, which is every heading at level 4 or below, so it is
sparse: 317 lines over the 35 document reference set, none removed.
Carried in the database as a column of its own, and read back by both
readers.

docHasFromAbstraction reads it instead of working it out. With that, and
on top of the two defect fixes, a document loaded from an artefact and
one parsed from markup agree:

  key sets       identical, all 35 documents
  values         3 entries differ of some 30,000, all of them
                 _the_title, where the abstraction supplies an epub
                 segment the parser leaves unset
  link targets   all 1,651 agree, against 89 differing before any
                 of this work and 1 after the defect fixes alone

Format stays v1.1. That version is new in this same run of work and
nothing outside spine has read it, so this belongs in it rather than in
a bump of its own.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The segment a cross reference to a heading lands on, gixed upstream in
ocda, where the field is created, rather than inferred by the loader.

A heading above level 4 opens no html segment of its own, so a link to
it has to land on the level 4 heading that follows. The build loop
cannot know that when it reads the heading, and says so in a comment of
its own: "for html segname need following lv4 not yet known". It
back-fills tag_assoc when the level 4 heading arrives (lv0to3_tags), and
the answer then lives only in that map.

tags.segment_lv4_is now holds it, resolved in a pass over the finished
head and body sections: walking backwards, the last level 4 heading seen
is the next one for everything above it, which is the same answer the
back-fill gives. Emitted in the .ssp only where it differs from
.segment_html_is, which is every heading at level 4 or below, so it is
sparse: 317 lines over the 35 document reference set, none removed.
Carried in the database as a column of its own, and read back by both
readers.

docHasFromAbstraction reads it instead of working it out. With that, and
on top of the two defect fixes, a document loaded from an artefact and
one parsed from markup agree:

  key sets       identical, all 35 documents
  values         3 entries differ of some 30,000, all of them
                 _the_title, where the abstraction supplies an epub
                 segment the parser leaves unset
  link targets   all 1,651 agree, against 89 differing before any
                 of this work and 1 after the defect fixes alone

Format stays v1.1. That version is new in this same run of work and
nothing outside spine has read it, so this belongs in it rather than in
a bump of its own.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>ocda: ST_DocHas from a loaded abstraction</title>
<updated>2026-09-12T16:14:39+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-11T16:02:23+00:00</published>
<link rel='alternate' type='text/html' href='https://doc-reform.org/projects/sisudoc-spine/commit/?id=3e23d7082e20d28971c7785db3c5e9dd724167e6'/>
<id>3e23d7082e20d28971c7785db3c5e9dd724167e6</id>
<content type='text'>
docHasFromAbstraction() builds ST_DocHas from the objects and the
@doc_has block, so a document read from a .ssp or a .ocda.db can reach
the value the parser reaches. Everything it builds is an index over
properties the artefact already carries, not a recomputation of
something it does not: the counts come from the header, imagelist from
the .image records, the segment name lists and the tag associations from
.anchor, .segment, .segment_epub, .heading_lev_anchor and
.segment_*_is, and section_keys_sequenced from which sections are
non-empty plus the run's own flags.

*The two defect fixes this now sits on did most of the closing.*
Measured before them and after, over some 30,000 tag_associations
entries and the 1,651 keys a document actually links to:

  imagelist        8 documents differed, now none. The parser was
                   the one that was wrong, and rgx.image being
                   anchored to image markup brought it into line
                   with what the .image records always said.
  key sets         four keys differed (_part_eof, "0", the empty
                   key, and "toc" the other way about), now none.
                   _part_eof came back the moment @tail was
                   carried, and the rest with it.
  values           89 link targets resolved differently, now 1.

The one left is free_culture's ocn 5, and it is the case this cannot
reach: a heading above level 4 takes its segment by back-filling when
the next level 4 heading arrives, so the object never holds the answer
and reading the objects in order only approximates it. Fixed properly in
the commit that follows, in ocda where the field is made, rather than
guessed at here.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
docHasFromAbstraction() builds ST_DocHas from the objects and the
@doc_has block, so a document read from a .ssp or a .ocda.db can reach
the value the parser reaches. Everything it builds is an index over
properties the artefact already carries, not a recomputation of
something it does not: the counts come from the header, imagelist from
the .image records, the segment name lists and the tag associations from
.anchor, .segment, .segment_epub, .heading_lev_anchor and
.segment_*_is, and section_keys_sequenced from which sections are
non-empty plus the run's own flags.

*The two defect fixes this now sits on did most of the closing.*
Measured before them and after, over some 30,000 tag_associations
entries and the 1,651 keys a document actually links to:

  imagelist        8 documents differed, now none. The parser was
                   the one that was wrong, and rgx.image being
                   anchored to image markup brought it into line
                   with what the .image records always said.
  key sets         four keys differed (_part_eof, "0", the empty
                   key, and "toc" the other way about), now none.
                   _part_eof came back the moment @tail was
                   carried, and the rest with it.
  values           89 link targets resolved differently, now 1.

The one left is free_culture's ocn 5, and it is the case this cannot
reach: a heading above level 4 takes its segment by back-filling when
the next level 4 heading arrives, so the object never holds the answer
and reading the objects in order only approximates it. Fixed properly in
the commit that follows, in ocda where the field is made, rather than
guessed at here.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
</feed>
