aboutsummaryrefslogtreecommitdiffhomepage
path: root/org/out_src_abstraction_ocda_peg_ssp.org
Commit message (Collapse)AuthorAgeFilesLines
* org renamesRalph Amissah2026-09-121-736/+0
|
* ocda: heading cross reference linkingRalph Amissah2026-09-121-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The segment a cross reference to a heading lands on, gixed upstream in ocda, where the field is created, rather than inferred by the loader. A heading above level 4 opens no html segment of its own, so a link to it has to land on the level 4 heading that follows. The build loop cannot know that when it reads the heading, and says so in a comment of its own: "for html segname need following lv4 not yet known". It back-fills tag_assoc when the level 4 heading arrives (lv0to3_tags), and the answer then lives only in that map. tags.segment_lv4_is now holds it, resolved in a pass over the finished head and body sections: walking backwards, the last level 4 heading seen is the next one for everything above it, which is the same answer the back-fill gives. Emitted in the .ssp only where it differs from .segment_html_is, which is every heading at level 4 or below, so it is sparse: 317 lines over the 35 document reference set, none removed. Carried in the database as a column of its own, and read back by both readers. docHasFromAbstraction reads it instead of working it out. With that, and on top of the two defect fixes, a document loaded from an artefact and one parsed from markup agree: key sets identical, all 35 documents values 3 entries differ of some 30,000, all of them _the_title, where the abstraction supplies an epub segment the parser leaves unset link targets all 1,651 agree, against 89 differing before any of this work and 1 after the defect fixes alone Format stays v1.1. That version is new in this same run of work and nothing outside spine has read it, so this belongs in it rather than in a bump of its own. (assisted by Claude-Code)
* ocda: the abstraction carries the document metadata output readsRalph Amissah2026-09-121-1/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | .ssp format v0.1 -> v1.1, and the .ocda.db with it: they are two serialisations of one format and now move together. The gap this closes, measured against the writers' read set: of the 59 conf_make_meta fields outputs/io_out/ reads, 29 were in neither artefact. Five are conf.* and stay out, being site and run scoped (urls, papersize, the search db filename): the same abstraction published to two sites must take each site's. Of the rest, three are never assigned anywhere (title_short, publisher, original_publisher; read only by sqlite.d, always empty) and one is a copy of a field already carried (title_sub = title_subtitle), so 17 properties actually had to travel and now do: make breaks, footer, home_button_text meta title.edition, date.added_to_site, language.document_char, original.{title,source,language,language_char}, rights.copyright_{text,translation,illustrations, photographs,cover,audio,video} make splits by when it acts, which is worth keeping in mind: these three are read in outputs/io_out/ and must travel, while italics, bold, emphasis, substitute and headings are read at parse time and their effect is already in the objects. New @source block, and source.* rows in the database, so a reader can say which markup an abstraction came from rather than working from stale content in silence: language the document's own languages the pod's list, which is what the inter-language links in html need and neither artefact carried digest sha256 of the .sst; equals its digests.txt entry The database adds source.ssp_digest, the sha256 of the .ssp it was built from, since a file cannot hold its own hash. So the chain .sst -> .ssp -> .ocda.db is checkable end to end. The database's metadata table is now filled from the header blocks the .ssp gives back, not from a second list read off doc_matters. One list, in ssp.d: a property added there arrives in the database with nothing else changed, and one that is not in the .ssp cannot be in the database at all. That was the last place the two could drift; the objects stopped being able to on 2026-09-07. Version is checked on load. The major part must match, a newer minor is accepted (a minor bump only adds properties, and an unknown property line is ignored). A v0.1 artefact is now refused with a message saying to regenerate it, rather than loading half populated. test-abstraction-db.sh now compares the two artefacts' header blocks property by property, 36 per document on the wealth of networks, where it previously only checked that a schema.version row existed. Verified non-vacuous: dropping one property is reported. Reference regenerated, and the whole diff is this change and nothing else: 320 lines added, 35 removed over 35 files, being 35 format lines changed, 35 @source blocks (4 lines each), 35 language.document_char, 35 home_button_text (it has a default), 18 breaks, 16 footer, 4 date.added_to_site, 1 title.edition, 1 original.source. Existing outputs unaffected. (assisted by Claude-Code)
* ocda: .ssp should have all sections of abstractionRalph Amissah2026-09-121-2/+4
| | | | | | | The abstraction has nine sections and includes "tail". Both artefact writers hardcoded a list of eight, so "tail" was silently dropped. (assisted by Claude-Code)
* ssp: abstraction directory cleared onceRalph Amissah2026-09-091-13/+33
| | | | | | | | | | | | | | | | | abstraction directory cleared once, by whichever language is first The .ssp writer clears stale files out of pod/<doc>/media/abstraction/, and that one directory is shared by every language of a document. The clearing is now done once per directory per run, by whichever language reaches it first, with the lock held across it. A language that finds the directory already prepared has passed through that same lock before writing, so the clearing it skipped had completed before its own write began: no .ssp produced on a given run can be removed during it. (removes possibility of a race condition on parallelisation) (assisted by Claude-Code)
* output: the abstraction artefacts live with the podRalph Amissah2026-09-091-8/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | pod/ holds all document source representations: pod/<doc>/ source tree pod/<doc>/media/abstraction/<uid>.ssp abstraction, as text pod/<doc>.zip tree, zipped, .ssp included pod/<doc>.digests.txt sha256s of what is in them pod/<uid>.ocda.db abstraction, as sqlite db <lang>/abstraction/ is gone. pod/<doc>/media/abstraction/<uid>.ssp preferred as having the images (found within the pod tree) which .ssp needs to reproduce a document but does not carry on its own. The .ocda.db sits carries the images as well and (like the pod.zip) can be used to reproduce a document directly. digests.txt now covers the database as well as the zip, the source and the .ssp; (as does the metadata html page). Two ordering issues addressed: - the pod builder clean-slates pod/<doc>/ before regenerating it, and the .ssp is now written before that runs. The clean slate now leaves media/abstraction/ alone, and the .ssp writer clears that directory itself on the first language of a run, so a .ssp for a language the document no longer is removed and cannot be bundled. - for a multi-language document the .ssp files accumulate one language at a time and are bundled on the last, which is why the directory cannot simply be emptied by whichever (language) gets there first. (assisted by Claude-Code)
* ocda db: built from the .ssp, (tethered)Ralph Amissah2026-09-091-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ocda database is now built from the .ssp itself: the writer's lines are emitted, read straight back by ssp_in, and the objects that come out populate the database. Anything the .ssp does not carry, the database will not have either, by construction (rather than by test). [instead of as previously through a second walk over the in-memory abstraction] - spineAbstractionTxt is split: sspDocumentLines(doc) returns the whole .ssp as lines, and the file writer emits them. Output neutral, the reference test confirms. - spineAbstractionDb takes the abstraction as an argument rather than taking doc.abstraction. - sspRoundTripAbstraction(doc) in ssp_in is the join: lines out, lines in, abstraction returned. Both call sites in spine.d use it. - the header blocks and the image blobs still come from doc_matters (as: the .ssp does not carry image bytes). All (35) markup sample sourced databases built through the .ssp have byte identical SQL dumps to the one built directly before the change. That comparison also found one reader inaccuracy, which the .ssp round trip could not see because the writer omits the field either way: an absent identifier was restored as the ocn in every case, but for an object with no ocn it was empty ("a"~N identifiers are always written). Fixed; the two artefacts checking each other is what caught it. (assisted by Claude-Code)
* ocda: clean heading text used for navigationRalph Amissah2026-09-091-1/+48
| | | | | | heading text used for navigation is normalised, and | escaped (assisted by Claude-Code)
* ssp: definition of an object record, for readersRalph Amissah2026-09-091-301/+310
| | | | | | | | | | | | | | | | | | | one definition of an object record, so a reader can be held to it. Pure refactor: the per-object emission moves out of the write loop into template sspObjectRecord, taking the object and its section and returning the lines. The writer mixes it in and calls it. The point is the reader that follows. With one definition of what an object record looks like, a round trip can be checked against the writer itself rather than against a second, hand-kept description of the format, which is how the .ssp and the abstraction db drifted apart in the first place. (emitted text is byte identical) (assisted by Claude-Code)
* .ssp: three additional fields the writers readRalph Amissah2026-09-091-1/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With these the .ssp carries every field the output writers read. The two that were missing were both read by the sqlite search writer, and one of them is needed by anything that wants to know where an object sits in a segmented output: - .segment_html_is / .segment_epub_is (tags.html_segment_anchor_tag_is, epub_segment_anchor_tag_is), the segment an object *belongs to*, as against .segment (in_segment_html), which is the segment a heading *opens* and is set on headings only: 32 of 36 headings and 0 of 109 paragraphs in the autonomous contract. Without them a reader cannot tell which segment a paragraph is in. - .identifier for non-heading objects, the "a"~N series given to objects whose number is suppressed. A heading already carried it on its declaration line; every other object now carries it as a property, on the same condition, that it differs from the ocn. The same two segment fields are added to the abstraction db (segment_html_is, segment_epub_is) and to the cross-check test, so the two serialisations stay level. .ssp reference regenerated output run (--text --latex --epub --html --curate) byte identical to before the change, as it must be since no output writer reads either artefact. (assisted by Claude-Code)
* ocda: remove dead fieldsRalph Amissah2026-09-091-9/+9
| | | | | | | remove four dead fields, two of them serialised, none of the them were assigned (or called for reading). (assisted by Claude-Code)
* ocda a fix & ssp code-style use curly bracesRalph Amissah2026-09-031-71/+146
| | | | | | | | ocda: fixed inverted object_number_off on body paragraphs ssp: make use of curly braces (code-style) (assisted by Claude-Code)
* file rename: out_src_abstraction_ocda_peg_ssp.orgRalph Amissah2026-08-301-0/+442