Skip to content

1st crack at re-writing rationale section of TCW31 (update p5subset in Styleseets) #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sydb wants to merge 2 commits into
base: master
Choose a base branch
from
Open

1st crack at re-writing rationale section of TCW31 (update p5subset in Styleseets) #28

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e77a679
First crack at re-writing entire rationale section
sydb Dec 6, 2025
9b358f7
Tweaks per @ebeshero
sydb Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 112 additions & 21 deletions TCW/tcw31.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,11 @@
</sourceDesc>
</fileDesc>
<revisionDesc>
<change when="2025-12-03" who="#POC">Update procedure with the improved <code>/Test</code> process which moves
comparisons to the end of Makefile, replaced <code>DIFFNOW=0</code> with <code>DIFFEND=1</code>.</change>
<change when="2025-12-03" who="#POC">Update procedure with the
improved process for <code>Test/</code>, which moves comparisons
to the end of Makefile. It replaces <code>DIFFNOW=0</code>
(which deferred comparisons completely, making the user do them
manually) with <code>DIFFEND=1</code>.</change>
<change when="2025-06-10" who="#POC">Revise and update procedure to keep it in line with
current practices as outlined in the Council shared document</change>
</revisionDesc>
Expand All @@ -35,26 +38,114 @@
<body>
<div>
<head>TEI Council: Update P5 subset</head>
<head>Rationale</head>
<p>The Stylesheets tests can succeed only against either the release branch of P5, or the
dev branch of P5, but often not against both simultaneously. For instance, the Stylesheets
dev branch sometimes successfully builds against P5 release, but fails against P5 dev
branch. On release day, we need the Stylesheets release branch to build against P5 release
branch successfully i.e. we need the tests to work with the release branch against the
release branch without producing any errors.</p>
<p>Changes on either side (Stylesheets or Guidelines) may break either build. The
Stylesheets test procedure in particular is quite fragile, and if it fails, the build
fails. For instance, since the contents of <gi>desc</gi> elements are copied over into
schemas, even a small change (like adding a comma) to an element description in the
Guidelines might break the Stylesheets build. </p>
<p>Since the Stylesheets are built against the Guidelines, there needs to be a (more or
less) up-to-date copy of the Guidelines in the Stylesheet repository. This copy of the
Guidelines is a file called “p5subset.xml” and it is stored in the <code>source/</code>
directory. Building the Guidelines in the TEI repository does <hi rend="italics">not</hi>
put a new copy of p5subset.xml into the <code>Stylesheets/source/</code> directory. In
order to keep the version of the p5subset.xml in the Stylesheets repository up to date,
that file is updated monthly by Council members. </p>
<div>
<head>Rationale</head>
<p>Many of the Stylesheets’ tests generate schemas from the
TEI Guidelines (using customization ODD files). This means
that the test process needs to have access to a copy of
P5. This in itself is not a particularly big problem — after
all, there is always a copy available at <ptr
target="https://www.tei-c.org/Vault/P5/current/xml/tei/odd/p5subset.xml"/>.
But the way many of these tests work is to compare the
schema generated by the test process with the expected
schema: if they are the same, the test succeeds, if there
is a difference, the test fails.</p>
<p>On the face of it, this seems a sensible way to test. If
you have changed the Stylesheets, you want to make sure your
changes have not altered the generated schemas, at least not
in a manner you did not expect. But this means there is an
brittle dependency between the Guidelines and the
Stylesheets’ test process, which itself is quite
fragile.</p>
<div>
<head>Dependency</head>
<p>To understand the dependency problem, imagine that
shortly after the release of P5 version 629.514.13 we drop
the <gi>datatype</gi> element from the ODD language: the
possible value(s) of an attribute are expressed with a
<gi>dataRef</gi> rather than a <gi>dataRef</gi> inside a
<gi>datatype</gi>. (This means that <gi>dataRef</gi> would
need to become a member of <name
type="class">att.repeatable</name>.) Once we change the
Guidelines so that <gi>datatype</gi> is not an element, we
need to actually remove <gi>datatype</gi> from the
Guidelines, so that they remain valid.</p>
<p>After these changes, we change the Stylesheets so that
they know how to handle a <gi>dataRef</gi>, possibly with
<att>minOccurs</att> or <att>maxOccurs</att>, that is a
direct child of <gi>attDef</gi>. If we try to test the
Stylesheets against P5 v. 629.514.13 (or any previous
version) there will definitionally be no cases of a
<gi>dataRef</gi> without <gi>datatype</gi> in the base
ODD, because in that version of the Guidelines a
<gi>dataRef</gi> was <emph>required</emph> to be inside a
<gi>datatype</gi>. So (as long as we did not remove the
code that handles a <gi>dataRef</gi> inside a
<gi>datatype</gi>) the Stylesheets tests may not fail, but
neither will they exercise our new code, that which we
most need to test.</p>
<p>So we need to tell the Stylesheets to use the new
version of the Guidelines (v. 629.515.0α) as the base
ODD. The way we do this is just to put a copy of the new
version of the Guidelines into the Stylesheets repository.</p>
</div>
<div>
<head>Fragility</head>
<p>Because the test procedure involves comparing two
schemas (the actual output generated by the test, and the
output that is expected from the test) using a string
comparison tool, even very minor changes to the
Guidelines may cause a test to fail. In fact, because the
glosses and descriptions of XML constructs are included in
the output schemas (so that tools like oXygen can make use
of them), there are revisions that do not change the actual
XML defined by the schema, but nonetheless will cause a
test to fail.</p>
<p><label>Example 1</label> If the description of the
<gi>constraint</gi> element were corrected from <quote>the
formal rules of a constraint</quote> to <quote>contains
the formal rules of a constraint.</quote>, every test that
involves creating a schema that contains the
<gi>constraint</gi> element would fail.</p>
<p><label>Example 2</label> If the content model of the
<gi>foo</gi> element were simplified from
<egXML xmlns="http://www.tei-c.org/ns/Examples">
<sequence>
<elementRef key="bar"/>
<elementRef key="bar"/>
<elementRef key="bar" minOccurs="0" maxOccurs="unbounded"/>
</sequence>
</egXML>
to just
<egXML xmlns="http://www.tei-c.org/ns/Examples">
<elementRef key="bar" minOccurs="2" maxOccurs="unbounded"/>
</egXML>
the same set of documents (those that have two or more
<gi>bar</gi> elements as children of each <gi>foo</gi>)
would be valid against the new schema. But since the
actual grammar is different — it now has <code>bar,
bar+</code> where it used to have <code>bar, bar,
bar*</code>, the test would fail.</p>
</div>
<div>
<head>Solution</head>
<p>Since the Stylesheets are tested against a specific
version of the Guidelines, it is very helpful to have a
(more or less) up-to-date copy of the Guidelines in the
Stylesheet repository. If nothing else, this minimizes the
possibility of testing with the wrong version of P5.</p>
<p>Thus we store a copy of the Guidelines in a file called
<name type="file">p5subset.xml</name> in the
<code>source/</code> directory. Just building the Guidelines
in the TEI repository does <hi rend="italics">not</hi> put a
new copy of p5subset.xml into the
<code>Stylesheets/source/</code> directory. The file needs
to be manually copied over, hence these instructions for
Copy link
Member

@ebeshero ebeshero Dec 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(At least that is the state of things right now. It really seems strange that we don’t have an automatic process that either moves the built file to Stylesheets or just looks for the built P5 in place. The more I think of it, the more sensible it seems to find a way to automate this with no need for manual copying of files.)

That is just a comment looking forward to yet another revision of this documentation, not suggesting any change here—unless some brief mention of discussions to change all this for the better.

doing so. This task is typically performed at least monthly
by a Council member.</p>
</div>
</div>
<div>
<head>Step-by-Step Instructions:</head>
<list rend="numbered">
<item n="1">Update your local copies of both the TEI and Stylesheets
Expand Down