Skip to content

Commit 41375e7

Browse files
tidousttidoust
committed
Add patch mechanism to add CSS syntaxes manually
This is meant to provide a simple patching mechanism for #1647 and #1737, based on a JSON-like structure, similar to the patching mechanism that we use for events (`amend-event-data.js`). Patches get applied during curation after dropping duplicate definitions, and before the CSS extracts get consolidated. The patching logic is initially restricted to adding syntaxes where they do not exist (no way to override an existing one for now) and to adding syntaxes to base definitions (no way to set the syntax of a `newValues` property). The logic allows to say: 1. Here is the syntax, period. That's the most direct way. But also the one that requires manual maintenance. 2. Compute the syntax from the list of values, and optionally complete that list with this additional syntax. This allows to compute syntaxes such as `<system-color>` indirectly but still automatically from the spec. 3. Use the syntax from that other construct (must be defined in the same spec). This allows to say that the syntax of `-webkit-user-select` is the same as that of `user-select`. It will probably be useful to extend the patching logic slightly to allow overriding existing syntaxes over time. This would allow us to replace most patches in `csspatches` with a more convenient mechanism. The initial list of patches is based on missing syntaxes identified in the mdn-webref analysis: https://github.com/tidoust/mdn-webref/blob/main/report-syntax.md#syntax-mismatches-between-mdn-data-and-webref ... completed with some of the types highlighted in #1647. At-rules and selectors are excluded for now. They can be patched too, just not done in that iteration. A few other ones are missing when it was not obvious to me that the syntax was correct, or when it seemed that the underlying spec could perhaps be updated. In other words, the initial list of patches should be a good start but is not meant to be exhaustive.
1 parent 2f1bd5c commit 41375e7

File tree

3 files changed

+399
-1
lines changed

3 files changed

+399
-1
lines changed

‎ed/csspatches/README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,8 @@
22

33
These are patches applied to the CSS extracts scraped from specs to produce the `@webref/css` package. These patches can break as specs are updated and thus need ongoing maintenance.
44

5-
For details on how to create and update patches, please see the [Web IDL patches documentation](../idlpatches/README.md)
5+
For details on how to create and update patches, please see the [Web IDL patches documentation](../idlpatches/README.md).
6+
7+
Note that Webref has two additional mechanisms to patch CSS data:
8+
- The [`tools/drop-css-property-duplicates.js`](../../tools/drop-css-property-duplicates.js) script drops duplicates for situations where definition in one spec is known to override the same definition in another spec.
9+
- The [`tools/amend-css-syntaxes.js`](../../tools/amend-css-syntaxes.js) script adds syntaxes of CSS constructs in situations where the spec details a syntax in non-machine-readable prose, e.g., because the syntax needs to be derived from a list of values.

‎tools/amend-css-syntaxes.js‎

Lines changed: 388 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,388 @@
1+
/**
2+
* Amend extracted CSS data to add/update syntaxes that could not be extracted
3+
* from the specifications, typically because the syntax is defined in non
4+
* machine-readable prose. To run the script:
5+
*
6+
* node tools/amend-css-syntaxes.js [folder]
7+
*
8+
* ... where:
9+
* - [folder] is the name of the folder that contains the data to parse and
10+
* update (default is "curated")
11+
*/
12+
13+
/******************************************************************************
14+
* List of CSS syntax patches to apply.
15+
*
16+
* The list is an indexed object where keys are the shortnames of the
17+
* specifications to patch and where values are themselves an indexed object
18+
* where:
19+
* - keys are the name of the CSS constructs to patch, for example:
20+
* `<system-color>`
21+
* - values define the patch to apply
22+
*
23+
* If the CSS construct to target is nested (for example, the descriptor of an
24+
* at-rule), start with the name of the root constructor, add a `/`, and then
25+
* the name of the actual construct. For example: `@page/page/<page-size>` to
26+
* target the `<page-size>` type of the `page` descriptor of the `@page`
27+
* at-rule.
28+
*
29+
* The patch value can either be:
30+
* - A string, which gets interpreted as the CSS syntax to set. For example:
31+
* '<custom-ident>'
32+
* - An object with one or more of the following keys:
33+
* - `useValues`: a boolean flag. When set, the code builds the syntax of the
34+
* CSS construct from the list of CSS values that are defined for it in the
35+
* spec (and that could be extracted).
36+
* - `newValues`: a string that contains some additional syntax to add on top
37+
* of existing values. The `useValues` flag must also be set (there is no
38+
* way to set the syntax of an extended definition for now)
39+
* - `sameAs`: a reference to another CSS construct in the same spec that
40+
* provides the syntax to use. Cannot be used with any of the other keys.
41+
*
42+
* Trailing spaces in syntaxes are trimmed, and so are extra spaces. That's
43+
* to ease patch readability. For example: `
44+
* <basic-shape-rect> |
45+
* <circle()> | <ellipse()>
46+
* `
47+
* ... becomes '<basic-shape-rect> | <circle()> | <ellipse()>'
48+
*
49+
* It is good practice to start a patch with a comment that links to the
50+
* the construct definition in the spec.
51+
*
52+
* Patches should only be used for syntaxes that cannot be correctly extracted
53+
* from specs.
54+
*
55+
* Note: the logic could be extended over time as needed to:
56+
* - Allow overriding an existing syntax. That would allow replacing most
57+
* patches in "csspatches". The underlying PR should be reported in a
58+
* `pending` key and/or the incorrect syntax should be recorded to make
59+
* sure we do not miss updates.
60+
* - Allow dropping an existing syntax. That has never been needed until now.
61+
* - Allow adding/overriding new values of an extended definition. That could
62+
* also prove useful.
63+
*****************************************************************************/
64+
const patches = {
65+
// https://drafts.csswg.org/css-cascade-6/#scope-syntax
66+
'css-cascade-6': {
67+
'<scope-start>': '<selector-list>',
68+
'<scope-end>': '<selector-list>'
69+
},
70+
71+
// https://drafts.csswg.org/css-color-4/#typedef-system-color
72+
// https://drafts.csswg.org/css-color-4/#typedef-deprecated-color
73+
// https://drafts.csswg.org/css-color-4/#typedef-named-color
74+
'css-color-4': {
75+
'<system-color>': {
76+
useValues: true,
77+
newValues: '<deprecated-color>'
78+
},
79+
'<deprecated-color>': {
80+
useValues: true
81+
},
82+
'<named-color>': {
83+
useValues: true,
84+
newValues: 'transparent'
85+
}
86+
},
87+
88+
// https://drafts.csswg.org/css-counter-styles-3/#typedef-counter-style-name
89+
'css-counter-styles-3': {
90+
'<counter-style-name>': '<custom-ident>'
91+
},
92+
93+
// https://drafts.csswg.org/css-fonts-4/#typedef-font-palette-palette-identifier
94+
'css-fonts-4': {
95+
'font-palette/<palette-identifier>': '<dashed-ident>'
96+
},
97+
98+
// https://drafts.csswg.org/css-lists-3/#typedef-counter-name
99+
'css-lists-3': {
100+
'<counter-name>': '<custom-ident>'
101+
},
102+
103+
// https://drafts.csswg.org/css-overflow-5/#typedef-scroll-button-direction
104+
'css-overflow-5': {
105+
'<scroll-button-direction>': {
106+
useValues: true
107+
}
108+
},
109+
110+
// https://drafts.csswg.org/css-page-3/#typedef-page-size-page-size
111+
'css-page-3': {
112+
'@page/size/<page-size>': {
113+
useValues: true
114+
}
115+
},
116+
117+
// https://drafts.csswg.org/css-shapes-1/#supported-basic-shapes
118+
'css-shapes-1': {
119+
'<basic-shape>': `
120+
<basic-shape-rect> |
121+
<circle()> | <ellipse()> | <polygon()> |
122+
<path()> | <shape()>
123+
`
124+
},
125+
126+
// https://drafts.csswg.org/css-syntax-3/#the-anb-type
127+
'css-syntax-3': {
128+
'<n-dimension>': '<dimension-token>',
129+
'<ndash-dimension>': '<dimension-token>',
130+
'<ndashdigit-dimension>': '<dimension-token>',
131+
'<ndashdigit-ident>': '<ident-token>',
132+
'<dashndashdigit-ident>': '<ident-token>',
133+
'<signed-integer>': '<number-token>',
134+
'<signless-integer>': '<number-token>'
135+
},
136+
137+
// https://drafts.csswg.org/css-transforms-2/#transform-functions
138+
'css-transforms-2': {
139+
'<transform-function>': `
140+
<scale()> | <scaleX()> | <scaleY()> | <scaleZ()> |
141+
<translate3d()> | <translate()> | <translateX()> | <translateY()> | <translateZ()> |
142+
<rotate3d()> <rotate()> | <rotateX()> | <rotateY()> | <rotateZ()> |
143+
<skew()> | <skewX()> | <skewY()> |
144+
<matrix3d()> | <matrix()> | <perspective()>
145+
`,
146+
},
147+
148+
// https://drafts.csswg.org/css-ui-4/#propdef--webkit-user-select
149+
// https://drafts.csswg.org/css-ui-4/#typedef-outline-line-style
150+
'css-ui-4': {
151+
'-webkit-user-select': {
152+
sameAs: 'user-select'
153+
},
154+
// Same as <line-style> but 'auto' replaces 'hidden'
155+
'<outline-line-style>': `
156+
none | auto | dotted | dashed | solid | double |
157+
groove | ridge | inset | outset
158+
`
159+
},
160+
161+
// https://drafts.csswg.org/css-values-4/#integers
162+
'css-values-4': {
163+
'<integer>': '<number-token>'
164+
},
165+
166+
// https://drafts.csswg.org/css2/#value-def-absolute-size
167+
// https://drafts.csswg.org/css2/#value-def-relative-size
168+
// https://drafts.csswg.org/css2/#value-def-shape
169+
'CSS2': {
170+
'<absolute-size>': `
171+
xx-small | x-small | small | medium |
172+
large | x-large | xx-large
173+
`,
174+
'<relative-size>': 'larger | smaller',
175+
'<shape>': 'rect(<top>, <right>, <bottom>, <left>)'
176+
},
177+
178+
// https://svgwg.org/svg2-draft/pservers.html#StopColorProperty
179+
// https://svgwg.org/svg2-draft/pservers.html#StopOpacityProperty
180+
'SVG2': {
181+
'stop-color': `<'color'>`,
182+
'stop-opacity': `<'opacity'>`
183+
}
184+
};
185+
186+
187+
/******************************************************************************
188+
* Patching logic
189+
*****************************************************************************/
190+
import fs from 'node:fs/promises';
191+
import path from 'node:path';
192+
import { fileURLToPath } from 'node:url';
193+
import { loadJSON } from './utils.js';
194+
import { expandCrawlResult, isLatestLevelThatPasses } from 'reffy';
195+
196+
/**
197+
* Trim a patch syntax
198+
*/
199+
function trimSyntax(syntax) {
200+
return syntax.trim().replace(/\s+\|\s+/g, ' | ');
201+
}
202+
203+
/**
204+
* Recursive function to look for a CSS construct within an extract.
205+
*
206+
* Levels are separated with '/'. When looking at a spec, pass `spec.css` as
207+
* second parameter. For example:
208+
* const construct = findCssConstruct('@page/page/<page-size>', spec.css);
209+
*/
210+
function findCssConstruct(key, currentRoot) {
211+
if (!key) {
212+
return currentRoot;
213+
}
214+
const keys = key.split('/');
215+
const types = ['properties', 'atrules', 'selectors', 'values', 'descriptors'];
216+
for (const key of keys) {
217+
for (const type of types) {
218+
for (const entry of (currentRoot[type] ?? [])) {
219+
if (entry.name === keys[0]) {
220+
return findCssConstruct(keys.slice(1).join('/'), entry);
221+
}
222+
}
223+
}
224+
}
225+
return null;
226+
}
227+
228+
/**
229+
* Apply patches that apply to the given spec.
230+
*
231+
* The function returns a list of errors, an empty array if all went fine.
232+
*/
233+
function applyCssSyntaxPatches(spec) {
234+
let errors = [];
235+
console.log(`- applying CSS syntax patches for ${spec.shortname}`);
236+
for (let [key, patch] of Object.entries(patches[spec.shortname])) {
237+
// Expand the patch as needed, and check it against current patching
238+
// constraints (note the logic could be extended over time to support more
239+
// scenarios if that proves needed).
240+
if (typeof patch === 'string') {
241+
patch = { value: patch };
242+
}
243+
if (!patch.value && !patch.useValues && !patch.newValues && !patch.sameAs) {
244+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} does not define any syntax`);
245+
continue;
246+
}
247+
if (patch.sameAs && (patch.value || patch.useValues || patch.newValues)) {
248+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} is invalid, cannot have "sameAs" and another syntax at the same time`);
249+
continue;
250+
}
251+
if (patch.value && patch.useValues) {
252+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} is invalid, cannot have "value" and "useValues" at the same time`);
253+
continue;
254+
}
255+
if (patch.value && patch.newValues) {
256+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} is invalid, cannot have "value" and "newValues" at the same time`);
257+
continue;
258+
}
259+
if (patch.newValues && !patch.useValues) {
260+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} is invalid, cannot have "newValues" without "useValues"`);
261+
continue;
262+
}
263+
264+
// Locate the underlying CSS construct in the spec extract
265+
const construct = findCssConstruct(key, spec.css);
266+
if (!construct) {
267+
errors.push(`Could not find a CSS construct with name ${key} in ${spec.shortname} for syntax patching`);
268+
continue;
269+
}
270+
271+
// Patching mechanism is for adding new syntaxes for now.
272+
// (It could be extended later on to override existing syntaxes)
273+
if (construct.value || construct.newValues) {
274+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} cannot be applied: construct already has a syntax`);
275+
continue;
276+
}
277+
278+
if (patch.useValues) {
279+
if (!construct.values || construct.values.length === 0) {
280+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} cannot be applied: "useValues" is set but no values are associated with the construct`);
281+
continue;
282+
}
283+
construct.value = construct.values.map(v => v.value).join(' | ');
284+
}
285+
else if (patch.sameAs) {
286+
const sameConstruct = findCssConstruct(patch.sameAs, spec.css);
287+
if (!sameConstruct) {
288+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} cannot be applied: could not find "sameAs" target ${patch.sameAs}`);
289+
continue;
290+
}
291+
if (!sameConstruct.value) {
292+
errors.push(`The CSS syntax patch for ${key} in ${spec.shortname} cannot be applied: "sameAs" target ${patch.sameAs} has no syntax`);
293+
continue;
294+
}
295+
}
296+
else {
297+
construct.value = trimSyntax(patch.value);
298+
}
299+
if (patch.newValues) {
300+
construct.value += ' | ' + trimSyntax(patch.newValues);
301+
}
302+
}
303+
if (!errors.length) {
304+
spec.needsSaving = true;
305+
}
306+
return errors;
307+
}
308+
309+
/**
310+
* Save the updated CSS extract of the spec
311+
*
312+
* Note: CSS extracts are named after the series shortname by default.
313+
*/
314+
async function saveCss(spec, folder, list) {
315+
let filename = spec.series.shortname;
316+
if ((spec.seriesComposition === 'delta') ||
317+
!isLatestLevelThatPasses(spec, list, spec => spec.css)) {
318+
filename = spec.shortname;
319+
}
320+
const pathname = path.join(folder, 'css', filename + '.json');
321+
const css = Object.assign({
322+
spec: {
323+
title: spec.title,
324+
url: spec.crawled
325+
}
326+
}, spec.css);
327+
const json = JSON.stringify(css, null, 2) + '\n';
328+
await fs.writeFile(pathname, json);
329+
};
330+
331+
332+
/**
333+
* Apply CSS syntax patch to the crawl results in the given folder.
334+
*
335+
* The function expects to find an `index.json` file in that folder that links
336+
* to individual CSS extracts.
337+
*
338+
* The function throws with a list of errors when patches could not all be
339+
* applied.
340+
*/
341+
async function amendCssSyntaxes(folder) {
342+
const rawIndex = await loadJSON(path.join(folder, 'index.json'));
343+
const index = JSON.parse(JSON.stringify(rawIndex));
344+
await expandCrawlResult(index, folder, ['css']);
345+
346+
let errors = [];
347+
for (const specShortname of Object.keys(patches)) {
348+
const spec = index.results.find(s => s.shortname === specShortname);
349+
if (!spec) {
350+
errors.push(`Could not find spec with shortname ${specShortname} for CSS syntax patching`);
351+
continue;
352+
}
353+
if (!spec.css) {
354+
errors.push(`Could not find any CSS in spec with shortname ${specShortname} for CSS syntax patching`);
355+
continue;
356+
}
357+
errors = errors.concat(applyCssSyntaxPatches(spec));
358+
}
359+
360+
for (const spec of index.results) {
361+
if (spec.needsSaving) {
362+
await saveCss(spec, folder, index.results);
363+
delete spec.needsSaving;
364+
}
365+
}
366+
367+
if (errors.length) {
368+
throw new Error("\n- " + errors.join("\n- "));
369+
}
370+
}
371+
372+
373+
/******************************************************************************
374+
* Export methods for use as module
375+
*****************************************************************************/
376+
export { amendCssSyntaxes };
377+
378+
/******************************************************************************
379+
* Code run if the code is run as a stand-alone module
380+
*****************************************************************************/
381+
if (process.argv[1] === fileURLToPath(import.meta.url)) {
382+
const folder = process.argv[2] ?? 'curated';
383+
384+
amendCssSyntaxes(folder).catch(e => {
385+
console.error(e);
386+
process.exit(1);
387+
});
388+
}

0 commit comments

Comments
 (0)