Smart quotation for other languages

[hirnsalat]

Languages generally use different forms of quotation marks, Wikipedia has an overview.

The style of smart quotation in djot is specific to English. This, for example, makes German text with quotations look weird. Correct would be „Guten Morgen“, but djot outputs “Guten Morgen“.

Djot generally seems to avoid language-specific features, so this stands out. Personally, I would prefer having English-style smart quotations as a filter or optional feature, rather than part of the core syntax. Different styles could then be implemented in a similar fashion. What do you think?

[bpj]

Writing a filter using the quote AST elements is rather trivial: just filter on the double-quoted/single-quoted tags and replace them with a span including the literal quote marks you want. Without those AST elements it would be a lot harder. I have written two Pandoc filters which do this. Both let you specify which quotes to use in Pandoc metadata, which unfortunately doesn't work yet in djot. The new one also lets you specify "local" quotes as attributes on divs, to be applied only within that div. It also lets you specify characters to replace the " and ' characters with in string elements, corresponding to the \" and \' escapes in the source. This might be useful for continuation quotes or third-level quotes as used in some languages, and for "literal" apostrophes.

[hirnsalat]

Thanks for your comment, I didn't know that you can do that. Guess I should look into filters :)

[jgm]

Are you wanting to have a system where "Guten Morgen" turns into „Guten Morgen“ for German, or a system where „Guten Morgen“ is parsed as a quoted element in the AST? The latter seems useless, because you're already typing the curly quotes. It surprises me that people have sometimes wanted the former -- I would have thought that it would be odd to use " when writing German. But apparently it is easier on standard keyboards. Is that the motivation?

[hirnsalat]

I'm talking about the former, where "Guten Morgen" turns into „Guten Morgen“.

As for the motivation, you are correct. Standard German keyboard layouts only include the " character, and not „“. I don't even know how to directly type „“ on a standard keyboard. Almost always, one will type "Guten Morgen". Most writing software would then autocorrect it to „Guten Morgen“ when set to German, just like it would autocorrect to “Guten Morgen” when set to English. Basically, as long as the document language is set correctly, you can use " for everything and it works. In messengers and similar software, where it won't be automatically replaced, it's normal to just use straight quotes (") for everything.

As a result, at least to me, automatically substituting straight/curly quotes feels language-specific in either case. More intuitive would be to treat quotes the same as symbols – parse opening/closing quotes into the AST, but output the straight quotes literally by default. Different filters can then substitute the correct symbols for the context. (This is not a suggestion to change the default behavior, just an illustration of how it feels to me as a native German speaker.)

Quotation marks are weird, I hope this can provide some insight. When writing the original post, I wasn't aware just how easy it is to change the output with filters. That is definitely the place to handle language-specific typography details.

[jgm]

djot's reference parser does create a double_quoted node in the AST, rather than directly converting to curly quotes; the rendering of curly quotes is done in the renderer.

I wasn't sure whether you were using djot.js's CLI tool, or something else such as pandoc. In either case, filters could be used.

It could be worth adding a CLI option to specify quote style.

[adiabatic]

Are you wanting to have a system where "Guten Morgen" turns into „Guten Morgen“ for German

For what it's worth, Typst does exactly that, and it’s made by at least one German guy:

#set text(lang: "de")
"Das ist in Anführungszeichen."

// shows up as „Das ist Anführungszeichen.“

[Omikhleia]

For what is worth too, in pure Djot syntax, my SILE renderer does exactly that too, via a lang attribute (on inline or block-level elements)¹

He said: ["Das ist in Anführungszeichen."]{lang=de}

{lang=de}
"Das ist in Anführungszeichen."

Both show up, in the generated PDF, as „Das ist Anführungszeichen.“ (+ also switching to the target language for hyphenation and other language-specific typography rules, etc.)

(As @jgm actually wrote) it merely consists in the renderer implementing an appropriate rule on the AST nodes for double/single quoted items.
For HTML, one would have several options

• a variant-implementation of the default HTML renderer provided with Djot...
• ... or a simple filter, to the same effect, for intercepting these nodes and transforming them.

There are various ways for the transformation to operated. E.g. either one keeps track of the current scope to generate the correct quotes directly into the (then standalone) HTML, or one just generates special-class empty spans instead of the default English quotes, and handles the replacement and scoping via CSS. I prefer the latter option (so that adding more languages is a matter of tuning the CSS, rather than extending the filter and reconverting the document), but of course it needs a CSS file to also be used for proper rendering.

Footnotes

1. For dynamically switching language in the course of a document. How to set the default (top-level) language is another topic, with various possible approaches too. ↩