Coalesced words - tokenization

[drahnr]

I've attempted to deal abbreviated forms of type we've and I'd and it's as part of drahnr/cargo-spellcheck#186 which is a mere workaround.

Probably out of scope of nlprule, yet a pitfal for real life usage.

Since nlprule is going to support spellchecking as well it might be worth discussing / keeping in mind.

[bminixhofer]

Good point. This should already be handled by the spellchecking PR, but I'm not exactly sure how to be honest :) I'll make sure to check it when I revisit spellchecking.

Some rarer words which are tokenized as multiple tokens like Côte d'Azur, L'Oréal and cc'ing have a special spelling.txt file, but I'm not so sure right now how common contractions are handled.