-
PythonParser.g4 is the ANTLR4 parser grammar that based on the last "traditional" Python grammar which not yet written in PEG
-
PythonLexerBase:
- handles the Python indentations
- creates encoding token
- and manage many other things
- tokenizing BOM Unicode character at the start of the file so it is skipped in the token stream
- moved encoding detection from PythonLexerBase to a separate component (grun4py)