-
PythonParser.g4 is the ANTLR4 parser grammar that based on the official Python PEG grammar
-
PythonLexerBase class
- handles the Python indentations
- creates encoding token
- tokenizes f-string, t-string literals
- and manage many other things
- parser grammar update for Python 3.14.2
- tokenizing t-string literals
- tokenizing BOM Unicode character at the start of the file so it is skipped in the token stream
- moved encoding detection from PythonLexerBase to a separate component (grun4py)