ENH: AppearanceStream: Consider more encodings for Type1 core fonts#3905
Open
PJBrs wants to merge 14 commits into
Open
ENH: AppearanceStream: Consider more encodings for Type1 core fonts#3905PJBrs wants to merge 14 commits into
PJBrs wants to merge 14 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3905 +/- ##
==========================================
+ Coverage 97.86% 97.90% +0.03%
==========================================
Files 57 57
Lines 10738 10777 +39
Branches 2006 2017 +11
==========================================
+ Hits 10509 10551 +42
+ Misses 127 125 -2
+ Partials 102 101 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PJBrs
marked this pull request as draft
June 27, 2026 14:28
The unembedded Adobe core fonts contain around 315 characters and therefore
support more encodings than only cp1252 (WinAnsi). I've found that the
following encodings appear to be supported:
"cp1252", # Western European (WinAnsi base)
"cp1250", # Central / Eastern European
"cp1254", # Turkish
"cp1257", # Baltic Rim
"iso8859_15", # Western European ISO Alternate
Test:
import unicodedata
from pypdf._codecs import fill_from_encoding
from pypdf._codecs.core_font_metrics import CORE_FONT_METRICS
test_encodings = (
"cp1250", "cp1252", "cp1254", "cp1257", "iso8859_15",
)
for encoding in test_encodings:
encoding_list = fill_from_encoding(encoding)
for font in CORE_FONT_METRICS:
if CORE_FONT_METRICS[font].font_descriptor.name not in {"Symbol", "ZapfDingbats"}:
for char in encoding_list:
if unicodedata.category(char).startswith("C") or char.isspace():
continue
if char not in CORE_FONT_METRICS[font].character_widths.keys():
print (f"Unmatched: {encoding}: {char} ({ascii(char)})")
When generating a font resource for one of the 14 Type1 fonts, add a Differences encoding when applicable.
…_file test_font_from_font_file performs a series of tests that mimic a damaged font file in some way or other. One such test was the test for zero units per em. Just integrate it with the others.
PJBrs
force-pushed
the
3318
branch
2 times, most recently
from
July 5, 2026 12:00
3f029ba to
5c4aeec
Compare
PJBrs
marked this pull request as ready for review
July 5, 2026 12:26
The PDF specification of font subtypes can be somewhat confusing, because the font subtypes do not align exactly with font file types. First, the specification uses "Simple Fonts", which can consist of Type1 or TrueType font files (or Type3 fonts where "glyphs" are PDF objects). These fonts are 8-bit encoded. Then there are composite fonts (Type0), which are 16-bit encoded. When we initialise a font from a font file, we should set font.sub_type in line with the PDF font type, not with the file type. This patch therefore uses Type0 for fonts that we initialise from TrueType font files, in accordance with the fact the we 16-bit encode them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The unembedded Adobe core fonts contain around 315 characters and therefore support more encodings than only cp1252 (WinAnsi). I've found that the following encodings appear to be supported:
"cp1252", # Western European (WinAnsi base)
"cp1250", # Central / Eastern European
"cp1254", # Turkish
"cp1257", # Baltic Rim
"iso8859_15", # Western European ISO Alternate
When we encounter form field text with unencodable characters with one of the 14 Adobe core fonts, we try one of the above encodings instead. If successful, we make a new Font instance with that encoding, add it to the appearance stream, and add it to the document's font resources with a differences encoding and an associated ToUnicode cmap.
Fixes #3318
Achieves full coverage for _font.py.
This PR does include a small bug fix very similar in character to the added code, and refactors one test.
Assisted by Google Gemini, especially the changes for generating the ToUnicode cmap.
EDIT
I added some patches to always fall back to StandardEncoding for simple fonts when /Encoding is not defined in the font resource (and when a Type1 font does not provide its own encoding). This simplifies my original PR (don't need to deal with charmap encoding anymore. I've changed code elsewhere accordingly.