Skip to content

Linter: Fix byte-offset/UTF-16 index mismatch corrupting autofix and offense locations#1866

Open
drjayvee wants to merge 1 commit into
marcoroth:mainfrom
drjayvee:fix/string-interpolation
Open

Linter: Fix byte-offset/UTF-16 index mismatch corrupting autofix and offense locations#1866
drjayvee wants to merge 1 commit into
marcoroth:mainfrom
drjayvee:fix/string-interpolation

Conversation

@drjayvee

@drjayvee drjayvee commented Jul 21, 2026

Copy link
Copy Markdown

Heads up: this commit and PR are LLM-generated and only superficially reviewed by me. The bug is real though! End of human output :D

Summary

Prism reports node locations as UTF-8 byte offsets, but several linter rules and the erb-prefer-direct-output autofix used those offsets directly as UTF-16 string-index offsets when calling source.substring(). This is only correct when the file is pure ASCII — any multi-byte character earlier in the file (even inside an unrelated tag or comment) shifts every subsequent byte offset, corrupting the result.

This is what caused reports like:

-            <%= "#{card_event.event_type}(#{data})" %>
+            <%= ard_event.event_type} %>(<%= ata} %>)

when running herb-lint --fix on a file that has any non-ASCII character (accented letters, emoji, etc.) earlier in the source.

  • Adds stringIndexFromByteOffset to @herb-tools/core to correctly convert a Prism UTF-8 byte offset to a UTF-16 string index.
  • Uses it in the erb-prefer-direct-output autofix (ERBStringToDirectOutputRewriter.extractStringContent / extractExpressionSource), which was corrupting the rewritten source.
  • Uses it in locationFromOffset (shared by erb-no-instance-variables-in-partials, erb-no-debug-output, erb-no-unused-literals, erb-no-unused-expressions, erb-prefer-direct-output), which was reporting the wrong offense line/column whenever non-ASCII characters preceded the offense.
  • Adds a substringFromByteOffset helper and uses it in the 3 rules that extracted a source snippet for their message text (erb-no-debug-output, erb-no-unused-literals, erb-no-unused-expressions), which were showing a corrupted snippet in the same scenario.

Test plan

  • New autofix regression tests in erb-prefer-direct-output.autofix.test.ts reproducing the reported bug (preceded by an accented character / emoji)
  • New unit tests for stringIndexFromByteOffset in core/test/prism/byte-offset.test.ts
  • New regression tests for the location/message corruption in erb-no-debug-output.test.ts, erb-no-unused-literals.test.ts, erb-no-unused-expressions.test.ts, erb-no-instance-variables-in-partials.test.ts
  • Verified end-to-end against real Prism-parsed byte offsets (via the published 0.10.2 packages, patched with this fix) that all of the above scenarios produce corrupted output before the fix and correct output after

…offense locations

Prism reports node locations as UTF-8 byte offsets, but several rules and the
erb-prefer-direct-output autofix used them directly as UTF-16 string indices
via `source.substring()`. Any multi-byte character earlier in the file (even
in an unrelated tag) shifted every subsequent offset, corrupting the
generated output and misreporting offense locations/messages.

Adds `stringIndexFromByteOffset` to `@herb-tools/core` and uses it wherever
Prism byte offsets are converted to string positions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant