Skip to content

parser: dedupe the twin string/Buffer large-JSONL scanners #643

Description

@avs-io

Problem

src/parser.ts contains two structurally identical hand-rolled JSON scanners: the string version (findJsonStringEnd and friends, src/parser.ts:152-478) and a Buffer twin of every function (findJsonStringEndBuffer and friends, src/parser.ts:483-822). Roughly 340 lines exist twice. Any scanner fix (escape handling, nesting off-by-one) must land in both; a fix applied to one branch produces divergent parsing between small-string and large-buffer inputs, which surfaces as a silent size-dependent miscount rather than an error.

Evidence

  • src/parser.ts:152 function findJsonStringEnd(source: string, ...)
  • src/parser.ts:483 function findJsonStringEndBuffer(source: Buffer, ...)
  • Verified on main at 1a75484.

Proposed fix

Extract one scanner module generic over an indexable source (both string and Buffer support charCodeAt-style access; a tiny accessor closure keeps the hot path monomorphic). Delete the twin. Existing parser tests plus a fixture run through both entry paths guard the refactor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions