Skip to content

Postgres batch reads silently discard child rows when scanning fails #70

Description

@abhinavkr26104

Description

PostgresStore.GetChildJobs ignores rows.Scan errors and continues (server/internal/store/postgres.go:81-103).

A malformed or NULL value, schema mismatch, driver conversion error, or future query change can remove a child from the returned slice without returning an error to the handler.

Reproduction

Use a SQL mock or database fixture that causes one selected child column to fail conversion, such as an incompatible duration_ms value.

Call GetChildJobs. The method returns the other rows and a nil error instead of failing the batch read.

At the API level, GET /v1/url-scraper/batch/:id builds results from the incomplete slice and may report a misleading derived status as though the missing child did not exist.

Expected behavior

Any row decoding failure should abort the read and be surfaced to the caller, or be explicitly represented as a failed child.

Actual behavior

The failing row is silently skipped at postgres.go:89-90, and the caller receives partial data with no indication that it is incomplete.

Why it matters

Batch results can omit URLs, causing clients to make incorrect completion or retry decisions. Silent data loss is particularly difficult to diagnose because the database row remains present.

Suggested fix

Return a wrapped scan error immediately and still check rows.Err() after iteration. Add a store test that verifies row-scan failures are propagated. Consider a shared JobStore contract test for both storage implementations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions