[STRATCONN-6533] Fix inconsistent field ordering in LiveRamp CSV generation

[abhandage]

Problem

The LiveRamp Audiences destination was generating CSV files with inconsistent field ordering across different batches. The field order depended on which payload was processed first in each batch, leading to:

• Different column positions for the same fields across files
• Potential issues with LiveRamp ingestion expecting consistent schemas

Solution

Modified the generateFile() function in operations.ts to sort fields alphabetically (after audience_key):

1. Collect all unique field names from all payloads
2. Sort the fields alphabetically (excluding audience_key which always comes first)
3. Use this consistent sorted order for all CSV generation

Changes

• Modified operations.ts:53-57 to add alphabetical sorting logic
• Updated all references from headerArray to sortedHeaderArray
• Added 2 new test cases demonstrating consistent field ordering
• Updated 8 existing test cases with new alphabetically sorted expectations

Field Ordering

Before: Field order varied based on payload processing order
After:

• audience_key (always first)
• All other fields alphabetically sorted (e.g., age, city, email, first_name, phone, etc.)

Testing

Tested in stage with instruction here

Generated dummy data with 2 batches with randomly missing properties. When this fix was not applied the CSV looked like below where the headers are not alphabetically ordered.

After applying the fix, the data is ordered.

Ticket

STRATCONN-6533

[Copilot]

Pull request overview

This PR fixes inconsistent CSV header ordering in the LiveRamp Audiences destination by making the header generation deterministic across batches, preventing schema/column drift when payload ordering varies.

Changes:

• Sort CSV headers alphabetically while keeping audience_key as the first column.
• Update CSV generation to use the new sorted header array for row alignment and header output.
• Update and add tests to validate consistent header ordering across different payload orders.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
packages/destination-actions/src/destinations/liveramp-audiences/operations.ts	Ensures deterministic CSV header ordering by sorting non-audience_key fields alphabetically.
packages/destination-actions/src/destinations/liveramp-audiences/tests/operations.test.ts	Updates expected outputs and adds coverage to confirm consistent ordering across batches.

[sayan-das-in]

Changes looks good.
Can we check with LR Support if changing field order impacts existing configured audiences ?

[abhandage]

thing to note
https://docs.liveramp.com/connect/en/formatting-column-based-files.html

Make sure each header matches the contents of its column.

Subsequent uploads must be consistent with the original file's formatting, including header order and syntax. We recommend that you format headers using all capital letters with no spaces or underscores so that you can reliably format subsequent files the same way.

[abhandage]

thing to note https://docs.liveramp.com/connect/en/formatting-column-based-files.html

Make sure each header matches the contents of its column.
Subsequent uploads must be consistent with the original file's formatting, including header order and syntax. We recommend that you format headers using all capital letters with no spaces or underscores so that you can reliably format subsequent files the same way.

confirmed with Liveramp team, this is not a problem.

[codecov]

Codecov Report

❌ Patch coverage is 82.35294% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.35%. Comparing base (50adcc7) to head (4334d46).
⚠️ Report is 7 commits behind head on main.

Files with missing lines	Patch %	Lines
...ions/liveramp-audiences/audienceEnteredS3/index.ts	66.66%	1 Missing and 2 partials ⚠️
...ns/liveramp-audiences/audienceEnteredSftp/index.ts	66.66%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3587      +/-   ##
==========================================
+ Coverage   80.31%   80.35%   +0.03%     
==========================================
  Files        1283     1283              
  Lines       23567    23604      +37     
  Branches     4726     4742      +16     
==========================================
+ Hits        18928    18967      +39     
+ Misses       3757     3753       -4     
- Partials      882      884       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.