Skip to content

Add Test{{.StructName}}WriteReadLong#330

Open
tigrannajaryan wants to merge 1 commit intomainfrom
tigran/add-test-long-stream
Open

Add Test{{.StructName}}WriteReadLong#330
tigrannajaryan wants to merge 1 commit intomainfrom
tigran/add-test-long-stream

Conversation

@tigrannajaryan
Copy link
Collaborator

The new test uses longer test sequences. The existing Test{{.StructName}}WriteRead uses short sequences and does not mutate the records enough to test all interesting codepaths. In the past we had bugs that were only visible when large number of record varieties were generated. The new test generates approximately 500000 records in the slow mode, compared to only 1000 records generated by existing Test{{.StructName}}WriteRead.

@github-actions
Copy link

github-actions bot commented Jan 14, 2026

Benchmark Result

Benchmark diff with base branch
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │     sec/op     │    sec/op      vs base              │
SerializeNative/STEF/serialize-4                     9.886m ± 20%   10.011m ± 15%       ~ (p=0.818 n=6)
SerializeNative/STEFU/serialize-4                    35.13m ±  3%    35.16m ±  1%       ~ (p=0.937 n=6)
DeserializeNative/STEF/deser-4                       2.469m ±  2%    2.468m ±  1%       ~ (p=0.589 n=6)
DeserializeNative/STEFU/deser-4                      7.255m ±  1%    7.247m ±  0%       ~ (p=0.937 n=6)
SerializeFromPdata/STEF/serialize-4                  140.4m ±  3%    141.5m ±  3%       ~ (p=0.180 n=6)
SerializeFromPdata/STEFU/serialize-4                 34.89m ±  1%    35.25m ±  2%  +1.01% (p=0.004 n=6)
DeserializeToPdata/STEF/deserialize-4                46.31m ±  1%    46.14m ±  1%       ~ (p=0.937 n=6)
DeserializeToPdata/STEFU/deserialize-4               63.55m ±  1%    64.31m ±  2%       ~ (p=0.132 n=6)
STEFReaderRead-4                                     2.528m ±  3%    2.524m ±  1%       ~ (p=0.818 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4        3.490 ± 22%     3.417 ± 25%       ~ (p=1.000 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     73.93m ± 10%    74.91m ± 10%       ~ (p=0.485 n=6)
ReadSTEF-4                                           2.650m ±  3%    2.595m ±  3%  -2.08% (p=0.041 n=6)
ReadSTEFZ-4                                          3.462m ±  1%    3.242m ±  4%  -6.37% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                 7.987m ±  3%    7.297m ±  1%  -8.63% (p=0.002 n=6)
geomean                                              21.42m          21.20m        -1.05%

                                                 │ bench-main.txt │           bench-new.txt            │
                                                 │   sec/point    │  sec/point    vs base              │
SerializeNative/STEF/serialize-4                     147.8n ± 20%   149.8n ± 15%       ~ (p=0.818 n=6)
SerializeNative/STEFU/serialize-4                    525.4n ±  3%   525.8n ±  1%       ~ (p=0.786 n=6)
DeserializeNative/STEF/deser-4                       36.92n ±  2%   36.91n ±  1%       ~ (p=0.589 n=6)
DeserializeNative/STEFU/deser-4                      108.5n ±  1%   108.4n ±  1%       ~ (p=0.935 n=6)
SerializeFromPdata/STEF/serialize-4                  2.100µ ±  3%   2.117µ ±  3%       ~ (p=0.126 n=6)
SerializeFromPdata/STEFU/serialize-4                 521.9n ±  1%   527.2n ±  2%  +1.01% (p=0.004 n=6)
DeserializeToPdata/STEF/deserialize-4                692.7n ±  1%   690.2n ±  1%       ~ (p=0.937 n=6)
DeserializeToPdata/STEFU/deserialize-4               950.6n ±  1%   961.9n ±  2%       ~ (p=0.132 n=6)
STEFReaderRead-4                                     37.81n ±  3%   37.75n ±  1%       ~ (p=0.818 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4       4.436µ ± 22%   4.344µ ± 25%       ~ (p=1.000 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     93.96n ± 10%   95.21n ± 10%       ~ (p=0.485 n=6)
ReadSTEF-4                                           39.66n ±  3%   38.83n ±  3%  -2.08% (p=0.037 n=6)
ReadSTEFZ-4                                          51.82n ±  1%   48.52n ±  4%  -6.38% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                 119.5n ±  3%   109.2n ±  1%  -8.66% (p=0.002 n=6)
geomean                                              225.3n         222.9n        -1.06%

                                                 │ bench-main.txt │            bench-new.txt             │
                                                 │      B/op      │     B/op      vs base                │
SerializeNative/STEF/serialize-4                     3.337Mi ± 0%   3.338Mi ± 0%       ~ (p=0.855 n=6)
SerializeNative/STEFU/serialize-4                    7.557Mi ± 0%   7.557Mi ± 0%       ~ (p=0.983 n=6)
DeserializeNative/STEF/deser-4                       951.4Ki ± 0%   951.4Ki ± 0%       ~ (p=1.000 n=6) ¹
DeserializeNative/STEFU/deser-4                      1.715Mi ± 0%   1.715Mi ± 0%       ~ (p=1.000 n=6)
SerializeFromPdata/STEF/serialize-4                  76.55Mi ± 0%   76.55Mi ± 0%       ~ (p=0.535 n=6)
SerializeFromPdata/STEFU/serialize-4                 7.557Mi ± 0%   7.557Mi ± 0%       ~ (p=0.840 n=6)
DeserializeToPdata/STEF/deserialize-4                31.99Mi ± 0%   31.99Mi ± 0%       ~ (p=0.667 n=6)
DeserializeToPdata/STEFU/deserialize-4               38.88Mi ± 0%   38.88Mi ± 0%       ~ (p=0.745 n=6)
STEFReaderRead-4                                     953.1Ki ± 0%   953.1Ki ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4       3.385Gi ± 0%   3.383Gi ± 0%       ~ (p=0.394 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     20.30Mi ± 0%   20.30Mi ± 0%       ~ (p=0.671 n=6)
ReadSTEF-4                                           953.2Ki ± 0%   953.2Ki ± 0%       ~ (p=0.922 n=6)
ReadSTEFZ-4                                          10.29Mi ± 0%   10.29Mi ± 0%  +0.00% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                 13.44Mi ± 0%   13.44Mi ± 0%       ~ (p=0.320 n=6)
geomean                                              10.66Mi        10.66Mi       -0.00%
¹ all samples are equal

                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │   allocs/op    │  allocs/op   vs base                │
SerializeNative/STEF/serialize-4                      2.650k ± 0%   2.651k ± 0%       ~ (p=0.846 n=6)
SerializeNative/STEFU/serialize-4                      885.0 ± 0%    885.0 ± 0%       ~ (p=1.000 n=6)
DeserializeNative/STEF/deser-4                         463.0 ± 0%    463.0 ± 0%       ~ (p=1.000 n=6) ¹
DeserializeNative/STEFU/deser-4                        496.0 ± 0%    496.0 ± 0%       ~ (p=1.000 n=6) ¹
SerializeFromPdata/STEF/serialize-4                   134.7k ± 0%   134.7k ± 0%       ~ (p=0.242 n=6)
SerializeFromPdata/STEFU/serialize-4                   886.0 ± 0%    886.0 ± 0%       ~ (p=1.000 n=6) ¹
DeserializeToPdata/STEF/deserialize-4                 756.2k ± 0%   756.2k ± 0%       ~ (p=1.000 n=6) ¹
DeserializeToPdata/STEFU/deserialize-4                944.9k ± 0%   944.9k ± 0%       ~ (p=1.000 n=6) ¹
STEFReaderRead-4                                       463.0 ± 0%    463.0 ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4        13.15M ± 0%   13.15M ± 0%       ~ (p=0.069 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4      1.956k ± 0%   1.956k ± 0%       ~ (p=1.000 n=6)
ReadSTEF-4                                             464.0 ± 0%    464.0 ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEFZ-4                                            502.0 ± 0%    501.5 ± 0%       ~ (p=0.182 n=6)
ReadSTEFZWriteSTEF-4                                  1.231k ± 0%   1.231k ± 0%       ~ (p=1.000 n=6) ¹
geomean                                               6.305k        6.305k       -0.00%
¹ all samples are equal
Benchmark result
benchstat bench-new.txt
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-new.txt │
                                                 │    sec/op     │
SerializeNative/STEF/serialize-4                    10.01m ± 15%
SerializeNative/STEFU/serialize-4                   35.16m ±  1%
DeserializeNative/STEF/deser-4                      2.468m ±  1%
DeserializeNative/STEFU/deser-4                     7.247m ±  0%
SerializeFromPdata/STEF/serialize-4                 141.5m ±  3%
SerializeFromPdata/STEFU/serialize-4                35.25m ±  2%
DeserializeToPdata/STEF/deserialize-4               46.14m ±  1%
DeserializeToPdata/STEFU/deserialize-4              64.31m ±  2%
STEFReaderRead-4                                    2.524m ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4       3.417 ± 25%
STEFDeserializeMultipart/astronomy-otelmetrics-4    74.91m ± 10%
ReadSTEF-4                                          2.595m ±  3%
ReadSTEFZ-4                                         3.242m ±  4%
ReadSTEFZWriteSTEF-4                                7.297m ±  1%
geomean                                             21.20m

                                                 │ bench-new.txt │
                                                 │   sec/point   │
SerializeNative/STEF/serialize-4                    149.8n ± 15%
SerializeNative/STEFU/serialize-4                   525.8n ±  1%
DeserializeNative/STEF/deser-4                      36.91n ±  1%
DeserializeNative/STEFU/deser-4                     108.4n ±  1%
SerializeFromPdata/STEF/serialize-4                 2.117µ ±  3%
SerializeFromPdata/STEFU/serialize-4                527.2n ±  2%
DeserializeToPdata/STEF/deserialize-4               690.2n ±  1%
DeserializeToPdata/STEFU/deserialize-4              961.9n ±  2%
STEFReaderRead-4                                    37.75n ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4      4.344µ ± 25%
STEFDeserializeMultipart/astronomy-otelmetrics-4    95.21n ± 10%
ReadSTEF-4                                          38.83n ±  3%
ReadSTEFZ-4                                         48.52n ±  4%
ReadSTEFZWriteSTEF-4                                109.2n ±  1%
geomean                                             222.9n

                                                 │ bench-new.txt │
                                                 │     B/op      │
SerializeNative/STEF/serialize-4                    3.338Mi ± 0%
SerializeNative/STEFU/serialize-4                   7.557Mi ± 0%
DeserializeNative/STEF/deser-4                      951.4Ki ± 0%
DeserializeNative/STEFU/deser-4                     1.715Mi ± 0%
SerializeFromPdata/STEF/serialize-4                 76.55Mi ± 0%
SerializeFromPdata/STEFU/serialize-4                7.557Mi ± 0%
DeserializeToPdata/STEF/deserialize-4               31.99Mi ± 0%
DeserializeToPdata/STEFU/deserialize-4              38.88Mi ± 0%
STEFReaderRead-4                                    953.1Ki ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4      3.383Gi ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4    20.30Mi ± 0%
ReadSTEF-4                                          953.2Ki ± 0%
ReadSTEFZ-4                                         10.29Mi ± 0%
ReadSTEFZWriteSTEF-4                                13.44Mi ± 0%
geomean                                             10.66Mi

                                                 │ bench-new.txt │
                                                 │   allocs/op   │
SerializeNative/STEF/serialize-4                     2.651k ± 0%
SerializeNative/STEFU/serialize-4                     885.0 ± 0%
DeserializeNative/STEF/deser-4                        463.0 ± 0%
DeserializeNative/STEFU/deser-4                       496.0 ± 0%
SerializeFromPdata/STEF/serialize-4                  134.7k ± 0%
SerializeFromPdata/STEFU/serialize-4                  886.0 ± 0%
DeserializeToPdata/STEF/deserialize-4                756.2k ± 0%
DeserializeToPdata/STEFU/deserialize-4               944.9k ± 0%
STEFReaderRead-4                                      463.0 ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4       13.15M ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4     1.956k ± 0%
ReadSTEF-4                                            464.0 ± 0%
ReadSTEFZ-4                                           501.5 ± 0%
ReadSTEFZWriteSTEF-4                                 1.231k ± 0%
geomean                                              6.305k

@tigrannajaryan tigrannajaryan force-pushed the tigran/add-test-long-stream branch 3 times, most recently from 3f59e6d to e27eab3 Compare January 19, 2026 19:09
@tigrannajaryan tigrannajaryan marked this pull request as ready for review January 19, 2026 19:09
@tigrannajaryan tigrannajaryan force-pushed the tigran/add-test-long-stream branch from e27eab3 to 54e086c Compare January 25, 2026 18:27
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new long-running test Test{{.StructName}}WriteReadLong to test writer/reader functionality with a much larger number of records (up to 500,000 in slow mode vs 1,000 in existing tests). The PR also refactors MemReaderWriter into an exported type in a separate file for broader use.

Changes:

  • Added new Test{{.StructName}}WriteReadLong test function to the template that generates approximately 10,000-100,000 records (controlled by STEF_ENABLE_SLOW_TESTS environment variable)
  • Moved and exported MemReaderWriter from frame_test.go to a new public file memreaderwriter.go for reuse in generated tests
  • Removed unnecessary require.NotNil assertions for reader.Record since it's always initialized
  • Fixed fuzz tests to use proper CopyFrom pattern instead of mutating writer.Record directly
  • Added documentation clarifying that mutateRandom leaves modified flags in an undefined state

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
stefc/templates/go/writer_test.go.tmpl Template file: adds new long-running test, removes NotNil check, fixes fuzz test to use CopyFrom pattern
stefc/templates/go/struct.go.tmpl Adds documentation comment about undefined state of modified flags after mutateRandom
go/pkg/memreaderwriter.go New file: exports MemReaderWriter for use in generated tests
go/pkg/frame_test.go Updates to use exported MemReaderWriter (but has critical bugs due to unexported field access)
go/otel/otelstef/*.go Generated files: adds mutateRandom documentation comments
go/otel/otelstef/*writer_test.go Generated test files: adds new long-running tests and fixes fuzz tests
examples/*/*writer_test.go Generated test files: adds new long-running tests and fixes fuzz tests
examples//.go Generated files: adds mutateRandom documentation comments

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@tigrannajaryan tigrannajaryan force-pushed the tigran/add-test-long-stream branch 3 times, most recently from 564e535 to c8558b4 Compare February 4, 2026 21:23
The new test uses longer test sequences. The existing Test{{.StructName}}WriteRead
uses short sequences and does not mutate the records enough to test all
interesting codepaths. In the past we had bugs that were only visible when large
number of record varieties were generated. The new test generates approximately
500000 records in the slow mode, compared to only 1000 records generated by
existing Test{{.StructName}}WriteRead.
@tigrannajaryan tigrannajaryan force-pushed the tigran/add-test-long-stream branch from c8558b4 to a489b32 Compare February 4, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant