Merged
Conversation
Fix so it does not fallback to insertAt for a new column, it tries to widen if it can early to allow usage of InsertFrom method to get performance increase.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves IntegerColumn.InsertFrom behavior so that inserting into a new/empty integer column can widen the backing storage up-front (instead of falling back to per-element InsertAt), enabling the fast bulk InsertFrom path for better performance.
Changes:
- Added
WidenToBitWidth(int targetBitWidth)helper to create/upgrade_datato a specific integer bit width. - Updated
IntegerColumn.InsertFromto widen_datato the source column’s bit width when needed, then use_data.InsertFrom(...)where possible. - Retained a per-element
InsertAtfallback when the target column is already wider than the source.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: 95d7a7c | Previous: 859330d | Ratio |
|---|---|---|---|
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.InnerJoin |
247822980 ns (± 7631222.986855457) |
307785430 ns (± 9897277.576973714) |
0.81 |
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.LeftJoin |
354946455.5555556 ns (± 8563566.613437288) |
461337240 ns (± 19940786.10567074) |
0.77 |
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ProjectionAndNormalization |
86386230 ns (± 8191863.148922024) |
125533570 ns (± 10748358.533159066) |
0.69 |
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.SumAggregation |
90700170 ns (± 8083875.481860307) |
138528520 ns (± 8580649.062577441) |
0.65 |
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ListAggWithMapAggregation |
1580879811.1111112 ns (± 110911654.86823787) |
1823440280 ns (± 56615189.59691922) |
0.87 |
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.WindowSum |
319136730 ns (± 30364692.09269213) |
359591925 ns (± 5946342.432057349) |
0.89 |
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ListAggWithStructAggregation |
1373551750 ns (± 95322866.55643602) |
1507727890 ns (± 61734781.22500134) |
0.91 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix so it does not fallback to insertAt for a new column, it tries to widen if it can early to allow usage of InsertFrom method to get performance increase.