-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Created a version of the parser that makes use of a memmap for the underlying file read. It currently results in a small 1-3% performance win. However, I believe by simplifying the parse_txt mmap version we could see a larger win. This would be by for example by eliminating the branching in the main loop where we have to check if we're on the 1st line or not and if so we need to figure out how many columns are being provided.
Next, I think it might be worth checking to see if we can't create parallel version of things for the memmap version, and if we end up seeing any sort of performance wins for large files. I have an idea for how to do this that should work pretty well, but it'll require some work to get there.
Metadata
Metadata
Assignees
Labels
No labels