Bug in MonoBufferFn

[dhansel]

I believe there is a bug in the MonoBufferFn implementation in the smlnj-lib library in the current version (and the 20241230 release) of MLton.

The bug can be triggered by calling Random.toString (Random.rand (0,0)) (or any other initial seeds).
Random.toString uses a Word8Buffer (an instance of MonoBufferFn) to construct its result.
After a few seconds this call produces an "Uncaught exception: Overflow".

The bug appears to have been introduced when switching to SML/NJ 110.99.6.1 libraries. The ensureCapacity() function in the MonoBufferFn functor adds an extra line

mlton/lib/smlnj-lib/smlnj-lib.patch

Line 392 in ce19014

+ val amt = Int.max(curCap - len + curCap, amt)

that was not present in the smlnj-lib.patch file for the SML/NJ 110.99.3 libraries. If I see it correctly, that line doubles the size of the array each time ensureCapacity() is called, which after some calls leads to the Overflow condition (in that same line).

I assume the intent of the new code was to avoid the (worst-case) scenario where only a single byte of buffer space is allocated at a time (by doubling the buffer size when more space is needed). But as is the array size gets doubled for every call to ensureCapacity().

David

[MatthewFluet]

The MonoBufferFn code isn't actually from SML/NJ (directly). It's taken from https://github.com/SMLFamily/BasisLibrary/wiki/2018-001-Addition-of-monomorphic-buffers and it sample implementation https://github.com/SMLFamily/BasisLibrary/tree/master/Code/2018/001, since at some point the upstream SML/NJ Libraries were updated to make use of the MonoBufferFn.

Will investigate if the SML/NJ implementation of MonoBufferFn has the same issue.

[MatthewFluet]

The SML/NJ implementation of Word8Buffer (https://github.com/smlnj/legacy/blob/main/base/system/Basis/Implementation/word8-buffer.sml) uses a different resizing heuristic, but the intention of the change to mono-buffer.fun was to double the buffer size when growing it (SMLFamily/BasisLibrary@8d5681c).

A bugfix has been submitted to the BasisLibrary repo (SMLFamily/BasisLibrary#36).

Will shortly port the bugfix from the BasisLibrary repo to MLton's compatibility layer for SML/NJ Library.