Trim trailing whitespace without slicing per step

[dkuku]

trim_trailing/1 walked the string with binary_part/3, which allocated the three byte lookahead plus a fresh prefix on every step even though every prefix but the last one is discarded, and paid one allocation even when there was nothing to trim.

Carry the position instead and slice once at the end. The one-byte whitespace codepoints, which are the common case, now need no lookahead at all, so the lookahead tables only hold the multi-byte ones.

It starts to be slower with around 1k trailing spaces than the previous implementation, but the memory usage does not change with ascii white spaces. I think it is a very uncommon case.

list = Enum.map(1..100, &to_string/1) |> Enum.join("") 
one = list |> Kernel.<>(" ")
l20 = list |> Kernel.<>("  \n     \r          \n  ")
onek = list |> Kernel.<> Enum.map_join(1..1000, "", fn _ -> " " end)

Benchee.run(
  %{
    "old_no_trailing" => fn -> String.Break.trim_trailing(list) end,
    "new_no_trailing" => fn -> String.BreakNew.trim_trailing(list) end,
    "old_one_trailing" => fn -> String.Break.trim_trailing(one) end,
    "new_one_trailing" => fn -> String.BreakNew.trim_trailing(one) end,
    "old_20_trailing" => fn -> String.Break.trim_trailing(l20) end,
    "new_20_trailing" => fn -> String.BreakNew.trim_trailing(l20) end,
    "old_1k_trailing" => fn -> String.Break.trim_trailing(onek) end,
    "new_1k_trailing" => fn -> String.BreakNew.trim_trailing(onek) end
  },
  time: 1,
  memory_time: 1
)

results

Name                       ips        average  deviation         median         99th %
new_no_trailing         3.59 M      278.81 ns  ±2436.56%         241 ns         571 ns
new_one_trailing        3.43 M      291.26 ns  ±2290.17%         260 ns         441 ns
old_no_trailing         3.20 M      312.65 ns  ±1931.60%         280 ns         461 ns
old_one_trailing        2.95 M      338.92 ns  ±1279.22%         311 ns         491 ns
new_20_trailing         1.69 M      592.09 ns  ±1222.26%         541 ns         802 ns
old_20_trailing         1.30 M      769.91 ns   ±542.88%         691 ns        1623 ns
old_1k_trailing       0.0827 M    12094.93 ns    ±32.20%       11061 ns       22492 ns
new_1k_trailing       0.0757 M    13202.34 ns    ±19.70%       12994 ns       17172 ns

Comparison: 
new_no_trailing         3.59 M
new_one_trailing        3.43 M - 1.04x slower +12.45 ns
old_no_trailing         3.20 M - 1.12x slower +33.84 ns
old_one_trailing        2.95 M - 1.22x slower +60.11 ns
new_20_trailing         1.69 M - 2.12x slower +313.28 ns
old_20_trailing         1.30 M - 2.76x slower +491.10 ns
old_1k_trailing       0.0827 M - 43.38x slower +11816.12 ns
new_1k_trailing       0.0757 M - 47.35x slower +12923.53 ns

Memory usage statistics:

Name                Memory usage
new_no_trailing            432 B
new_one_trailing           432 B - 1.00x memory usage +0 B
old_no_trailing            472 B - 1.09x memory usage +40 B
old_one_trailing           512 B - 1.19x memory usage +80 B
new_20_trailing            432 B - 1.00x memory usage +0 B
old_20_trailing            912 B - 2.11x memory usage +480 B
old_1k_trailing          13832 B - 32.02x memory usage +13400 B
new_1k_trailing            432 B - 1.00x memory usage +0 B

**All measurements for memory usage were the same**

Btw - should the do_trim_leading above be private ?

[josevalim]

💚 💙 💜 💛 ❤️