Maintenance: float2nr and comments - Fix numeric accuracy issues#1724
Open
claude[bot] wants to merge 2 commits intomasterfrom
Open
Maintenance: float2nr and comments - Fix numeric accuracy issues#1724claude[bot] wants to merge 2 commits intomasterfrom
claude[bot] wants to merge 2 commits intomasterfrom
Conversation
- BitwiseLeftShiftOperatorTest: `1<<63` in Vim gives Long.MIN_VALUE (-9223372036854775808), not `-Long.MAX_VALUE` (-9223372036854775807). These differ by 1 due to two's complement representation. - DivisionHandler: the previous comment only described the 0/0 → NaN → 0 case, missing that N/0 → ±Infinity → ±Int.MAX/MIN_VALUE, which also matches Vim's specified integer-division-by-zero behaviour. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Vim's float2nr() truncates out-of-range floats to ±0x7fffffff (±2147483647), not ±0x80000000. The previous implementation used Double.toInt() directly, which maps very large negative doubles to Int.MIN_VALUE (-2147483648) instead of -Int.MAX_VALUE (-2147483647). Fix by clamping the double to [-Int.MAX_VALUE, Int.MAX_VALUE] before converting to Int, which matches the Vim documentation behaviour. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Inspected the VimScript expression operators and float function implementations, starting from
BitwiseLeftShiftOperatorTest.kt.Issues found
Inaccurate comment in
BitwiseLeftShiftOperatorTest.kt: The comment statedecho 1<<63returns-long.max_value(i.e., -9223372036854775807), but the correct value isLong.MIN_VALUE(-9223372036854775808). In 64-bit two's complement, setting only the MSB gives the minimum value, which differs from-Long.MAX_VALUEby 1.Misleading comment in
ArithmeticOperatorHandlers.kt(DivisionHandler): The original comment said "Doubles give NaN, which becomes 0 when converted to integer." This only described the0/0case. ForN/0where N≠0, doubles give±Infinity, which converts to±Int.MAX/MIN_VALUE— also the correct Vim behaviour for integer division by zero. The fix makes the comment accurate for all cases.Bug in
Float2NrFunctionHandler:float2nr(-1.0e150)returned -2147483648 (Int.MIN_VALUE) instead of -2147483647 (-Int.MAX_VALUE). The Vim docs specify that out-of-range values are clamped to±0x7fffffff(not±0x80000000). The previous implementation usedDouble.toInt()directly, which maps very large negative doubles toInt.MIN_VALUEdue to JVM conversion rules. The fix usescoerceIn(-Int.MAX_VALUE.toDouble(), Int.MAX_VALUE.toDouble())before converting, matching Vim's specification.Changes made
BitwiseLeftShiftOperatorTest.kt: Corrected-long.max_value→Long.MIN_VALUE (-9223372036854775808)in commentArithmeticOperatorHandlers.kt: ImprovedDivisionHandlercomment to accurately describe all three division-by-zero casesFloat2NrFunctionHandler.kt: UsecoerceInto clamp to[-Int.MAX_VALUE, Int.MAX_VALUE]before converting to IntFloat2NrFunctionTest.kt: Updated expected output from-2147483648to-2147483647, removed@VimBehaviorDiffersannotation, removed now-unused importWhy these changes improve the code
The comment fixes prevent future developers from being misled about numeric behaviour. The
float2nrfix corrects a documented deviation from Vim's behaviour (it was annotated@VimBehaviorDiffers(shouldBeFixed = true)), making IdeaVim more compatible with Vim.🤖 Generated with Claude Code