Skip to content

Print output bit count#25

Open
GediminasMasaitis wants to merge 1 commit intoemmanuel-marty:masterfrom
GediminasMasaitis:print_bits
Open

Print output bit count#25
GediminasMasaitis wants to merge 1 commit intoemmanuel-marty:masterfrom
GediminasMasaitis:print_bits

Conversation

@GediminasMasaitis
Copy link
Copy Markdown

@GediminasMasaitis GediminasMasaitis commented Nov 8, 2025

This PR adds reporting of the total bits used in the compressed stream, as well as how many bytes were used in the last byte.

The reasoning for this is that I have this project 4k.c, a chess engine made to be <4kb, for the TCEC 4K tournament https://wiki.chessdom.org/TCEC_4k_Rules. Source of my project: https://github.com/GediminasMasaitis/4k-dot-c It has a thing I call the "blackbox compressor" - if you look at the actual source file 4k.c, you will see that it has all sorts of macros ('G', 'H', 'S') which don't really do anything. They are markers for the blackbox.

How this blackbox works is - I marked every permutation that can be made in code, that doesn't drastically affect the output of the engine. Then I run makefile multiple thousands of times, seeing if the permutation of the "group" made gcc output a binary that is more compressible using apultra, and results in a smaller size.

The optimization I could do if this was in place is instead of checking the size of the binary after compression, I could check the bit size of the output. Which would just give 8x the resolution. Then, for example, if a permutation saves 2 bits it can be applied, then another one that saves 3 bits, and so on, and in the end, a whole byte of few may be saved.

Example of output after patch, the line added is "Compressed bits:":

make loader MINI=true NOSTDLIB=true
mkdir -p build
gcc -std=gnu2x -Wno-deprecated-declarations -Wno-format -DNOSTDLIB -nostdlib -fno-pic -fno-builtin -fno-stack-protector -march=haswell -Oz -DNDEBUG -c 4k.c
gcc -nostdlib -Wl,-Map=./build/4kc.map -Wl,-T 64bit-noheader.ld -o ./build/4kc 4k.o
ls -la ./build/4kc
-rwxrwxrwx 1 gedas gedas 5267 Nov  8 17:23 ./build/4kc
apultra -stats -v ./build/4kc ./build/4kc.ap
Compressed './build/4kc' in 0.025032 seconds, 0.2 Mb/s, 3022 tokens (1.74289 bytes/token), 5267 into 3817 bytes ==> 72.4701 %
Tokens: literals: 1640 short matches: 400 normal matches: 293 large matches: 549 rep matches: 139 EOD: 1
Compressed bits: 30530 last byte: 2 bits
Offsets: min: 1 avg: 444 max: 4258 count: 981
Match lens: min: 2 avg: 3 max: 23 count: 981
RLE1 lens: min: 2 avg: 4 max: 7 count: 11
RLE2 lens: none
Safe distance: 1452 (0x5AC)
fasm -d START_LOCATION=$(grep '_start' ./build/4kc.map | awk '{print $1}') loader.asm loader.o
flat assembler  version 1.73.32  (16384 kilobytes memory)
2 passes, 4736 bytes.
gcc -nostdlib -Wl,-T 64bit-loader.ld -Wl,-Map=./build/loader.map -o ./build/4kc loader.o
ls -la ./build/4kc
-rwxrwxrwx 1 gedas gedas 4090 Nov  8 17:23 ./build/4kc
md5sum ./build/4kc
9d007849c46799034deb7669f0867292  ./build/4kc

This patch would likely me allow to optimize my binary better, but even without that, it's just nice info to have in the verbose output.

Let me know if you have further questions

@GediminasMasaitis
Copy link
Copy Markdown
Author

GediminasMasaitis commented Nov 8, 2025

Just right off-the-bat, using this patch, with adjusting the blackbox to parse the ouput bits, this saved 2 bytes in the main branch: GediminasMasaitis/4k-dot-c@b699655

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant