✨ NeuralNetworks

• ⚙️ Improved:
• • Performance of USE_INT_QUANTIZATION, removed unnecessary repetitions of MULTIPLY_BY_INT_IF_QUANTIZATION 965c08d

Important

In a few days, I will be leaving home for a year to complete my mandatory military service. I don't have a choice, expect less frequent updates and slower responses from me. Update 26/2/2026 They rejected me [...]

✨ NeuralNetworks

• ⚠️ Changed:
• • Terminology related to SIMD, corrected to DSP-acceleration thanks to @oursland #16 (reply in thread) e3bdead

Important

In a few days, I will be leaving home for a year to complete my mandatory military service. I don't have a choice, expect less frequent updates and slower responses from me.

✨ NeuralNetworks

• 🛠️ Fixed:
• • (Undocumented) ENABLE_SINGLE_TIMESTEP_THRESHOLD when used with GRU or LSTM Layers 30b83cf
• • USE_PROGMEM macro-logic effecting non-AVR_PROGMEM_LOGIC for both USE_INT_QUANTIZATION & USE_64_BIT_DOUBLE 0f77ddd
• • Examples acad3d6
• ✨ Added:
• • RAM_EFFICIENT_HILL_CLIMB_WITHOUT_NEW-support for USE_EXTERNAL_FRAM or USE_INTERNAL_EEPROM 3e9ca07
• • Examples 60a598e 55e0adb
• ⚙️ Improved:
• • Source-code conceptually c9edb14 902ce62 457cafd
• • Examples 9726e0f f3c726d 0ece88a 35afcbc

Important

In a few days, I will be leaving home for a year to complete my mandatory military service. I don't have a choice, expect less frequent updates and slower responses from me.

✨ NeuralNetworks

• 🛠️ Fixed:
• • Removed file closing from save() to avoid inconsistency with external file management 0cc7830
• ✨ Added:
• • Partial bare-metal & native-os support, including FS d075085 f0e031b ad346a5 3ef9a2b d91a235
• • REDUCE_RAM_RESET_STATES_BY_DELETION _4_OPTIMIZE via 0B1 38c780f
• • DISABLE_NN_SERIAL_SUPPORT-macro just in case 7ce4c22
• ⚙️ Improved:
• • Misleading macro-logic that could potentially mess up hill-climb-alternatives in the future f377f87
• • Enabling F()-macro now simply gets ignored instead of raising an #error for ESP32 d075085
• • Source-code conceptually 515fc3f 3bf5414
• • Source-code Macros f6d7182 43d22e2
• • examples 123a187
• ⚠️ Changed:
• • Macro As__No_Common_Serial_Support renamed AS__NO_COMMON_SERIAL_SUPPORT 64782b4

✨ NeuralNetworks

• 🛠️ Fixed:
• • (Changed) #51 Missing LamdaSELU in SELU-activation function & >= at fx > 0 in derivative 4246a8f
• • (v3.1.7) REDUCE_RAM_DELETE_OUTPUTS unproper NULL-ification of last-layer's outputs, resulting in potential undefined behavior. e92d3dc
• • Invalid (IDFLOAT instead of DFLOAT)-type when using single-bias in EEPROM\FRAM-FeedForward with int-quantization 7356089 b25819f
• • USE_PROGMEM compatibility of non-__AVR__ MCUs aa77d01 2f768af
• • A closing-parentheses related to Softmax somewhere...
• • Few compiler warnings 72ad515
• ✨ Added:
• • RNN support 443962f 475f283 172c88e 6852083 d9e791b 5bf44c9 9359793 8594613 8350fc4 5f38248 e473aa2
• • GRU support 0648099 b7cab3b d9e791b 5bf44c9 6b482b7 8594613 8350fc4 2fe31ef c92230b ab1a948 5b82cfa
• • LSTM support 8a5f26d d9e791b 5bf44c9 6b482b7 8594613 8350fc4 2fe31ef 0439db9 c92230b 5b82cfa
• • DENSE+RNN support a5b5467
• • REDUCED_SKETCH_SIZE_DOT_PROD in favor of unimplemented optimization-bit 0B00001 0bd8a1f
• • #24 Support for saving into EEPROM\FRAM when you USE_PROGMEM bffe09c
• • New examples 399d0b5
• ⚙️ Improved:
• • Replaced unimplemented 0B001 optimization-bit in favor of DISABLE_SIMD_SUPPORT 7ee208f
• • Safety, by adding a macro-condition that checks if MCU supports 64bit\8byte double-precision 9e7514e
• • (Changed) Activation functions & their variable are now static 622e1f7 f8f222b
• • #49 the use of multiple activation-functions 72010df 21b5e44
• • AlphaSELU & LamdaSELU precision 8dbf5ab
• • Source-code mostly Visually 2d5002d 7bf22ac 6d286a8 642c2e9
• • Source-code Macros 80ae017 d14806b acb92cc f06c20d cc23ab2 67c1138 d7d00d4 c3bb87e d14f301
• • Source-code 06b9ebd c3bda35 96b5f34 7300d7e f815118 a114a5c df0ef21 f1e92de
• • examples 54a759a 630f4e8 ef7fa9e 7fe5db9 0bd8a1f 399d0b5
• ⚠️ Changed:
• • Library is now under the GPLv3 LICENSE a5e9b36
• • FS-support now gets enabled via _OPTIMIZE_3 0B00000010 fd20e89

Note

Special thanks to Vibhutesh Kumar Singh for using my library for his latest paper "Memory-Efficient Neural Network Deployment Methodology for Low-RAM Microcontrollers Using Quantization and Layer-Wise Model Partitioning", our excellent collaboration led to the actual support for int-quantization [...] Moreover, I'd like to thanks kritonix.ai - a startup company - for giving me the motivation to continue developing this library, with the promise of bringing me on board once the company stabilizes. Last but not least I'd like to thanks Jiajun Guan for also using the library for his Master's Thesis "Neural Network for Monitoring Infant Feeding Process in the SmartBottle Device"

🐜 NeuralNetworks

• 🛠️ Fixed:
• • 1d140d2 REDUCE_RAM_DELETE_OUTPUTS unproper NULL-ification of last-layer's outputs, resulting in potential undefined behavior.

Note

Anouncement. tl;dr: A few weeks ago, a company specializing in embedded AI offered me an opportunity to join their team. In return, the development of this library would become private under their ownership. What's your opinion?

🐜 NeuralNetworks

• 🛠️ Fixed:
• • 0c295c8 (NOT)REDUCE_RAM_WEIGHTS_COMMON memmory-leak in the destructor, effecting SD-load() & not-pretrained-NNs
• • a4a566f MULTIPLE_BIASES_PER_LAYER Potential memmory-leak in the destructor, effecting SD-load() & not-pretrained-NNs, due to undefined-behavior
• • aa39e38 Potential undefined behavior in destructor or load() when REDUCE_RAM_DELETE_OUTPUTS is used with SUPPORTS_SD_FUNCTIONALITY
• • 86c9e59 Expected ';' error-typo, effecting FeedForward_Individual for EEPROM or FRAM when NO_BIAS && ACTIVATION__PER_LAYER
• ✨ Added:
• • a337501 7bf00d0 f02ee45 Saving and loading directly to FS/File thanks to @Shirkit
• • 38908de Support backpropagation for NNs that don't utilize hidden-layers. (SUPPORT_NO_HIDDEN_BACKPROP)
• • f004782 6400e4e 2ac7e06 a5d9545 A computationally-expensive but memmory-efficient Hill-Climbing training algorithm + example
• • 93e7baa HILL_CLIMB_DYNAMIC_LEARNING_RATES-optimization to allow user-changes in learning-rate(s) during HillClimb
• ⚙️ Improved:
• • 738491f preLgamma logic
• • 01f5244 Constructor, via deledation
• • 00a6e8f GELU via erf() improvements
• • e162b40 FeedForward_Individual when USE_INTERNAL_EEPROM or USE_EXTERNAL_FRAM
• • a4b0367 bc5bc47 Softmax implementation & solved (rare but) potetnial issue
• ⚠️ Changed:
• • 4e9d595 Added #error message when ESP32 is used with F_MACRO-optimization
• • 1080b87 Added #error message when ESP32 is used with USE_PROGMEM-optimization
• • 4f0cf1e Fixed Embarrassing typo LeakyELU -> LeakyReLU (appropriate #error gets thrown, so don't worry)

Note

Anouncement. tl;dr: A few weeks ago, a company specializing in embedded AI offered me an opportunity to join their team. In return, the development of this library would become private under their ownership. What's your opinion?

🐜 NeuralNetworks

• 🛠️ Fixed:
• • a2707b8 Crucial Softmax issue: when used with ACTIVATION__PER_LAYER and not ALL_ACTIVATION_FUNCTIONS
• • 7b52a56 Potential issue with CATEGORICAL_CROSS_ENTROPY & BINARY_CROSS_ENTROPY when you USE_64_BIT_DOUBLE with REDUCE_RAM_WEIGHTS_LVL2
• ✨ Added:
• • 64dcb9d FRAM Examples
• • c572333 Example for SD migration to v3.0.0
• • dab6f55 Support for NN execution (partially) via external FRAM
• ⚙️ Improved:
• • a03b160 Unnecessary me->i_j++ logic
• • 26d2f20 Logic related to int and unsigned int
• • 2912a86 Unnecessary EEPROM-logic effecting sketch size
• • 6432955 Backpropagation algorithm, cutting flash memory usage by up to 200 bytes.
• • 9ac2b51 Prioritized "reduced-logic" over performance at FeedForward_Individual()
• ⚠️ Changed:
• • d4ce5e0 Optimized SD load() & save()

Warning

load() & save() previous implementations (although perfectly working) had significant design flaws, but the 3.0.0 release brings a much-improved versions of them. Note the breaking change! I’ve included a clear migration guide to help easily convert old NN-files to the new format via just a simple sketch. Alternatively, I'm providing limited backwards compatibility through save_old() and load_old(). However, please note that these legacy methods won't receive further updates or improvements over time.

🕸️ NeuralNetworks

• Fixed:
• • SD load(...) & save(...) for Teensy + return true
• • Conflicting macro definitions for Teensy & probably other MCUs\Boards
• Improved:
• • Removed redundant NULL-check above delete[] when REDUCE_RAM_DELETE_OUTPUTS is used

🕸️ NeuralNetworks

• Fixed:
• • #32 #39 warnings (+improvement)