RTOS implementation of nextvi

[vmodamio]

Hi all,

I have made a typewriter which I enjoy a lot. It just runs linux in a small SBC, where I launch my VIM session. It has a keyboard, a display, and a battery. 
Now I want to reimplement it in a ESP32 microcontroller running a RTOS, where the main loop/application is a VI-like program. This nextvi would be perfect, if I manage to make the necessary modifications and strip the parts that couldnt be implemented out of the linux kernel. 
Before embracing the challenge: do you have any advise, or hint about how difficult would be this task? for example, with the program fully allocated in static memory, and the buffer/s loaded in a fix/limited piece of the PSRAM, while the write command writing the buffer into the SD card. 

Any comment please?

[kyx0r]

I find this amusing because I engineer custom RTOS solutions for a living. Nextvi has been designed with this in mind; to be as minimal as possible but still useable in a pragmatic way. Making RTOS out of it shouldn't be super bad, especially if you link it with something like picolibc. This RTOS idea have crossed my mind previously, though without practical use case or need.

[0] https://github.com/picolibc/picolibc

Of course, you could also just roll out your own allocator for malloc and free, snprintf, memcpy, etc.
More tricky functions to substitute would be sigaction, readdir, lstat, ioctl, getenv, tcsetattr, pipe, fork.

Since there would be no TTY subsystem and terminal on an RTOS, you need to roll out your own way to control the terminal and parsing the ANSI escapes. This should be your the first focus after you get basic functions like write() and read() working. Other nice to have things can be commented out without much risk in term.c

Do you plan on publishing any of this open source? I would love to see it happen :)

[kyx0r]

ESP 32 is quite tight, with 520 KiB RAM you may need to strip down some things, current smallest statically linked binary I can make is around 181.9K . You can save around 15K if you unmacro this case sensitive path which duplicates the whole regex engine just to get rid of the costly tolower branching.

[0] https://github.com/kyx0r/nextvi/blob/master/regex.c#L628

You can also replace the sbuf macros with actual functions to force compiler to reuse more. This will hurt the cpu, but leave you with smaller binary with more capacity to edit.

[1] https://github.com/kyx0r/nextvi/blob/master/vi.h#L79

You can adjust the sbuf allocator down to exact size instead of 1.5x growth here:

[2] https://github.com/kyx0r/nextvi/blob/master/vi.h#L42
exactly:
#define NEXTSZ(o, r) o + r

Buffer growth cost won't be amortized, but no byte of memory will go wasted.

Everything else in the editor I consider optimal, there isn't much more you can do without removing features.

[kyx0r]

Well, you can also edit this struct and replace pointers with int offsets for chrs.
Also, int *wid; can be made unsigned char *wid; since you may be willing to assume that character cannot be wider than 255 cells.
[0] https://github.com/kyx0r/nextvi/blob/master/vi.h#L204

[vmodamio]

Well, my expertise in RTOS is null. But I can tell why this idea would be huge for me! (implementing a text editor in RTOS).
Right now I'm running a full armbian Linux in a SBC just to write text files.
Implementing vi in an esp32 MCU with rtos means I could literally have the board in deep sleep almost all time, with interrupts on a keystroke.

I don't need multiple windows, not more than one text buffer at a time. Don't need syntax highligh neither (I'm just writing books in markdown).
So, except all the vi functionality for navigating and regrex operations, together with utf-8 and the fantastic feature of nextVI for changing the language keyboard layout, I think It could be stripped down quite a lot?

Another question: the esp32-s3 feather Im working with have extra 16MB flash, and 8MB PSRAM, so, do i still have the 512 kB dram constraint?

So far, I have built an example Zephyr shell, and this brings a beautiful terminal with customizable commands. That's how I started thinking in porting a small vi to RTOS.

If course, I would need a filesystem, and a static space in PSRAM where a text can be allocated.
Think on market digital typewriters: they lack all features from a text editor to make them practical. And now with this hype of eInk displays they work extremely slow.

The one I made have the keyboard on a dedicated MCU (2 mA when typing, 30 uA when idle), a reflective, memory in pixel display (600 uA when refreshing, 100 uA idle), and my bloated Linux running (downscaled ram Freq, and removed most peripherals, etc , running at 100 mA).
If I could use the ESP32, the whole thing would run for weeks!

[vmodamio]

btw, I saw just now that Zephyr uses now picolibc as default c lib!

So, the Zephyr shell running
https://docs.zephyrproject.org/latest/services/shell/index.html
is a good playground for nextVi on RTOS ;)

[kyx0r]

If you make your malloc allocator use PSRAM, then you will have 8MB to spare. I would imagine that program code and static memory and stack would be loaded into 512kB DRAM, which is likely enough as in Nextvi the most memory heavy parts are malloced anyway.

[vmodamio]

Hi Kyryl, I finished my dream project based on your nextVi, embedded in a ESP32s3. Thanks for all the encouragement.
Initially, I removed many features like highlight syntax (the display is monochrome) , right-to-left support, all the shell interaction and filesystem/ paths interaction. Later I realized, the amount of extra code added is like twice if not more the original scope.

[kyx0r]

Congratulations!

Later I realized, the amount of extra code added is like twice if not more the original scope.

If this is referring to Nextvi's code, yeah it can be surprisingly deceptive. Nextvi aims to do many things with very little, which is the ultimate goal. If you just wanted minimalism and almost no capability, text editor like kilo would have been a better fit. However, I'm glad that you figured how to strip down the things you don't need to fit your project. That's very powerful.

Nextvi is getting a lot more capability slowly with every major release. Though I suspect that will plateau soon. 6.0 will have infinite registers and marks which is the last bit of code that was limiting it & somewhat suboptimal. I intend on stopping changing the core soon, or I hope so. This is important for your project since memory is limited. Though its been years in development and finding perfection on a open ended task is pretty difficult.

Kind Regards,
Kyryl