Maximum number of ACElement

[Koxx3]

Hello,
I added few elements to my page. I now have around 50 ACElement.
The first display is OK, but it fail to reload the page.
What could be the cause ? too much memory ? it doesn't crash. It tried with or without json, same issue.
Thanks !

[Koxx3]

by the way, the page is loading but without any of the ACElements in the HTML content.

[Koxx3]

normal loading :

[Koxx3]

failed loading :

[Koxx3]

the elements loaded in the page are here ;)
https://github.com/Koxx3/SmartController_SmartDisplay_ESP32/blob/master/src/WifiSettingsPortal/WifiSettingsPortalSpecs.h

[Koxx3]

activating AC/PB debug give this :

22:27:51.184 > [AC] http server started
22:27:51.184 > wifi portal open !
22:28:01.026 > [AC] Host:,/settingspage,generated:/settingspage, allocated
22:28:01.051 > [AC] Host:192.168.10.236,/settingspage,already allocated
22:28:01.054 > [PB] Free heap:80872, content len.:unknown
22:28:01.054 > [PB] Res:200, Chunked:2
22:28:01.056 > [PB] Transfer-Encoding:chunked
22:28:01.079 > [AC] fetch /settingssave,elements stored
22:28:01.080 > [AC] CB in AHEAD /settingspage
22:28:01.081 > portal where = /settingssave
22:28:01.081 > load begin
22:28:01.089 > load end
22:28:01.231 > [PB] Failed building, free heap:71360
22:28:01.234 > [PB] at leaving build: 71388 free

it doesn't seems to be a free heap issue ...

[Koxx3]

• first load OK
• save
• reload

22:31:50.587 > [AC] /settingspage on hands
22:31:50.587 > [AC] /settingssave on hands
22:31:50.587 > [AC] /calibpage on hands
22:31:50.587 > [AC] /otapage on hands
22:31:50.587 > [AC] /otaflash on hands
22:31:50.587 > [AC] Current:KoxxWrls
22:31:50.587 > [AC] WiFi.config(IP=0.0.0.0, Gateway=0.0.0.0, Subnetmask=0.0.0.0, DNS1=0.0.0.0, DNS2=0.0.0.0)
22:31:50.587 > [AC] WiFi.begin().established IP:192.168.10.236
22:31:50.587 > start advertising
22:31:50.587 > Current sensor is not detected -> disable reading
22:31:50.587 > [AC] http server started
22:31:50.587 > wifi portal open !
22:32:39.161 > [AC] Host:,/settingspage,generated:/settingspage, allocated
22:32:39.184 > [AC] Host:192.168.10.236,/settingspage,already allocated
22:32:39.187 > [PB] Free heap:80756, content len.:unknown
22:32:39.187 > [PB] Res:200, Chunked:2
22:32:39.188 > [PB] Transfer-Encoding:chunked
22:32:39.212 > [AC] fetch /settingssave,elements stored
22:32:39.214 > [AC] CB in AHEAD /settingspage
22:32:39.215 > portal where = /settingssave
22:32:39.216 > load begin
22:32:39.221 > load end
22:32:39.331 > [PB] at leaving build: 57820 free
22:32:50.126 > [AC] Host:192.168.10.236,/settingssave,generated:/settingssave, allocated
22:32:50.308 > [AC] Host:192.168.10.236,/settingssave,already allocated
22:32:50.325 > [PB] Free heap:72372, content len.:unknown
22:32:50.326 > [PB] Res:200, Chunked:2
22:32:50.327 > [PB] Transfer-Encoding:chunked
22:32:50.351 > [AC] fetch /settingspage,elements stored
22:32:50.610 > [AC] CB in AHEAD /settingssave
22:32:50.610 > save begin
22:32:51.060 > [PB] at leaving build: 62544 free
22:32:54.437 > [AC] Host:192.168.10.236,/settingspage,generated:/settingspage, allocated
22:32:54.458 > [AC] Host:192.168.10.236,/settingspage,already allocated
22:32:54.462 > [PB] Free heap:79900, content len.:unknown
22:32:54.462 > [PB] Res:200, Chunked:2
22:32:54.463 > [PB] Transfer-Encoding:chunked
22:32:54.485 > [AC] fetch /settingssave,elements stored
22:32:54.489 > [AC] CB in AHEAD /settingspage
22:32:54.489 > portal where = /settingssave
22:32:54.489 > load begin
22:32:54.498 > load end
22:32:54.612 > [PB] Failed building, free heap:70744
22:32:54.615 > [PB] at leaving build: 70776 free

[Koxx3]

the html resulting page is 21KB

[Koxx3]

hummmm ... looking at the PageBuilder code, It seems to be String concat fault ...

https://stackoverflow.com/questions/55342103/how-many-bytes-can-you-store-inside-a-arduino-string

[Hieromon]

@Koxx3 The cause is probably as you expected. Could it be same case as #331.

[PB] Failed building, free heap:70744

Could you test with AutoConnect v1.3.0 combining PageBuilder v1.5.0? Each RC has been staged in the development branch, enhance/v130 for AutoConnect and enhance/v150 for PageBuilder.

[Koxx3]

for now, I reduced all variables names and used CSS styles instead of line, and tha page went from 21KB to 17KB, it seem to pass.

What changed in the new dev branches which could solve this ?

[Hieromon]

tha page went from 21KB to 17KB, it seem to pass.

Is this the result of testing with the development branch?
In PageBuilder v150, the memory management structure has changed significantly. It is devised to minimize string buffer fragmentation associated with HTML generation.

[Koxx3]

I changed my ACElement / HTML generation to reduce all variable names ;) I'll try to switch versions after but I'm not sure how to do it in PlatformIO. Le lun. 29 mars 2021 à 15:02, Hieromon Ikasamo ***@***.***> a écrit :

…

tha page went from 21KB to 17KB, it seem to pass. Is this the result of testing with the development branch? In PageBuilder v150, the memory management structure has changed significantly. It is devised to minimize string buffer fragmentation associated with HTML generation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#342 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACXMP3FF5U6PNRYAT5JLI3TTGB253ANCNFSM4Z6MDJNQ> .

-- Francois Deslandes

[Hieromon]

I diverted the WiFiSettingsPortal from the SmartController_SmartDisplay_ESP32 repository and tested your AutoConnectAux sets which have the numerous AutoConnectElements definitions you coded using AutoConnect v1.3.0 combing PageBuilder v1.5.0.

As a result, it was displayed normally, but by the time I captured the source code, the HEAD was already advanced and the size of the generated HTML was 16818 bytes. This is close to the size you mentioned the reduction, so this reproduction test could not determine if it could generate the original 21KB length.

Now that I've roughly caught up with the program structure of your code, and I can proceed with the validation for AutoConnectAux with numerous AutoConnectElements. Do you want the verification to continue? If you wish, can you provide the WifiSettingsPortal that produces 21KB?

[Koxx3]

you have to go back the commit fc34ddd32269c950ea91c959faa42365bc89fd37 ;)

I tried to switch to AutoConnect v1.3.0 combing PageBuilder v1.5.0 in platformIO, but it doesn't work.

Library Manager: Installing hieromon/AutoConnect @ ^1.3.0
Warning! Could not find the package with 'hieromon/AutoConnect @ ^1.3.0' requirements for your system 'windows_amd64'
Library Manager: Installing hieromon/PageBuilder @ ^1.5.0
Warning! Could not find the package with 'hieromon/PageBuilder @ ^1.5.0' requirements for your system 'windows_amd64'

[Hieromon]

It has not been released yet, so it is before the registration of PlatformIO's global repository.
Clone the AutoConnect Repo locally and switch branches to use. You can choose a branch without polluting the locally built Arduino library folder.

[Hieromon]

I used the reduced code leaving the logic part related to AutoConnect based on the specified commit hash and tested it with AutoConnect v1.3.0 combing PageBuilder v1.5.0.
As a result, all of the various AutoConnectAux pages you have prepared had display successfully. The largest HTML page is the SmartElec settings, which is 21438 bytes in size.

[PB] Chunked, HEAD AUX_TITLE CSS_BASE CSS_UL blk:1270 CSS_INPUT_BUTTON CSS_INPUT_TEXT blk:1270 CSS_LUXBAR blk:1270 blk:1270 AUX_CSS blk:1270 MENU_PRE MENU_AUX MENU_POST blk:1270 ENC_TYPE AUX_ELEMENT [AC] CB in AHEAD /settingspage
portal where =
load begin
load end
blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 AUX_URI blk:1163

Chunked block (as size is 1270 bytes) count is 16, adding last block size as 1163 is 21483 bytes as total size.
In the next PageBuilder update turn, the internal string buffer size is fixed, and its version adopts the method of the streaming model for the HTML dynamic generation. Perhaps it can bear on more than 21KB.
AutoConnect v1.3.0 combing PageBuilder v1.5.0 will solve your problem.

[Koxx3]

excellent ! thanks A LOT for your tests 😉 Le mer. 31 mars 2021 à 11:11, Hieromon Ikasamo ***@***.***> a écrit :

…

I used the reduced code leaving the logic part related to AutoConnect based on the specified commit hash and tested it with AutoConnect v1.3.0 combing PageBuilder v1.5.0. As a result, all of the various AutoConnectAux pages you have prepared had display successfully. The largest HTML page is the SmartElec settings, which is 21438 bytes in size. [PB] Chunked, HEAD AUX_TITLE CSS_BASE CSS_UL blk:1270 CSS_INPUT_BUTTON CSS_INPUT_TEXT blk:1270 CSS_LUXBAR blk:1270 blk:1270 AUX_CSS blk:1270 MENU_PRE MENU_AUX MENU_POST blk:1270 ENC_TYPE AUX_ELEMENT [AC] CB in AHEAD /settingspage portal where = load begin load end blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 blk:1270 AUX_URI blk:1163 Chunked block (as size is 1270 bytes) count is 16, adding last block size as 1163 is 21483 bytes as total size. In the next PageBuilder update turn, the internal string buffer size is fixed, and HTML generation is used as the streaming output. Perhaps it can bear on more than 21KB. AutoConnect v1.3.0 combing PageBuilder v1.5.0 will solve your problem. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#342 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACXMP3ARSVE4EZ3E4QYJKPTTGLRNZANCNFSM4Z6MDJNQ> .

[Koxx3]

by the way, I tried the AC 1.3 / PB 1.5. not better in my case. I can't load the settings page the first time (but test conditions are not the same).

12:59:08.898 > RAM left = 29752 / HM idle = 6168 / HM tft = 7036 / HM wifi = 6376 / HM btn = 2472
12:59:10.259 > [AC] Host:www.google.com,/generate_204,ignored
12:59:10.266 > [E][WebServer.cpp:632] _handleRequest(): request handler not found
12:59:10.269 > [AC] Detected application, connectivitycheck.gstatic.com, 0.0.0.0
12:59:14.019 > [AC] Host:172.217.28.1,/settingspage,generated:/settingspage, allocated
12:59:14.037 > [AC] Host:172.217.28.1,/settingspage,already allocated
12:59:14.041 > [PB] Free heap:60768, content len.:unknown
12:59:14.041 > [PB] Res:200, Chunked:2
12:59:14.042 > [PB] Transfer-Encoding:chunked
12:59:14.061 > [AC] CB in AHEAD /settingspage
12:59:14.062 > portal where =
12:59:14.062 > load begin
12:59:14.068 > load end
12:59:14.197 > [PB] Failed building, free heap:50544
12:59:14.199 > [PB] at leaving build: 50572 free
12:59:14.300 > [AC] Host:172.217.28.1,/generate_204,ignored
12:59:14.310 > [E][WebServer.cpp:632] _handleRequest(): request handler not found
12:59:14.313 > [AC] Detected application, connectivitycheck.gstatic.com, 0.0.0.0
12:59:14.381 > [AC] Host:connectivitycheck.gstatic.com,/favicon.ico,ignored
12:59:14.403 > [E][WebServer.cpp:632] _handleRequest(): request handler not found
12:59:14.406 > [PB] at leaving build: 59940 free
12:59:14.480 > [AC] Host:172.217.28.1,/generate_204,ignored
12:59:14.490 > [E][WebServer.cpp:632] _handleRequest(): request handler not found
12:59:14.492 > [AC] Detected application, connectivitycheck.gstatic.com, 0.0.0.0

[Koxx3]

I realize I am short in memory.
Do you think I would gain if :
1/ I use a static mold in spiffs and tokens to include loaded values OR
2/ I use a static page and load data from a side json

[Hieromon]

I can't find the cause of running out of heap in PageBuilder even though the free heap size remains as much as 50Kb. I can continue to validate your code for a little. But that requires your help, which is to provide me with the minimum code that allows the reproduction of the problem with my environment. The code your project owns is too big for me and not suitable for focusing on AutoConnect issues. You need to reduce it to the smallest code that can reproduce the problem. Could you?

I suspect your code is doing something like a large heap consumption in the AutoConnectAux callback that induces a lot of heap consumption. It may be caused by the BT library, for example.
PageBuilder repeatedly and continuously calls the AutoConnectAux handler until the page generation is complete. If the AutoConnectAux handler temporarily consumes a large amount of heap, a phenomenon similar to this problem may occur. It will be a cause that is far from the mold size of the page.