Cannot create a string longer than 0x1fffffe8 characters

[flythenimbus]

This is happening on this method call:

await dockerCompose.pullAll( dockerComposeConfig );

The full error looks like this:

⠧ Downloading sources.Container 3599aeb67a0e501419c5faae3c737c23-mysql-1  Started
✖ Cannot create a string longer than 0x1fffffe8 characters
Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:621:37)
    at Buffer.toString (node:buffer:812:14)
    at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:63:23) {
  code: 'ERR_STRING_TOO_LONG'
}

Can't help with the actual source because this is coming from a 3rd party library that I'm using. Right here:

https://github.com/WordPress/gutenberg/blob/trunk/packages/env/lib/commands/start.js#L115

Help would be appreciated.

[AlexZeitler]

0x1fffffe8 is 512 MB. Is this happening on a machine (maybe a CI process) which only provides this amount of RAM?

[Steveb-p]

Depending on the actual code we might be able to circumvent the issue by using Buffer.slice instead. But we would need to pinpoint the exact location where is happens in non-minified code.

Nonetheless, could you provide more context @dugajean ? We had reports related to Wordpress before, but overall it seemed to be working properly. Is there anything different in your application, that could be related?

[flythenimbus]

@AlexZeitler This is happening on my machine. I have 8 GB of RAM. This never happened on my other machine though, so it could be resources related. But then again, I think it should work on every machie.

@Steveb-p Like I mentioned above, this is coming from wp-env, but I'm on a slightly weaker laptop at this time, which is probably what's causing it. My specs look like this:

The exact line of code throwing is in wp-env: https://github.com/WordPress/gutenberg/blob/trunk/packages/env/lib/commands/start.js#L115

I believe that simply buffering would solve this issue rather than trying to read a gigantic string at once. Like you said. That's the most common suggestion for this error around the internet.

[flythenimbus]

Any news here? This is still happening sometimes, unfortunately.

[AlexZeitler]

@dugajean Which images do you try to pull here?
@Steveb-p Do you understand where this happens? Is this stdout/stderr buffer?

[flythenimbus]

@AlexZeitler Here's the docker-compose file

{
    "version": "3.7",
    "services": {
        "mysql": {
            "image": "mariadb",
            "ports": [
                "3306"
            ],
            "environment": {
                "MYSQL_ROOT_PASSWORD": "password",
                "MYSQL_DATABASE": "wordpress"
            },
            "volumes": [
                "mysql:/var/lib/mysql"
            ]
        },
        "tests-mysql": {
            "image": "mariadb",
            "ports": [
                "3306"
            ],
            "environment": {
                "MYSQL_ROOT_PASSWORD": "password",
                "MYSQL_DATABASE": "tests-wordpress"
            },
            "volumes": [
                "mysql-test:/var/lib/mysql"
            ]
        },
        "wordpress": {
            "build": ".",
            "depends_on": [
                "mysql"
            ],
            "image": "wordpress:php7.4",
            "ports": [
                "${WP_ENV_PORT:-8890}:80"
            ],
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "wordpress"
            },
            "volumes": [
                "/Users/project/local/wordpress:/var/www/html",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ]
        },
        "tests-wordpress": {
            "depends_on": [
                "tests-mysql"
            ],
            "image": "wordpress:php7.4",
            "ports": [
                "${WP_ENV_TESTS_PORT:-8891}:80"
            ],
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "tests-wordpress",
                "WORDPRESS_DB_HOST": "tests-mysql"
            },
            "volumes": [
                "/Users/username/.wp-env/feeccf209cc7c87fe55317936d452b23/tests-wordpress:/var/www/html",
                "/Users/project/local/wordpress/.htaccess:/var/www/html/.htaccess",
                "/Users/project/local/wordpress/index.php:/var/www/html/index.php",
                "/Users/project/local/wordpress/license.txt:/var/www/html/license.txt",
                "/Users/project/local/wordpress/mysql.sql:/var/www/html/mysql.sql",
                "/Users/project/local/wordpress/phpunit-wp-config.php:/var/www/html/phpunit-wp-config.php",
                "/Users/project/local/wordpress/readme.html:/var/www/html/readme.html",
                "/Users/project/local/wordpress/wp-activate.php:/var/www/html/wp-activate.php",
                "/Users/project/local/wordpress/wp-admin:/var/www/html/wp-admin",
                "/Users/project/local/wordpress/wp-blog-header.php:/var/www/html/wp-blog-header.php",
                "/Users/project/local/wordpress/wp-comments-post.php:/var/www/html/wp-comments-post.php",
                "/Users/project/local/wordpress/wp-cron.php:/var/www/html/wp-cron.php",
                "/Users/project/local/wordpress/wp-includes:/var/www/html/wp-includes",
                "/Users/project/local/wordpress/wp-links-opml.php:/var/www/html/wp-links-opml.php",
                "/Users/project/local/wordpress/wp-load.php:/var/www/html/wp-load.php",
                "/Users/project/local/wordpress/wp-login.php:/var/www/html/wp-login.php",
                "/Users/project/local/wordpress/wp-mail.php:/var/www/html/wp-mail.php",
                "/Users/project/local/wordpress/wp-settings.php:/var/www/html/wp-settings.php",
                "/Users/project/local/wordpress/wp-signup.php:/var/www/html/wp-signup.php",
                "/Users/project/local/wordpress/wp-trackback.php:/var/www/html/wp-trackback.php",
                "/Users/project/local/wordpress/xmlrpc.php:/var/www/html/xmlrpc.php",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ]
        },
        "cli": {
            "depends_on": [
                "wordpress"
            ],
            "image": "wordpress:cli-php7.4",
            "volumes": [
                "/Users/project/local/wordpress:/var/www/html",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ],
            "user": "33:33",
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "wordpress"
            }
        },
        "tests-cli": {
            "depends_on": [
                "tests-wordpress"
            ],
            "image": "wordpress:cli-php7.4",
            "volumes": [
                "/Users/dugi/.wp-env/feeccf209cc7c87fe55317936d452b23/tests-wordpress:/var/www/html",
                "/Users/project/local/wordpress/.htaccess:/var/www/html/.htaccess",
                "/Users/project/local/wordpress/index.php:/var/www/html/index.php",
                "/Users/project/local/wordpress/license.txt:/var/www/html/license.txt",
                "/Users/project/local/wordpress/mysql.sql:/var/www/html/mysql.sql",
                "/Users/project/local/wordpress/phpunit-wp-config.php:/var/www/html/phpunit-wp-config.php",
                "/Users/project/local/wordpress/readme.html:/var/www/html/readme.html",
                "/Users/project/local/wordpress/wp-activate.php:/var/www/html/wp-activate.php",
                "/Users/project/local/wordpress/wp-admin:/var/www/html/wp-admin",
                "/Users/project/local/wordpress/wp-blog-header.php:/var/www/html/wp-blog-header.php",
                "/Users/project/local/wordpress/wp-comments-post.php:/var/www/html/wp-comments-post.php",
                "/Users/project/local/wordpress/wp-cron.php:/var/www/html/wp-cron.php",
                "/Users/project/local/wordpress/wp-includes:/var/www/html/wp-includes",
                "/Users/project/local/wordpress/wp-links-opml.php:/var/www/html/wp-links-opml.php",
                "/Users/project/local/wordpress/wp-load.php:/var/www/html/wp-load.php",
                "/Users/project/local/wordpress/wp-login.php:/var/www/html/wp-login.php",
                "/Users/project/local/wordpress/wp-mail.php:/var/www/html/wp-mail.php",
                "/Users/project/local/wordpress/wp-settings.php:/var/www/html/wp-settings.php",
                "/Users/project/local/wordpress/wp-signup.php:/var/www/html/wp-signup.php",
                "/Users/project/local/wordpress/wp-trackback.php:/var/www/html/wp-trackback.php",
                "/Users/project/local/wordpress/xmlrpc.php:/var/www/html/xmlrpc.php",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ],
            "user": "33:33",
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "tests-wordpress",
                "WORDPRESS_DB_HOST": "tests-mysql"
            }
        },
        "composer": {
            "image": "composer",
            "volumes": [
                "/Users/project:/app"
            ]
        },
        "phpunit": {
            "image": "wordpressdevelop/phpunit:8-php-7.4-fpm",
            "depends_on": [
                "tests-wordpress"
            ],
            "volumes": [
                "/Users/dugi/.wp-env/feeccf209cc7c87fe55317936d452b23/tests-wordpress:/var/www/html",
                "/Users/project/local/wordpress/.htaccess:/var/www/html/.htaccess",
                "/Users/project/local/wordpress/index.php:/var/www/html/index.php",
                "/Users/project/local/wordpress/license.txt:/var/www/html/license.txt",
                "/Users/project/local/wordpress/mysql.sql:/var/www/html/mysql.sql",
                "/Users/project/local/wordpress/phpunit-wp-config.php:/var/www/html/phpunit-wp-config.php",
                "/Users/project/local/wordpress/readme.html:/var/www/html/readme.html",
                "/Users/project/local/wordpress/wp-activate.php:/var/www/html/wp-activate.php",
                "/Users/project/local/wordpress/wp-admin:/var/www/html/wp-admin",
                "/Users/project/local/wordpress/wp-blog-header.php:/var/www/html/wp-blog-header.php",
                "/Users/project/local/wordpress/wp-comments-post.php:/var/www/html/wp-comments-post.php",
                "/Users/project/local/wordpress/wp-cron.php:/var/www/html/wp-cron.php",
                "/Users/project/local/wordpress/wp-includes:/var/www/html/wp-includes",
                "/Users/project/local/wordpress/wp-links-opml.php:/var/www/html/wp-links-opml.php",
                "/Users/project/local/wordpress/wp-load.php:/var/www/html/wp-load.php",
                "/Users/project/local/wordpress/wp-login.php:/var/www/html/wp-login.php",
                "/Users/project/local/wordpress/wp-mail.php:/var/www/html/wp-mail.php",
                "/Users/project/local/wordpress/wp-settings.php:/var/www/html/wp-settings.php",
                "/Users/project/local/wordpress/wp-signup.php:/var/www/html/wp-signup.php",
                "/Users/project/local/wordpress/wp-trackback.php:/var/www/html/wp-trackback.php",
                "/Users/project/local/wordpress/xmlrpc.php:/var/www/html/xmlrpc.php",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ],
            "environment": {
                "LOCAL_DIR": "html",
                "WP_PHPUNIT__TESTS_CONFIG": "/var/www/html/phpunit-wp-config.php",
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "tests-wordpress",
                "WORDPRESS_DB_HOST": "tests-mysql"
            }
        }
    },
    "volumes": {
        "mysql": {},
        "mysql-test": {},
        "phpunit-uploads": {}
    }
}

[flythenimbus]

This is just a matter of the string becoming too large to process.

[Steveb-p]

As far as I know, we are not performing any pre-processing for the service declaration file (configuration). We pass the file name as-is as a parameter to the docker-compose binary - unless the config is provided as a string (using configAsString option).

So more likely culprit would be the output of a container. I'll need to check what we can do to store it in a different manner.

[flythenimbus]

@Steveb-p How's this looking?