Skip to content

System Test: Nightly invocation for soroban test casesΒ #10

@sreuland

Description

@sreuland

What problem does your feature solve?

Currently, system-tests are invoked automatically only on open PR's in soroban-tools.

What would you like to see?

  • Nightly runs system test with a matrix of staggered component versions and latest main/master branch source versions.
  • Notification via slack channel when the nightly run fails.
  • Run tests against multiple soroban enabled target networks:
    1. Futurenet?
    2. Local(i.e. standalone with inlined compiled core from source tag version of rc release)
    3. Testnet(configured against the remote sdf hosted rpc instance)

End result would be a series of local system-test docker images built, which can then be run in series or parallel and will execute the test suite within each container, based on a permutation of desired versions, for example:

  • current_cli_version
  • last_cli_version
  • current_rpc_version
  • last_rpc_version

What alternatives are there?

Manually invoke the system tests on command line with targer package or source code versions:

docker run --platform linux/amd64 --rm -it "stellar/system-test" --SorobanCLICrateVersion "0.4.0" --SorobanRPCDebianVersion "0.4.0-10" --SorobanExamplesGitHash "v0.4.1" --SorobanExamplesRepoURL "https://github.com/stellar/soroban-examples.git" --RustToolchainVersion "1.66.0" --CoreDebianVersion "19.6.1-1158.c0ad35aa1.focal~soroban" --HorizonDebianVersion "2.22.0~soroban-323" TestFilter "^TestDappDevelop$/^.*$" --VerboseOutput true 

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    To Do

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions