Skip to content

bulk-disseminate Actions will fail if more than 256 submission candidates are found #22

@rhigman

Description

@rhigman

"A matrix will generate a maximum of 256 jobs per workflow run"

If the run-script stage of any bulk-disseminate Action finds more than 256 works which need to be disseminated based on the Action's inclusion criteria, the run will fail and none of the works will be disseminated. This is because the list of relevant work IDs is passed to a GitHub Actions matrix to create one submission job per work.

Additionally, due to the fact that the list of work IDs found by run-script is not output anywhere (e.g. in logs or artifacts), it's not possible to make up for the failure by disseminating the required works in a more manual fashion.

At present scale, this will rarely be an issue; in most cases, it would require the total number of works produced in a month by all publishers opted-in to a specific dissemination platform to exceed 256 (the current average is closer to 10). However, this did occur in a run of cr-bulk-disseminate, after a bulk database update caused ~6000 works to show up as having been updatedWithRelations within a single dissemination period.

(The update itself wasn't actually one which warranted mass resubmission of metadata anyway; we should consider how to handle similar future updates without unnecessarily overloading distribution platforms.)

Failure report:

image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions