Right now, after you submit an example, the label selection randomly changes: (e.g. "contradictory" has a 1/3 chance of changing to neutral, etc.). See here: https://dynabench.org/tasks/nli/create
We want the label to be randomly initialized after switching to a new context, but we don't want it to be randomly initialized after every example submission. The label button is defined here:
https://github.com/facebookresearch/dynabench/blob/main/frontends/web/src/common/Annotation/AnnotationComponent.js#L67
Right now, after you submit an example, the label selection randomly changes: (e.g. "contradictory" has a 1/3 chance of changing to neutral, etc.). See here: https://dynabench.org/tasks/nli/create
We want the label to be randomly initialized after switching to a new context, but we don't want it to be randomly initialized after every example submission. The label button is defined here:
https://github.com/facebookresearch/dynabench/blob/main/frontends/web/src/common/Annotation/AnnotationComponent.js#L67