Skip to content
This repository was archived by the owner on Nov 1, 2024. It is now read-only.

Give task owners the ability to choose default setting for getting random contexts #902

Description

@maxbartolo

The current default method for getRandomContext() is min, however, there is no way for task owners to change this currently.

Ideally this should be controlled from the task config (e.g. https://dynabench.org/task-owner-interface/qa#advanced) similar to aggregation_metric.

We would need to modify: https://github.com/facebookresearch/dynabench/blob/main/frontends/web/src/common/Annotation/CreateInterface.js#L95.

I suspect the steps required to implement are:

  1. Move the taskConfig loading slightly earlier
  2. Process the new taskConfig.ATTRIBUTE_NAME attribute accordingly with a default value of min (not sure if we need to make any other changes to make it editable)
  3. Change .getRandomContext(this.state.task.id, this.state.task.cur_round) to .getRandomContext(this.state.task.id, this.state.task.cur_round, [], taskConfig.ATTRIBUTE_NAME)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions