@@ -82,27 +82,75 @@ <h2>Benchmark</h2>
8282 The released benchmark covers ALD and ALE papers across experimental
8383 and simulation use cases. It provides train, dev, and gold-standard
8484 test splits with figure images, captions, bounding boxes, and
85- task-specific annotations.
85+ task-specific annotations. Task definitions are summarized from the
86+ < a href ="https://sites.google.com/view/sci-imageminer/task-description "> official competition task description</ a > .
8687 </ p >
88+ < div class ="benchmark-figures " aria-label ="Benchmark figures ">
89+ < figure >
90+ < img
91+ src ="assets/fig_annotation-workflow-v2.png "
92+ alt ="Overview of the ALD/E-ImageMiner benchmark curation workflow "
93+ />
94+ < figcaption >
95+ Fig. 1: Overview of the ALD/E-ImageMiner benchmark end-to-end curation workflow.
96+ </ figcaption >
97+ </ figure >
98+ < figure >
99+ < img
100+ src ="assets/fig_dataset_organization_hierarchy_v2.png "
101+ alt ="ALD/E-ImageMiner dataset organization hierarchy "
102+ />
103+ < figcaption >
104+ Fig. 2: Illustration describing the ALD/E-ImageMiner dataset organization hierarchy.
105+ Research papers are numerically indexed, containing the PDF,
106+ < code > content.json</ code > (sections and figure captions), along with
107+ figures and annotation JSON files inside the < code > images</ code > directory.
108+ </ figcaption >
109+ </ figure >
110+ </ div >
87111 < div class ="task-grid ">
88112 < article >
89113 < h3 > Classification</ h3 >
90- < p > Predict the scientific figure type from the taxonomy. Primary metric: F1.</ p >
114+ < p >
115+ Supervised multi-class image classification. Systems receive a
116+ scientific figure, optionally with caption context, and predict one
117+ of the 49 figure types.
118+ </ p >
119+ < p class ="task-meta "> Primary metric: F1.</ p >
91120 < a href ="https://www.codabench.org/competitions/12901/#/results-tab "> Submit / results</ a >
92121 </ article >
93122 < article >
94123 < h3 > Data extraction</ h3 >
95- < p > Reconstruct chart data as Markdown tables. Primary metric: RMS + TEDS.</ p >
124+ < p >
125+ Structured reconstruction of tabular data encoded in scientific
126+ charts. Systems identify field labels, table structure, and textual
127+ or numeric cell values.
128+ </ p >
129+ < p class ="task-meta "> Output: machine-readable Markdown table. Metrics: RMS + TEDS.</ p >
96130 < a href ="https://www.codabench.org/competitions/12902/#/results-tab "> Submit / results</ a >
97131 </ article >
98132 < article >
99133 < h3 > Summarization</ h3 >
100- < p > Generate concise figure summaries grounded in scientific content.</ p >
134+ < p >
135+ Generate concise, factual summaries that capture key trends,
136+ relationships, and scientific insights grounded in the visual
137+ content, with caption context when available.
138+ </ p >
139+ < p class ="task-meta "> Output: 1-3 sentence textual summary.</ p >
101140 < a href ="https://www.codabench.org/competitions/12909/#/results-tab "> Submit / results</ a >
102141 </ article >
103142 < article >
104143 < h3 > Visual QA</ h3 >
105- < p > Answer domain-grounded questions with factoid, yes/no, list, and paragraph outputs.</ p >
144+ < p >
145+ Fine-grained reasoning over figures through natural-language
146+ questions about axes, legends, data patterns, and scientific
147+ meaning.
148+ </ p >
149+ < p class ="task-meta ">
150+ Covers process-oriented, comparative/trend, structure-property,
151+ and application/performance questions; outputs yes/no, factoid,
152+ list, or paragraph answers.
153+ </ p >
106154 < a href ="https://www.codabench.org/competitions/12908/#/results-tab "> Submit / results</ a >
107155 </ article >
108156 </ div >
0 commit comments