Skip to content

Commit 4235d5b

Browse files
D'SouzaD'Souza
authored andcommitted
Expand benchmark section on paper page
1 parent 9f0b68b commit 4235d5b

4 files changed

Lines changed: 100 additions & 7 deletions

File tree

513 KB
Loading
500 KB
Loading

docs/css/styles.css

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -173,14 +173,54 @@ a {
173173
text-align: center;
174174
}
175175

176-
.task-grid,
176+
.benchmark-figures {
177+
display: grid;
178+
gap: 24px;
179+
margin: 28px 0 30px;
180+
}
181+
182+
.benchmark-figures figure {
183+
margin: 0;
184+
}
185+
186+
.benchmark-figures img {
187+
display: block;
188+
width: 100%;
189+
height: auto;
190+
border: 1px solid var(--line);
191+
border-radius: 4px;
192+
}
193+
194+
.benchmark-figures figcaption {
195+
max-width: 900px;
196+
margin: 10px auto 0;
197+
color: var(--muted);
198+
font-size: 0.9rem;
199+
text-align: center;
200+
}
201+
202+
.benchmark-figures code {
203+
color: var(--ink);
204+
}
205+
206+
.task-grid {
207+
display: grid;
208+
grid-template-columns: repeat(2, minmax(0, 1fr));
209+
gap: 14px;
210+
margin: 22px auto 0;
211+
max-width: 900px;
212+
}
213+
177214
.result-grid {
178215
display: grid;
179-
grid-template-columns: repeat(4, minmax(0, 1fr));
180216
gap: 12px;
181217
margin-top: 22px;
182218
}
183219

220+
.result-grid {
221+
grid-template-columns: repeat(4, minmax(0, 1fr));
222+
}
223+
184224
.task-grid article,
185225
.result-card {
186226
border: 1px solid var(--line);
@@ -201,6 +241,11 @@ a {
201241
font-size: 0.93rem;
202242
}
203243

244+
.task-grid .task-meta {
245+
color: var(--ink);
246+
font-size: 0.9rem;
247+
}
248+
204249
.task-grid a,
205250
.result-card a {
206251
color: var(--accent);

docs/index.html

Lines changed: 53 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -82,27 +82,75 @@ <h2>Benchmark</h2>
8282
The released benchmark covers ALD and ALE papers across experimental
8383
and simulation use cases. It provides train, dev, and gold-standard
8484
test splits with figure images, captions, bounding boxes, and
85-
task-specific annotations.
85+
task-specific annotations. Task definitions are summarized from the
86+
<a href="https://sites.google.com/view/sci-imageminer/task-description">official competition task description</a>.
8687
</p>
88+
<div class="benchmark-figures" aria-label="Benchmark figures">
89+
<figure>
90+
<img
91+
src="assets/fig_annotation-workflow-v2.png"
92+
alt="Overview of the ALD/E-ImageMiner benchmark curation workflow"
93+
/>
94+
<figcaption>
95+
Fig. 1: Overview of the ALD/E-ImageMiner benchmark end-to-end curation workflow.
96+
</figcaption>
97+
</figure>
98+
<figure>
99+
<img
100+
src="assets/fig_dataset_organization_hierarchy_v2.png"
101+
alt="ALD/E-ImageMiner dataset organization hierarchy"
102+
/>
103+
<figcaption>
104+
Fig. 2: Illustration describing the ALD/E-ImageMiner dataset organization hierarchy.
105+
Research papers are numerically indexed, containing the PDF,
106+
<code>content.json</code> (sections and figure captions), along with
107+
figures and annotation JSON files inside the <code>images</code> directory.
108+
</figcaption>
109+
</figure>
110+
</div>
87111
<div class="task-grid">
88112
<article>
89113
<h3>Classification</h3>
90-
<p>Predict the scientific figure type from the taxonomy. Primary metric: F1.</p>
114+
<p>
115+
Supervised multi-class image classification. Systems receive a
116+
scientific figure, optionally with caption context, and predict one
117+
of the 49 figure types.
118+
</p>
119+
<p class="task-meta">Primary metric: F1.</p>
91120
<a href="https://www.codabench.org/competitions/12901/#/results-tab">Submit / results</a>
92121
</article>
93122
<article>
94123
<h3>Data extraction</h3>
95-
<p>Reconstruct chart data as Markdown tables. Primary metric: RMS + TEDS.</p>
124+
<p>
125+
Structured reconstruction of tabular data encoded in scientific
126+
charts. Systems identify field labels, table structure, and textual
127+
or numeric cell values.
128+
</p>
129+
<p class="task-meta">Output: machine-readable Markdown table. Metrics: RMS + TEDS.</p>
96130
<a href="https://www.codabench.org/competitions/12902/#/results-tab">Submit / results</a>
97131
</article>
98132
<article>
99133
<h3>Summarization</h3>
100-
<p>Generate concise figure summaries grounded in scientific content.</p>
134+
<p>
135+
Generate concise, factual summaries that capture key trends,
136+
relationships, and scientific insights grounded in the visual
137+
content, with caption context when available.
138+
</p>
139+
<p class="task-meta">Output: 1-3 sentence textual summary.</p>
101140
<a href="https://www.codabench.org/competitions/12909/#/results-tab">Submit / results</a>
102141
</article>
103142
<article>
104143
<h3>Visual QA</h3>
105-
<p>Answer domain-grounded questions with factoid, yes/no, list, and paragraph outputs.</p>
144+
<p>
145+
Fine-grained reasoning over figures through natural-language
146+
questions about axes, legends, data patterns, and scientific
147+
meaning.
148+
</p>
149+
<p class="task-meta">
150+
Covers process-oriented, comparative/trend, structure-property,
151+
and application/performance questions; outputs yes/no, factoid,
152+
list, or paragraph answers.
153+
</p>
106154
<a href="https://www.codabench.org/competitions/12908/#/results-tab">Submit / results</a>
107155
</article>
108156
</div>

0 commit comments

Comments
 (0)