EvalPort example notebook: importing/exporting Vertex AI Gen AI Evaluation datasets

[adhabnr-ux]

Hi team,

I maintain EvalPort, an open JSON Schema spec (Apache 2.0) for portable LLM evaluation datasets (test suites + result sets), with Python/TypeScript SDKs that validate against the schemas.

I went through a couple of the notebooks under gemini/evaluation/ (evaluate_with_your_python_code.ipynb, evaluating_observability_datasets.ipynb) to see how the Vertex AI Gen AI Evaluation SDK represents datasets. It's a clean pandas-based shape:

dataset = pd.DataFrame({"prompt": [...], "reference": [...], "response": [...]})
eval_dataset = types.EvaluationDataset(eval_dataset_df=dataset)
eval_result = client.evals.evaluate(dataset=eval_dataset, metrics=[...])

That maps cleanly onto an EvalPort EvalSuite/ResultSet pair — prompt → test_cases[].input, reference → test_cases[].expected_output, and the returned summary_metrics → ResultSet.results[].grader_results. A short notebook could show both directions, e.g.:

import pandas as pd

def suite_to_eval_dataset(suite: dict) -> pd.DataFrame:
    return pd.DataFrame({
        "prompt": [tc["input"] for tc in suite["test_cases"]],
        "reference": [tc.get("expected_output", "") for tc in suite["test_cases"]],
    })

# suite = json.load(open("my_suite.evalport.json"))
# eval_dataset = types.EvaluationDataset(eval_dataset_df=suite_to_eval_dataset(suite))
# eval_result = client.evals.evaluate(dataset=eval_dataset)

Since this repo is a samples/cookbook collection rather than the SDK itself, the ask here is modest: would an example notebook (something like gemini/evaluation/evalport_import_export.ipynb) showing this round-trip be a welcome addition? Happy to draft one following notebook_template.ipynb and scripts/format.sh if so — no pressure either way.

Spec: https://github.com/adhabnr-ux/evalport/blob/main/SPEC.md
Python SDK: pip install evalport-sdk