TML 2026 — Assignment 1: Membership Inference Attack

Team: team_XXVI | atml_team024

How to Recreate the Best Leaderboard Result (Score: 0.058860)

Step 1 — Connect to cluster (requires university VPN)

ssh atml_team024@conduit2.hpc.uni-saarland.de

Step 2 — Download dataset files

mkdir ~/tml26_task1 && cd ~/tml26_task1
wget "https://huggingface.co/datasets/SprintML/tml26_task1/resolve/main/pub.pt"
wget "https://huggingface.co/datasets/SprintML/tml26_task1/resolve/main/priv.pt"
wget "https://huggingface.co/datasets/SprintML/tml26_task1/resolve/main/model.pt"

Step 3 — Upload scripts to cluster

scp mia_attack.py atml_team024@conduit2.hpc.uni-saarland.de:~/tml26_task1/
scp run.sh atml_team024@conduit2.hpc.uni-saarland.de:~/tml26_task1/
scp mia.sub atml_team024@conduit2.hpc.uni-saarland.de:~/tml26_task1/
scp submit.py atml_team024@conduit2.hpc.uni-saarland.de:~/tml26_task1/

Step 4 — Submit job to cluster

cd ~/tml26_task1
chmod +x run.sh
condor_submit mia.sub

Step 5 — Monitor job

watch condor_q

Wait until job disappears, then check:

ls -la ~/tml26_task1/submission.csv

Step 6 — Submit to leaderboard

python3 ~/tml26_task1/submit.py

Method Summary

• 7 complementary memorization signals (loss, confidence, entropy, logit gap, correct-class logit, rank, LiRA score)
• Per-class z-score normalization to remove class-difficulty bias
• LiRA-inspired per-class likelihood ratio scoring
• Augmentation-based loss smoothing (N=20 passes per image)
• Logistic regression meta-classifier (C=0.1, class-balanced)

Best Result

• Score: 0.058860 (TPR@5%FPR)
• Best Score: 0.058860 (TPR@5%FPR) on public leaderboard
• Team: team_XXVI