🩺 AcuBench Leaderboard

What is this?

AcuBench benchmarks models that recommend an acupoint set for a clinical indication. This page only displays results — it does not run or score any model. Scoring happens offline: the operator holds the private test gold labels locally (built from AcuKG), scores each participant's predictions with eval.py / score_submission.py, and commits the resulting row as a JSON file under submissions/. Those rows are aggregated into leaderboard.json, which this page fetches and renders. No gold labels ever live in this Space, so it is safe to keep public.

How to submit

AcuBench uses operator-held gold: the private test split labels are never published, so they cannot be gamed.

1. Get the dataset (build script + skeleton, no AcuKG-derived labels) from the dataset repo: huggingface.co/datasets/<username>/acubench (replace <username> with the operator's HF username).
2. Produce predictions for the test indications as a JSONL file, one object per line, in the submission format:

   {"id": 12, "acupoints": ["LU1", "LU7"], "scores": {"LU1": 0.9, "LU7": 0.8}}

   acupoints drives the set metrics (Jaccard / F1) and the meridian-validity proxy; scores (optional, recommended) drives the ranking metrics (P@k / R@k / NDCG / PRAUC).
3. Send your predictions file to the operator (do not send gold labels — you should not have the private test gold).
4. The operator scores it with score_submission.py and commits the resulting row to submissions/; it appears here on the next rebuild.

Metrics: Jaccard, F1-micro, F1-macro, P@5, R@10, NDCG@10, PRAUC (higher is better). invalid% is a structural meridian-scatter proxy — lower is better — not a clinical-safety number. Rows tagged REF are the paper's reference baselines.