Methods overview
This page summarizes the public-safe benchmark methods. It intentionally omits local commands, local paths, row-level data, treatment identifiers, target sequences, and unpublished model results.
Objective
Perturb-LM evaluates whether biological language can retrieve perturbation-induced Cell Painting morphology under leakage-aware, perturbation-level evaluation.
Dataset inventory
The primary benchmark uses 4,524 CPJUMP1 morphology profiles in a consistent 904-feature feature space. A larger compatibility investigation is not the primary modeling dataset.
Identifier policy
Target sequences and direct treatment identifiers are prohibited from identifier-stripped query and candidate text. The primary lexical control uses gene, perturbation type, control type, and negative-control type only.
Controls
The benchmark reports full-metadata TF-IDF as an identifier-dominated reference, identifier-stripped TF-IDF as the main lexical control, plus random and shuffled-label controls.
Evaluation
Metrics are reported at the perturbation level with total query counts, evaluable query counts, Hit@K, Recall@K, mAP, enrichment over random, and query-bootstrap uncertainty.
Next experiment
The planned model test freezes a biomedical text encoder and trains a lightweight projection into the morphology-profile space. Learned model results are pending.
Current benchmark snapshot
- Profiles
- 4,524
- Morphology features
- 904
- Full benchmark queries
- 641
- Model status
- pending