Discussion about this post

User's avatar
Nicky's avatar

Amazing paper, super motivating read :)

About your question: I'd put my money on the right grid being the real one (haha, fingers crossed). My reasoning comes from your own framing: a perturbation response is not concentrated on a single outcome, it can be multimodal, heavy-tailed, and contain rare but important events. To me, the right grid seems to capture more of that tail behavior. I could totally be wrong, though; I'm basically coin-flipping.

On metrics, I agree with you about FID/KID. They compare samples in a frozen ImageNet embedding space and essentially ask, "Do these look like real cells overall?" That's why even an unconditional model can score well, as you showed.

Two things I think could help:

1. Compare samples in an embedding space that actually understands cells rather than ImageNet features.

2. In a perturbation setting, I think of each condition as a cloud of cells: control is one cloud, the real drug shifts it to another, and the model predicts a third. What really matters is whether the predicted cloud lands on the real perturbed one. I'd therefore evaluate that shift directly using distributional metrics such as Energy Distance or MMD, along with a discrimination test: rank the real drugs by distance to the prediction and see whether the true drug comes out #1.

That second idea also captures a failure mode that FID might miss. A model that simply outputs the "average treated cell" may look realistic and achieve a great FID score, yet predict essentially the same response for every drug. In that case, discrimination performance would collapse to a coin flip, which gets at the core question of perturbation modeling: not just whether the images look realistic, but whether the correct response is associated with the correct perturbation.

And the closing line "what do we still not understand well enough to simulate?" really stuck with me. Shifting the bottleneck from lab throughput to model fidelity feels like it could genuinely expand the drug-discovery design space. Excited to see where these trends go.

2 more comments...

No posts

Ready for more?