Discussion about this post

User's avatar
Adam Green's avatar

Good post -- the within-target Spearman values in Fig 4 look like the real result. Question on the Fig 5 calibration, though. In four of the five panels the points seem to fan away from the diagonal as KDapp increases rather than sitting parallel to it, which would suggest a slope below 1 rather than a pure offset. Target E looks like the clearest case: the points span roughly three logs on x and well under one log on y. Have you fit slopes per target, and do they come out near 1? Worth asking because r and rho are both slope-blind -- E gets rho=0.90 either way -- so nothing in the reported statistics would flag it.

If the slope is materially below 1, an intercept-only correction aligns near the anchor and drifts away from it in both directions, which makes the choice of anchor matter a lot. So, relatedly: how were the five calibration points chosen, and have you run the leave-one-out version, recalculating the offset from each candidate anchor and reporting how the 28/32 count moves? The per-point offsets within a target look wide from the panels -- Target D spans something like two logs across its eleven measurements.

Practically: when a partner runs this on a new target, how many paired BLI measurements do they actually need before the offset stabilizes? One, or is that a demonstration convenience and the real answer closer to three to five?

One suggestion, offered in the spirit of it being the same analysis you're already doing: a hierarchical model with a random intercept per target would give you standard errors on each offset rather than point estimates, and would shrink the small-n targets toward the global mean instead of estimating E's offset from five points. Adding a random slope would answer the first question directly. It also handles the partner case cleanly -- "I've measured two binders on a new target, how much should I trust the offset" is a posterior for a new group, with honest uncertainty attached. With five targets the variance components would be wobbly, but that seems worth knowing too.

3 more comments...

No posts

Ready for more?