[Questions] Which image or latent code to use for generation ? How to assess the efficiency of the pipeline ?

[HugoHakem]

Improving generation process of counterfactual

Right now, Image are generated using a randomly sample latent code or reference image.

Is this the expected way to proceed? Could we imagine a smarter generation?

Why does that matter?
Because our pipeline depends on the quality of the counterfactual generated

Idea of features:

1. We could generate the subspace of the mapping networks or of the style encoder.
   And then check for the image generated when selecting different latent code into that embedding space and see how much does that differ.
2. We could implement another marimo interface for efficient visualization.

Assessing the efficiency of the pipeline

1. We could try to have a specific use case where we know exactly what does a perturbation and try to see if the mask retrieve that specific modification.
2. We could use the classic pipeline of assessing the most important cell profiler features that should differentiate two perturbation. And try to see if we can better interpret them using the mask retrieved.
3. We could try using the python cell profiler on mask itself (can this even work ?) and try to see if the embedding retrieve can help explain what the mask should point as specific organels in cells or else.
4. Other information would be interesting: Can we assess the batch effect of real images vs counterfactual image. Maybe counterfactual average in some ways the batch effect, or maybe batch effect is till present (which might no be such a bad a news in a sense that counterfactual cell would really be close in the feature space of real image, hence displaying batch effect as well (put is simply: at least, only biology is transferred and batch effect remain preserved, good news because the stargan transfer biology!).

[HugoHakem]

To chose your reference image per class:

• earth mover's distance computed on every pair of reference image within one class and you choose the centroid.
• you put all of your image into the style encoder and you take the centroid of each cluster as a reference.

[HugoHakem]

On marimo you will plot the real image DMSO and your counterfactual CDK for example.
You compute with cp measure and cellpose, the cell profiler feature on real set of image. Then get the median.
And then do the cell profiler on counterfactual and comparing the embedding you are having with the median embedding.