Notes for running CrystFEL's `detector-shift`

[apeck12]

The refine_center task should be replaced by CrystFEL's detector-shift command for the sake of speed. The latter can be run from the command line as follows:

detector-shift {root_dir}/index/{tag}.stream {root_dir}/geom/r0XXX.geom
mv {root_dir}/geom/r0XXX.geom-predrefine.geom {root_dir}/geom/r0XXX.geom

to overwrite the old geometry file. The relevant runs will then need to be reindexed.

Some code for inspecting the per-run variation in detector offsets and unit cell lengths after performing indexing is below:

# extract parameters from stream files
metrics = {}
keys = ['det_shift_x', 'det_shift_y', 'lengths']
for key in keys:
    metrics[key] = {}

for run in run_range:
    print(f"On run {run}")
    fname = os.path.join(f"{root_dir}/index", f"r{run:04}_{tag}.stream")
    try:
        st = StreamInterface([fname], cell_only=True)
        metrics['det_shift_x'][run] = st.stream_data['det_shift_x']
        metrics['det_shift_y'][run] = st.stream_data['det_shift_y']    
        metrics['lengths'][run] = np.vstack([st.stream_data[length] for length in ['a','b','c']])
    except:
        print(f"could not process {fname}")

# plot the detector shift offsets
f, (ax1,ax2) = plt.subplots(1,2,figsize=(9,3))
for ax,key in zip([ax1,ax2],['det_shift_x','det_shift_y']):
    metric = np.hstack([metrics[key][run] for run in metrics[key].keys()])
    run_num = np.hstack([len(metrics[key][run])*[run] for run in metrics[key].keys()])
    ax.hexbin(run_num, metric, mincnt=1)
    ax.set_xlabel("Run")
ax1.set_ylabel("Offset (mm)")
ax1.set_title("Detector shift x")
ax2.set_title("Detector shift y")

# plot the cell lengths; here we assume cell-free indexing so provide expected cell parameters to focus on the valid cells
f, (ax1,ax2,ax3) = plt.subplots(1,3,figsize=(12,3))
for i,ax in enumerate([ax1,ax2,ax3]):
    metric = np.hstack([metrics['lengths'][key][i] for key in metrics['lengths'].keys()])
    run_num = np.hstack([len(metrics['lengths'][key][i])*[key] for key in metrics['lengths'].keys()])
    valid = (np.abs(metric- expected_cell[i])<5)
    ax.hexbin(run_num[valid], metric[valid], mincnt=1)
    ax.set_xlabel("Run")
ax1.set_ylabel("Length ($\mathrm{\AA}$)")
ax1.set_title("a")
ax2.set_title("b")
ax3.set_title("c")

The above will generate plots like the below: