Image registration can bring two or more images into spatial correspondence. Researchers try their utmost to make or adjust images as as to make them correspond exactly.

Introduction

Image registration is the process of transforming different sets of data into one coordinate system. Data may be multiple photographs, data from different sensors, times, depths, or viewpoints. Registration is necessary in order to be able to compare or integrate the data obtained from these different measurements.

Registration is an optimization problem.

Questions

• What type of information to use?

Intrinsic information or extrinsic information

• What type of similarity measure to use?

Correlation, sum of differences, mutual information

• What type of transformation to use?

Rigid, affine, nonrigid image transformation

• What type of interpolation to use?

Nearest-neighbour, linear, cubic interpolation

• How do we quantify similarity?

Normalized cross-correlation, maximize

Sum of squared differences, minimize

Mutual image information. maximize

Method

Constrain the problem as much as you can. The more degree of freedom you have in your algorithm, the more likely your are gonna to make mistakes.

Correlation in multimodality imaging

Mutual information

• Mutual information in overlapping regions

$$\text{MI} = E(I_1) + E(T(I_2)) - E(I_1, T(I_2))$$

Maximize

$$E(I) = \sum P(i) \text{log} (\frac{1}{P(i)})$$

• Shannon entropy as measure of uncertainty

Transformations

Interpolations

Applications

Inter/intra-subject brain registration and analysis.

Correcting for global motion to study local dynamics.

Typical algorithms used in the intensity-based registration methodologies.

A typical feature-based registration algorithm.

Matching and registration of two brain slices. At the top, fixed image overlapped by the contour segmented from the corpus callosum, moving image overlapped by the contour segmented from the corpus callosum, and the illustration of the matching established.

At the bottom,. Input images overlapped before the registration, the same images overlapped after the registration and the difference between the input images after the registration.

Optimize its parameters given a set of images from a collection of interest.

Weakly supervised

The upper part illustrates the training strategy of the proposed weakly-supervised registration framework. Make use of anatomical labels.

Overlap and displacement prediction for a toy example with translation.

Reference

[1] Wiki, Image registration

[2] Erik Meijering, Basics of Image Processing: Image Registration

[3] Oliveira, F.P. and Tavares, J.M.R., 2014. Medical image registration: a review. Computer methods in biomechanics and biomedical engineering, 17(2), pp.73-93.

[4] Hu, Y., Modat, M., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C.M., Emberton, M. and Ourselin, S., 2018. Weakly-supervised convolutional neural networks for multimodal image registration. Medical image analysis, 49, pp.1-13.

[5] Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J. and Dalca, A.V., 2019. VoxelMorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8), pp.1788-1800.

[6] Fu, Y., Lei, Y., Wang, T., Curran, W.J., Liu, T. and Yang, X., 2020. Deep learning in medical image registration: a review. Physics in Medicine & Biology, 65(20), p.20TR01.

[7] De Silva, T., Chew, E.Y., Hotaling, N. and Cukras, C.A., 2021. Deep-learning based multi-modal retinal image registration for the longitudinal analysis of patients with age-related macular degeneration. Biomedical Optics Express, 12(1), pp.619-636.