Self-inspired learning for denoising live-cell super-resolution microscopy
Schermelleh, L. et al. Super-resolution microscopy demystified. Nat. Cell Biol. 21, 72–84 (2019).
Google Scholar
Schermelleh, L. et al. Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy. Science 320, 1332–1336 (2008).
Google Scholar
Lawo, S., Hasegan, M., Gupta, G. D. & Pelletier, L. Subdiffraction imaging of centrosomes reveals higher-order organizational features of pericentriolar material. Nat. Cell Biol. 14, 1148–1158 (2012).
Google Scholar
Szymborska, A. et al. Nuclear pore scaffold structure analyzed by super-resolution microscopy and particle averaging. Science 341, 655–658 (2013).
Google Scholar
Xu, K., Zhong, G. & Zhuang, X. Actin, spectrin, and associated proteins form a periodic cytoskeletal structure in axons. Science 339, 452–456 (2013).
Google Scholar
Mishin, A. & Lukyanov, K. Live-cell super-resolution fluorescence microscopy. Biochemistry 84, 19–31 (2019).
Google Scholar
Godin, A. G., Lounis, B. & Cognet, L. Super-resolution microscopy approaches for live cell imaging. Biophys. J. 107, 1777–1784 (2014).
Google Scholar
Valli, J. et al. Seeing beyond the limit: a guide to choosing the right super-resolution microscopy technique. J. Biol. Chem. 297, 100791 (2021).
Google Scholar
Luisier, F., Vonesch, C., Blu, T. & Unser, M. Fast interscale wavelet denoising of Poisson-corrupted images. Signal Process. 90, 415–427 (2010).
Google Scholar
Huang, X. et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy. Nat. Biotechnol. 36, 451–459 (2018).
Google Scholar
Mandracchia, B. et al. Fast and accurate sCMOS noise correction for fluorescence microscopy. Nat. Commun. 11, 94 (2020).
Google Scholar
Weisong, Z. et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy. Nat. Biotechnol. 40, 606–617 (2022).
Google Scholar
Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).
Google Scholar
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
Google Scholar
Chen, J. et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat. Methods 18, 678–687 (2021).
Google Scholar
Fang, L. et al. Deep learning-based point-scanning super-resolution imaging. Nat. Methods 18, 406–416 (2021).
Google Scholar
Lehtinen, J. et al. Noise2Noise: learning image restoration without clean data. In Proceedings of the 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 2965–2974 (PMLR, 2018).
Batson, J. & Royer, L. Noise2self: blind denoising by self-supervision. In Proceedings of the 36th International Conference on Machine Learning (eds Chaudhuri, K. & Salakhutdinov, R.) 524–533 (PMLR, 2019).
Krull, A., Buchholz, T. -O. & Jug, F. Noise2void—learning denoising from single noisy images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2129–2137 (2019).
Eom, M. et al. Statistically unbiased prediction enables accurate denoising of voltage imaging data. Nat. Methods 20, 1581–1592 (2023).
Google Scholar
Zhang, G. et al. Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy. Nat. Methods 20, 1957–1970 (2023).
Google Scholar
Li, X. et al. Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising. Nat. Methods 18, 1395–1400 (2021).
Google Scholar
Lecoq, J. et al. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat. Methods 18, 1401–1408 (2021).
Google Scholar
Qiao, C. et al. Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes. Nat. Biotechnol. 41, 367–377 (2023).
Google Scholar
Muller, C. B. & Enderlein, J. Image scanning microscopy. Phys. Rev. Lett. 104, 198101 (2010).
Google Scholar
Hayashi, S. & Okada, Y. Ultrafast superresolution fluorescence imaging with spinning disk confocal microscope optics. Mol. Biol. Cell 26, 1743–1751 (2015).
Google Scholar
Hell, S. W. & Wichmann, J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. Opt. Lett. 19, 780–782 (1994).
Google Scholar
Vicidomini, G. et al. Sharper low-power STED nanoscopy by time gating. Nat. Methods 8, 571–573 (2011).
Google Scholar
Sun, D.-E. et al. Click-ExM enables expansion microscopy for all biomolecules. Nat. Methods 18, 107–113 (2021).
Google Scholar
Dertinger, T., Colyer, R., Iyer, G., Weiss, S. & Enderlein, J. Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI). Proc. Natl Acad. Sci. USA 106, 22287–22292 (2009).
Google Scholar
Zhao, W. et al. Enhanced detection of fluorescence fluctuations for high-throughput super-resolution imaging. Nat. Photonics 17, 806–813 (2023).
Google Scholar
Born, M. & Wolf, E. Principles of Optics, 7th Edn (Cambridge University Press, 1999).
Lequyer, J., Philip, R., Sharma, A., Hsu, W. -H. & Pelletier, L. A fast blind zero-shot denoiser. Nat. Mach. Intell. 4, 953–963 (2022).
Google Scholar
Li, X. et al. Spatial redundancy transformer for self-supervised fluorescence image denoising. Nat. Comput. Sci. 3, 1067–1080 (2023).
Google Scholar
Chen, X. et al. Self-supervised denoising for multimodal structured illumination microscopy enables long-term super-resolution live-cell imaging. PhotoniX 5, 1–22 (2024).
Google Scholar
Stein, S. C., Huss, A., Hähnel, D., Gregor, I. & Enderlein, J. Fourier interpolation stochastic optical fluctuation imaging. Opt. Express 23, 16154–16163 (2015).
Google Scholar
Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 234–241 (2015).
Yun, S. et al. Cutmix: regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF Conference on ICCV, 6023–6032 (2019).
Nieuwenhuizen, R. P. et al. Measuring image resolution in optical nanoscopy. Nat. Methods 10, 557–562 (2013).
Google Scholar
Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004).
Google Scholar
Lakshminarayanan, B., Pritzel, A. & Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. Adv. Neural Inf. Process. Syst. 30, 6402–6413 (2017).
Prakash, M., Lalit, M., Tomancak, P., Krul, A. & Jug, F. Fully unsupervised probabilistic Noise2Void. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), 154–158 (2020).
Quan, Y., Chen, M., Pang, T. & Ji, H. Self2self with dropout: learning self-supervised denoising from single image. In Proceedings of the IEEE/CVF Conference on CVPR, 1890–1898 (2020).
Pang, T., Zheng, H., Quan, Y. & Ji, H. Recorrupted-to-recorrupted: unsupervised deep learning for image denoising. In Proceedings of the IEEE/CVF Conference on CVPR, 2043–2052 (2021).
Richardson, W. H. Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 (1972).
Google Scholar
Lucy, L. B. An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974).
Google Scholar
Guo, Y. et al. Visualizing intracellular organelle and cytoskeletal interactions at nanoscale resolution on millisecond timescales. Cell 175, 1430–1442 (2018).
Google Scholar
Lu, M. et al. The structure and global distribution of the endoplasmic reticulum network are actively regulated by lysosomes. Sci. Adv. 6, eabc7209 (2020).
Google Scholar
Lu, M. et al. ERnet: a tool for the semantic segmentation and quantitative analysis of endoplasmic reticulum topology. Nat. Methods 20, 569–579 (2023).
Google Scholar
Sekh, A. A. et al. Physics-based machine learning for subcellular segmentation in living cells. Nat. Mach. Intell. 3, 1071–1080 (2021).
Google Scholar
Harke, B. et al. Resolution scaling in STED microscopy. Opt. Express 16, 4154–4162 (2008).
Google Scholar
Tortarolo, G. et al. Focus image scanning microscopy for sharp and gentle super-resolved microscopy. Nat. Commun. 13, 7723 (2022).
Google Scholar
Liu, T. et al. Multi-color live-cell STED nanoscopy of mitochondria with a gentle inner membrane stain. Proc. Natl Acad. Sci. USA 119, e2215799119 (2022).
Google Scholar
Wen, G. et al. High-fidelity structured illumination microscopy by point-spread-function engineering. Light Sci. Appl. 10, 70 (2021).
Google Scholar
Qiao, C. et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat. Methods 18, 194–202 (2021).
Google Scholar
Zhang, Y. et al. Mitochondria determine the sequential propagation of the calcium macrodomains revealed by the super-resolution calcium lantern imaging. Sci. China Life Sci. 63, 1543–1551 (2020).
Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021).
Google Scholar
Mirza, M. & Osindero, S. Conditional generative adversarial nets. Preprint at (2014).
Cao, H. et al. Swin-Unet: Unet-like pure transformer for medical image segmentation. In European Conference on Computer Vision, 205–218 (2022).
Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, 448–456 (2015).
Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. Preprint at (2014).
Biggs, D. S. & Andrews, M. Acceleration of iterative image restoration algorithms. Appl. Opt. 36, 1766–1775 (1997).
Google Scholar
Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. 9, 62–66 (1979).
Ershov, D. et al. TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines. Nat. Methods 19, 829–832 (2022).
Google Scholar
Qian, H., Sheetz, M. P. & Elson, E. L. Single particle tracking. Analysis of diffusion and flow in two-dimensional systems. Biophys. J. 60, 910–921 (1991).
Google Scholar
Ba, Q., Raghavan, G., Kiselyov, K. & Yang, G. Whole-cell scale dynamic organization of lysosomes revealed by spatial statistical analysis. Cell Rep. 23, 3591–3606 (2018).
Google Scholar
Damenti, M., Coceano, G., Pennacchietti, F., Boden, A. & Testa, I. STED and parallelized RESOLFT optical nanoscopy of the tubular endoplasmic reticulum and its mitochondrial contacts in neuronal cells. Neurobiol. Dis. 155, 105361 (2021).
Google Scholar
Zhao, W. et al. Quantitatively mapping local quality of super-resolution microscopy by rolling Fourier ring correlation. Light Sci. Appl. 12, 298 (2023).
Google Scholar
Li, M. et al. LuckyProfiler: an ImageJ plug-in capable of quantifying FWHM resolution easily and effectively for super-resolution images. Biomed. Opt. Express 13, 4310–4325 (2022).
Google Scholar
Lee, J. et al. Versatile phenotype-activated cell sorting. Sci. Adv. 6, eabb7438 (2020).
Google Scholar
Zhang, X. et al. Development of a reversibly switchable fluorescent protein for super-resolution optical fluctuation imaging (SOFI). ACS Nano 9, 2659–2667 (2015).
Google Scholar
Tillberg, P. et al. Protein-retention expansion microscopy of cells and tissues labeled using standard fluorescent proteins and antibodies. Nat. Biotechnol. 34, 987–992 (2016).
Google Scholar
link
