Mitosis domain generalization in histopathology images — The MIDOG challenge

Marc Aubreville; Nikolas Stathonikos; Christof A. Bertram; Robert Klopfleisch; Natalie ter Hoeve; Francesco Ciompi; Frauke Wilm; Christian Marzahl; Taryn A. Donovan; Andreas Maier; Jack Breen; Nishant Ravikumar; Youjin Chung; Jinah Park; Ramin Nateghi; Fattaneh Pourakpour; Rutger H.J. Fick; Saima Ben Hadj; Mostafa Jahanifar; Adam Shephard; Jakob Dexl; Thomas Wittenberg; Satoshi Kondo; Maxime W. Lafarge; Viktor H. Koelzer; Jingtang Liang; Yubo Wang; Xi Long; Jingxin Liu; Salar Razavi; April Khademi; Sen Yang; Xiyue Wang; Ramona Erber; Andrea Klang; Karoline Lipnik; Pompei Bolfa; Michael J. Dark; Gabriel Wasinger; Mitko Veta; Katharina Breininger

doi:10.1016/j.media.2022.102699

Mitosis domain generalization in histopathology images — The MIDOG challenge

Marc Aubreville^*, Nikolas Stathonikos, Christof A. Bertram, Robert Klopfleisch, Natalie ter Hoeve, Francesco Ciompi, Frauke Wilm, Christian Marzahl, Taryn A. Donovan, Andreas Maier, Jack Breen, Nishant Ravikumar, Youjin Chung, Jinah Park, Ramin Nateghi, Fattaneh Pourakpour, Rutger H.J. Fick, Saima Ben Hadj, Mostafa Jahanifar, Adam ShephardJakob Dexl, Thomas Wittenberg, Satoshi Kondo, Maxime W. Lafarge, Viktor H. Koelzer, Jingtang Liang, Yubo Wang, Xi Long, Jingxin Liu, Salar Razavi, April Khademi, Sen Yang, Xiyue Wang, Ramona Erber, Andrea Klang, Karoline Lipnik, Pompei Bolfa, Michael J. Dark, Gabriel Wasinger, Mitko Veta, Katharina Breininger

^*Corresponding author for this work

School of AI and Advanced Computing

Research output: Contribution to journal › Article › peer-review

43 Citations (Scopus)

Abstract

The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F₁ score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.

Original language	English
Article number	102699
Journal	Medical Image Analysis
Volume	84
DOIs	https://doi.org/10.1016/j.media.2022.102699
Publication status	E-pub ahead of print - Feb 2023

Keywords

Challenge
Deep Learning
Domain generalization
Histopathology
Mitosis

Access to Document

10.1016/j.media.2022.102699

Cite this

Aubreville, M., Stathonikos, N., Bertram, C. A., Klopfleisch, R., ter Hoeve, N., Ciompi, F., Wilm, F., Marzahl, C., Donovan, T. A., Maier, A., Breen, J., Ravikumar, N., Chung, Y., Park, J., Nateghi, R., Pourakpour, F., Fick, R. H. J., Ben Hadj, S., Jahanifar, M., ... Breininger, K. (2023). Mitosis domain generalization in histopathology images — The MIDOG challenge. Medical Image Analysis, 84, Article 102699. Advance online publication. https://doi.org/10.1016/j.media.2022.102699

@article{a97ddc73ccb64aa094fbdac7db17a2b7,

title = "Mitosis domain generalization in histopathology images — The MIDOG challenge",

abstract = "The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.",

keywords = "Challenge, Deep Learning, Domain generalization, Histopathology, Mitosis",

author = "Marc Aubreville and Nikolas Stathonikos and Bertram, {Christof A.} and Robert Klopfleisch and {ter Hoeve}, Natalie and Francesco Ciompi and Frauke Wilm and Christian Marzahl and Donovan, {Taryn A.} and Andreas Maier and Jack Breen and Nishant Ravikumar and Youjin Chung and Jinah Park and Ramin Nateghi and Fattaneh Pourakpour and Fick, {Rutger H.J.} and {Ben Hadj}, Saima and Mostafa Jahanifar and Adam Shephard and Jakob Dexl and Thomas Wittenberg and Satoshi Kondo and Lafarge, {Maxime W.} and Koelzer, {Viktor H.} and Jingtang Liang and Yubo Wang and Xi Long and Jingxin Liu and Salar Razavi and April Khademi and Sen Yang and Xiyue Wang and Ramona Erber and Andrea Klang and Karoline Lipnik and Pompei Bolfa and Dark, {Michael J.} and Gabriel Wasinger and Mitko Veta and Katharina Breininger",

note = "Publisher Copyright: {\textcopyright} 2022 Elsevier B.V.",

year = "2023",

month = feb,

doi = "10.1016/j.media.2022.102699",

language = "English",

volume = "84",

journal = "Medical Image Analysis",

issn = "1361-8415",

}

Aubreville, M, Stathonikos, N, Bertram, CA, Klopfleisch, R, ter Hoeve, N, Ciompi, F, Wilm, F, Marzahl, C, Donovan, TA, Maier, A, Breen, J, Ravikumar, N, Chung, Y, Park, J, Nateghi, R, Pourakpour, F, Fick, RHJ, Ben Hadj, S, Jahanifar, M, Shephard, A, Dexl, J, Wittenberg, T, Kondo, S, Lafarge, MW, Koelzer, VH, Liang, J, Wang, Y, Long, X, Liu, J, Razavi, S, Khademi, A, Yang, S, Wang, X, Erber, R, Klang, A, Lipnik, K, Bolfa, P, Dark, MJ, Wasinger, G, Veta, M & Breininger, K 2023, 'Mitosis domain generalization in histopathology images — The MIDOG challenge', Medical Image Analysis, vol. 84, 102699. https://doi.org/10.1016/j.media.2022.102699

TY - JOUR

T1 - Mitosis domain generalization in histopathology images — The MIDOG challenge

AU - Aubreville, Marc

AU - Stathonikos, Nikolas

AU - Bertram, Christof A.

AU - Klopfleisch, Robert

AU - ter Hoeve, Natalie

AU - Ciompi, Francesco

AU - Wilm, Frauke

AU - Marzahl, Christian

AU - Donovan, Taryn A.

AU - Maier, Andreas

AU - Breen, Jack

AU - Ravikumar, Nishant

AU - Chung, Youjin

AU - Park, Jinah

AU - Nateghi, Ramin

AU - Pourakpour, Fattaneh

AU - Fick, Rutger H.J.

AU - Ben Hadj, Saima

AU - Jahanifar, Mostafa

AU - Shephard, Adam

AU - Dexl, Jakob

AU - Wittenberg, Thomas

AU - Kondo, Satoshi

AU - Lafarge, Maxime W.

AU - Koelzer, Viktor H.

AU - Liang, Jingtang

AU - Wang, Yubo

AU - Long, Xi

AU - Liu, Jingxin

AU - Razavi, Salar

AU - Khademi, April

AU - Yang, Sen

AU - Wang, Xiyue

AU - Erber, Ramona

AU - Klang, Andrea

AU - Lipnik, Karoline

AU - Bolfa, Pompei

AU - Dark, Michael J.

AU - Wasinger, Gabriel

AU - Veta, Mitko

AU - Breininger, Katharina

PY - 2023/2

Y1 - 2023/2

N2 - The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.

AB - The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.

KW - Challenge

KW - Deep Learning

KW - Domain generalization

KW - Histopathology

KW - Mitosis

UR - http://www.scopus.com/inward/record.url?scp=85143519600&partnerID=8YFLogxK

U2 - 10.1016/j.media.2022.102699

DO - 10.1016/j.media.2022.102699

M3 - Article

C2 - 36463832

AN - SCOPUS:85143519600

SN - 1361-8415

VL - 84

JO - Medical Image Analysis

JF - Medical Image Analysis

M1 - 102699

ER -

Mitosis domain generalization in histopathology images — The MIDOG challenge

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this