Multi tasks RetinaNet for mitosis detection
Abstract
The account of mitotic cells is a key feature in tumor diagnosis. However, due to the variability of mitotic cell morphology, it is a highly challenging task to detect mitotic cells in tumor tissues. At the same time, although advanced deep learning method have achieved great success in cell detection, the performance is often unsatisfactory when tested data from another domain (i.e. the different tumor types and different scanners). Therefore, it is necessary to develop algorithms for detecting mitotic cells with robustness in domain shifts scenarios. Our work further proposes a foreground detection and tumor classification task based on the baseline(Retinanet), and utilizes data augmentation to improve the domain generalization performance of our model. We achieve the state-of-the-art performance (F1 score: 0.5809) on the challenging premilary test dataset.
Yongbing Zhang
bject Detection | Mitosis Detection | Multi Task
ybzhang08@hit.edu.cn
Equal contribution: Yang Chen, Ziyue Wang
Introduction
Histopathology is the gold standard for tumor diagnosis and prognosis, especially with the rapid development of deep learning technology and the popularization of whole slide image (WSI) scanners in recent years, computational pathology has received high attention.
In the clinical diagnosis of tumors, judging the degree of malignant proliferation of tumors is a very key prognostic indicator, and also a key reference for tumor grading and treatment. Usually pathologists need to use KI67 on histopathological sections by immunohistochemistry(cai2021generalizing). The tissue sections were stained with stains, and the degree of malignancy of tumor proliferation was judged according to the degree of immunohistochemical staining.
However, immunohistochemistry is more time-consuming than conventional HE-stained sections, and the cost is higher in some medically underdeveloped areas. In recent years, with the development of deep learning in the direction of target detection in computer vision, it has become an increasing challenge to identify mitotic cells directly from HE-stained histopathological sections by detection algorithm.
Material and Methods
0.1 Dataset description
Mitosis domain generalization challenge 2022 (MIDOG 2022) (mitosis) is a further expansion of tumor types and types of scanners based on the MIDOG 2021 competition (aubreville2022mitosis, aubreville2022biomedical). In the MIDOG 2021 competition, the competition only focused on the color generalization error caused by different scanners on the same tumor (breast cancer). In MIDOG 2022, the organizers of the competition digitized different tumor cell sections from different laboratories and different species (i.e. human, canine) under different scanners. MIDOG 2022 aims to explore mitosis cell detection algorithms with multiple domains (differences in tissue and collection methods).
The training dataset contains 405 WSIs, including 6 tumor tissue types and 2 species: Canine Lung Cancer, Human Breast Cancer, Canine Lymphoma, Human Neuroendocrine Tumor, Canine Cutaneous Mast Cell Tumor, and Human Melanoma; The test dataset contains 10 kinds of tumor tissues and the type is unknown.
The mitotic cells of the competition are determined by visual assessment by a trained pathologist. The competition organizer provided mitotic cell annotations for the five tumor tissue sections in the training dataset, excluding Human melanoma, and the corresponding scanner type for each tumor type. Since breast cancer contains 3 scanner types at the same time, and there is no specific scanner type corresponding to each WSI, we consider that the scanner type of breast cancer tissue section is unknown.
0.2 Methods
We propose a multi-task mitotic cell detection model based on RetinaNet, in which the main improvements include three parts:
-
An auxiliary classification network is used to classify tumors, including six categories: prostate cancer, lymphoma, lung cancer, melanoma, breast cancer and mast cell tumor.
-
An auxiliary classification network is used to classify whether the patch contains mitotic cells (or hard samples).
-
A data augmentation transform is used to improve the domain generalization ability of the model and detect mitotic cells of different types of tumors (different scanners).
0.3 Multi task auxiliary classification
As shown in the Fig.1, we firstly select the deep feature of FPN in RetinanNet (lin2020focal) and then add two fully connected network auxiliary classification tasks after this deep feature.
For the tumor auxiliary classification task, we use cross entropy. During the training, we consider 6 tissue types at the same time. The loss function is defined as follows:
| (1) |
where, represents the real tumor category of the ith object, indicates the predicted tumor category probability, and indicates the number of tumor categories.
For the foreground auxiliary classification task, we firstly regard the patches containing mitotic cells or hard samples as foreground classes, and the patches that do not contain mitotic cells or hard samples as background classes, and use Focal loss as the loss function, which is defined as follows :
| (2) |
Where and are two super parameters of focal loss, which are used to adjust the contribution of difficult samples to classification.
0.4 Data Augmentation
We use the Color Jitter data enhancement module to improve the domain generalization ability of the model, where the parameter settings of Color Jitter are: brightness = 0.35, contrast = 0.2, saturation=0.1, hue=0.1. Other data augmentation includes: random crop, random horizontal flip and vertical flip.
0.5 Hyperparameter setting
We use Adam optimizer. The learning rate is 1e-5, and the batch is 16. For RetinaNet’s FPN, we use the ResNet 50 network based on ImageNet 1K pre-training. For the hyperparameter of Focus loss, we set = 0.25 and = 2.
Results
In order to verify the mitotic cell detection performance and generalization performance of our proposed model, we re-divided the training set and validation set according to the original training set. The newly constructed training set includes four tumor tissues: canine lung cancer, human breast cancer, canine lymphoma and canine cutaneous mastocytoma., The test set includes human neuroendocrine tumor. Under the general hyperparametric setting, we verified the performance of multi task RetinaNet through ablation studies.
0.6 Ablation study
As show in Tabel 1, the basic performence of RetinaNet’s mAP (Iou is 0.5) on mitosis cell detection is 0.433. Compared with the baseline, the three components proposed by us on this basis can improve the detection performance.The most obvious performance improvement is foreground auxiliary classification component. When we integrate the three proposed components, the performance of the model reaches the maximum 0.486 mAP.
|
|
|
mAP(iou=0.5) | ||||||
| 0.433 | |||||||||
| ✓ | 0.449 | ||||||||
| ✓ | 0.458 | ||||||||
| ✓ | 0.451 | ||||||||
| ✓ | ✓ | 0.463 | |||||||
| ✓ | ✓ | 0.465 | |||||||
| ✓ | ✓ | 0.473 | |||||||
| ✓ | ✓ | ✓ | 0.486 |
Discussion
Our analysis shows that for the mitotic cell detection task, adding auxiliary classification loss helps to significantly improve the detection performance of the model. For the domain generalization problem from different scanners, compared with the previous domain adaptation algorithm to solve the problem In order to improve the generalization of the model, we propose that the use of colorjitter data augmentation can also increase the domain generalization performance of the model under different Color disturbance of scanners. Especially when the test images come from unknown, the method may be more robust than domain adversarial algorithms.