Review — Shifted Windows Transformers for Medical Image Quality Assessment

Swin Transformer for Foreign Object Classification and Left Ventricular Outflow Tract Classification

**Shifted Windows Transformers for Medical Image Quality Assessment Swin-MIQA, by Istanbul Technical University, Mehmet Akif Ersoy Thoracic and Cardiovascular Surgery Training and Research Hospital, Acibadem University, King’s College London 2022 MLMI **(Sik-Ho Tsang @ Medium)

Quality Assessment ==== My Healthcare and Medical Related Paper Readings ==== ==== My Other Paper Readings Are Also Over Here ====

• There are medical image quality problems which affect diagnosis such as patient breathing or arrhythmia in Cardiac Magnetic Resonance Imaging (MRI), ringing artifact presence in Computed Tomography (CT), and low dose or foreign object appearance on the Chest X-Rays (CXR).
• Eliminating or correcting the samples with poor image quality is necessary for downstream tasks such as segmentation and disease diagnosis.
• **Swin Transformer is used for poor-quality image classification that causes the degradation in medical image quality, particularly Foreign Object Classification problem on Chest X-Rays (Object-CXR) and Left Ventricular Outflow Tract Classification** problem on Cardiac MRI with a four-chamber view (LVOT).
• This is the first vision transformer application for medical image quality assessment.

Outline

1. **Swin Transformer for IQA**
2. Results

1. Swin Transformer for Medical IQA

1.1. Model Architecture

Swin Transformer for Medical IQA

• Given an input image, Swin Transformer first divides the image into patches using a Patch Partition layer that generates 48-dimensional representations for each of the 4 × 4 patches on the image.
• After that, as shown in Stage 1, the Linear Embedding layer transforms 48-d representations into C dimensions.
• The second component of Stage 1 is the Swin Transformer Blocks which has been inherited from the general Vision Transformer (ViT) architecture. Despite having the same high-level block structure as the one in ViT, multiheaded self-attention (MSA) blocks are replaced with an alternate ordering of window-based MSA (W-MSA) and shifted window-based MSA (SW-MSA), sequentially.
• There are also stages 2, 3, 4, which are similar to stage 1.
• Lastly, after using layer normalization followed by an average pooling layer to calculate the features of the Swin Transformer model, the output is used for classification in a single Linear layer.

This model classifies if there is anything present in the input image that can affect the image quality adversely.

Swin Transformer Variants (Stochastic Depth refers to the probability of disregarding the transformation that is applied on a sample within a batch during training.)

1.2. Model Training

• The learning rate warm-up scheme and learning rate scheduling and leverage color jittering, Stochastic Depth (only for Swin Transformer) [7], Random Erasing, MixUp, CutMix, and RandAugment, are used.
• For the foreign object classification model, batch size is 16, the number of gradient accumulation steps is 8 and window size is 8.
• For the LVOT classification model, batch size is 64, the number of gradient accumulation steps is 2 and window size is 7, due to lower input image resolution comparing chest X-Rays.
• ImageNet pretrained model is used.
• The fine-tuning procedure takes 60 epochs with 5 epochs of a warm-up period.
• The models are also tried to be trained from scratch for 300 epochs.
• Cross entropy is used as loss function.

2. Results

2.1. Datasets

• Object-CXR dataset contains 8,000 samples for training and 1,000 samples for each of the validation and testing splits. There is an equal number of positive and negative samples for each split.
• LVOT classification dataset contains 272 positive and 278 negative samples that are used for training, 34 positive and 35 negative samples for validation, and 35 positive and 35 negative samples for testing. 3D CMR patient scans are sliced into 2D images, which is also helpful in terms of increasing the number of samples for each split by a factor of 25.

2.2. Ablation Studies

Ablation Studies

• The accuracy is the highest for the Swin-Tiny model with an accuracy of 95.59%.
• **Swin-Small’s AUC (Area Under Curve) score is performing the best among these three models with a score of 0.980.**
• Switching from pre-trained model to training from scratch significantly decreases the accuracy of at least around 12.35% and AUC score of 0.059.

2.2. LVOT

LVOT

• LVOT classification task is trivial, where the goal is to check if fifth chamber exists on the Cardiac MRI scan — generally located at the centre.
• As a result, it becomes possible to obtain testing accuracy of over 95% and AUC scores exceeding 0.969 for all ResNet architectures and Swin-Small without overfitting.

2.3. Object-CXR

Object-CXR

• The best performance is obtained by the **Swin-Base model with 87.1% testing accuracy and 0.922 testing AUC score.**
• The closest performance to the best model is the **ResNet-152 model, which has 1.5× less number of parameters and 25% less floating point operations, at a cost of 2% less testing accuracy and 0.013 less AUC score.**

2.4. Qualitative Analysis

Object-CXR

• Regardless of the orientation and contrast factors, the proposed model can correctly classify the X-Ray images which does and does not contain foreign objects.
• Bottom Rightmost: Even though the foreign objects may not be visible to human eye without altering the contrast, the proposed model can still successfully classify the image as it contains foreign objects.

LVOT

• As shown in Fig. 3b, the proposed model can correctly classify the images containing LVOT regions under low contrast.