Digital staining in cervical cytology: Generative adversarial networks for digital staining

Abstract

Cervical cancer is the fourth most common malignant neoplasm in women worldwide. The diagnosis relies on Papanicolaou test, which includes the examination in a light microscope of a stained slide of exfoliated cervical cells. The aim of this dissertation is to transform unstained cytology into digitally stained cervical cytology. A dataset was obtained by acquired whole-slide images from the same sample before and after Papanicolaou staining. Whole-slide images of normal cervical cells were digitalised and subdivided into image patches for model training and evaluation. Four model configurations were evaluated: CycleGAN, pix2pix, CycleGAN-Turbo and pix2pix-Turbo. Among the non-Turbo approaches, CycleGAN produced a more coherent morphology and staining patterns than pix2pix. Overall, pix2pix-Turbo produced the most realistic results, better preserving cellular structure and staining appearance under imperfect image registration, due to its supervised approach and pretrained diffusion backbone. A quantitative assessment of pix2pix-Turbo yielded a SSIM value of 0.76 and a PSNR value of 18.18. However, these values should be interpreted with caution given the structural mismatches between the output and ground truth patches. A Visual Turing Test questionnaire involving 12 participants and 60 images (30 real and 30 synthetic) indicated that participants were able to distinguish between real and synthetic patches. In summary, digital staining in cervical cytology seems feasible, but further validation is required before its routine diagnostic use. Future work should include larger and more diverse datasets, including pathological cases, with improved acquisition and registration procedures methods.