Introduction: We previously developed a web service IsletNet, which allows the participating centers to fully automatically assess images of dithizone-stained isolated pancreatic islets before transplantation to human recipients. At the core of the web service resides a neural network trained on islet images, which are annotated by a single expert. Because the annotation process is subjective, a validation by independent experts is necessary. Here we present a novel platform inviting experts to evaluate the quality of islet images and the corresponding annotations in order to improve IsletNet service.
Methods: The system is composed of a central server containing an API with a management interface running on the PHP-based Symfony framework, and a mobile application for end users, which was created on the dart-based Flutter framework. Java method Collections.shuffle is used for randomization. The images were anonymized. The automatic entry system selects random annotated images from a larger collection. Next, the islet diameter is calculated (from islet area) and all islets in each image are classified into the standard islet size categories (50 µm increments), each category is represented by random islets selected for expert evaluation. The manual entry system allows for selection of images and islets at will.

Results: The mobile app is designed to display the number of annotated images queued in the current batch, visualizing three of them at a time as not to overwhelm the experts (Fig.1). Islets whose contours are to be assessed are marked by an arrow. The experts are expected to zoom at marked islets,  imagine the correct contour, then view the contour to be tested, and swipe right if agree, swipe left if disagree, or swipe down if undecided (Fig.2). After evaluating individual islets, experts are requested to assess in general terms the image and contour qualities. Prior to sending away the assessment, a note can be left. Expert opinions are collected in csv files for statistical evaluation. Individual islets can be displayed together with the assessment of all experts to learn or discuss. Expert assessment can be paused at any time. The app will reward the experts 'prizes' to keep them alert and joyful during otherwise monotonous exercise.

Conclusion: The platform is designed to share expert opinion among the participating centers, while preserving the privacy of the image sources. The purpose of this app is two-fold: (1) to reveal in quantive terms the consensus/discord among experts from independent islet centers; (2) to provide the necessary feedback regarding the quality of the islet image annotation for IsletNet training and validation. IsletSwipe will be operative in course of the conference; requests to join: daht@ikem.cz.