How can machine learning be biased? You might think that providing thousands of images to train an algorithm without inherent knowledge of age, sex, race or other demographic characteristics should by default develop a non biased result. However just because the machine is trained “blind” does not mean it’s actually blind.
Let’s discuss some mechanisms that lead to biases in diagnostic algorithms as mentioned by the authors of this article: Contrastive learning enhances fairness in pathology artificial intelligence systems
- Sample demographics and representation: Many of the data sets/whole slide images available for training algorithms skew towards certain populations that are not entirely representative of the population we treat locally or even nationally. So this allows the algorithms to inherently be skewed towards majority populations; potentially leading to lower diagnostic accuracy for under-represented groups.
- Visible histologic characteristics: Algorithms can latch onto tissue characteristics – such as the presence or intensity of inflammation – that may more often be associated with certain age, gender, or race demographics. If these characteristics differ in a minority group, the algorithm may misinterpret them, leading to over-diagnosis or missed diagnoses.
- Unseen molecular characteristics: It has been reported that the algorithms are picking up on molecular profiles, possibly through subtle morphologic changes that perhaps are not as visible to the standard pathologist eye such as TP53 or CDH1 mutations. These mutations may also vary in prevalence amongst different demographic characteristics. The AI may inadvertently use these molecular associated variations as a shortcut to the diagnosis which may make the tool less effective when applied to populations where those specific mutations are less prevalent.
So as different algorithms become available for use in our labs, as a pathologist we must consider what has been done to mitigate bias in these algorithms to not inadvertently worsen health care disparities.
So this study discusses several ways to mitigate bias in algorithms including adjusting the samples to more accurately reflect the demographics of the actual population and proactively attempting to remove data that associates with demographic or demographic associated variables. The authors of this study developed a framework called FAIR-Path (Fairness-aware Artificial Intelligence Review in Pathology). This model used contrastive learning to mitigate bias in AI based pathology evaluation. When comparing distinct diagnostic categories, such as lung squamous cell carcinoma versus lung adenocarcinoma, they determined that this model was able to mitigate nearly 90% of the disparities.
In the end as both locally trained and commercially available AI based tools to assist in pathologic diagnosis become readily used, we must ask the question “what has been done to mitigate training bias into the algorithms?”. We should look for measures that were either applied during training of the algorithm or are applied during post processing to ensure that these tools do not inadvertently worsen healthcare disparities. The goal is to ensure that AI tools serve as a robust assistant to our work regardless of the patient background.