Bias blog series: part 1
Introduction
In our world of increasingly artificial forms of intelligence, machines that are tasked with making complex decisions are becoming more and more prevalent. There is a growing body of literature indicating the usage of AI in various domains such as Business, high-stake decision-making, and over the past few years in the medical sector. With this growing prevalence, however, people have noticed concerning tendencies in said systems; That is, whilst being inherently designed to purely follow patterns in the data, they have shown signs of prejudice, in the sense that various sexist and discriminatory behavior can be observed. The recent European AI Act, also covers the matter of such prejudice rather extensively and sets a foundation for tackling problems associated with it.
Throughout the years of technical documentation, people have tended to use the term “bias” to describe this skewed type of behavior towards certain demographics; a word whose meaning varies, causing confusion and complicating the task of addressing it.
This article is the first in a series of blog posts covering the topic of bias. In this series, we’ll aim to give you a clear, digestible understanding of bias in AI. We’ll introduce ways to measure and minimize bias and explore the role of synthetic data in this path to more fair systems. We’ll also give you a peek into how Syntho, a leading player in synthetic data generation, can contribute to this effort. So, whether you’re a practitioner looking for actionable insights or just curious about this topic, you’re in the right place.
Bias in Action: A Real-World Example
You may be wondering, “This bias in AI is all important, but what does it mean for me, for ordinary people?” The truth is, the impact is far-reaching, often invisible but potent. Bias in AI is not a mere academic concept; it’s a real-world problem with serious consequences.
Take the Dutch child welfare scandal as an example. The automated system, supposedly a tool created to generate fair and efficient results with minimal human intervention, was biased. It wrongly flagged thousands of parents for fraud based on flawed data and assumptions. The result? Families thrown into turmoil, personal reputations damaged, and financial hardship, all due to biases in an AI System. It is examples like these that highlight the urgency of addressing bias in AI.
But let’s not stop there. This incident isn’t an isolated case of bias wreaking havoc. The impact of bias in AI extends to all corners of our lives. From who gets hired for a job, who gets approved for a loan, to who receives what kind of medical treatment – biased AI systems can perpetuate existing inequalities and create new ones.
Consider this: an AI system trained on biased historical data could deny a well-qualified candidate a job simply because of their gender or ethnicity. Or a biased AI system might deny a loan to a deserving candidate because of their postcode. These are not just hypothetical scenarios; they are happening right now.
The specific types of biases, such as Historical Bias and Measurement Bias, lead to such flawed decisions. They are inherent in the data, deeply rooted in societal biases, and reflected in the unequal outcomes among different demographic groups. They can skew the decisions of predictive models and result in unfair treatment.
In the grand scheme of things, bias in AI can act as a silent influencer, subtly shaping our society and our lives, often in ways we don’t even realize. All these above-mentioned points might lead you to question why haven’t taken any action to stop it, and whether it is even possible.
Indeed, with new technological advancements it becomes increasingly more accessible to tackle such problem. The first step to addressing this problem, however, is to understand and acknowledge its existence and impact. For now, the acknowledgement of its existence has been created, leaving the matter of “understanding” to still be quite vague.
Understanding Bias
Whilst the original definition of bias as presented by the Cambridge dictionary does not stray too far away from the main purpose of the word as it relates to AI, many different interpretations are to be made of even this singular definition. Taxonomies, such as those presented by researchers such as Hellström et al (2020) and Kliegr (2021), provide deeper insights into the definition of bias. A simple glance at these papers will reveal, however, that a great narrowing of the definition of the term is required to effectively tackle the problem.
Whilst being a change of events, in order to optimally define and convey the meaning of bias one can better define the opposite, that is Fairness.
Defining Fairness
As it is defined in various recent literature such as Castelnovo et al. (2022), fairness can be elaborated upon given an understanding of the term potential space. As it exists, potential space (PS) refers to the extent of capabilities and knowledge of an individual regardless of their belonging to a certain demographic group. Given this definition of the concept of PS, one can easily define fairness to be the equality of treatment between two individuals of equal PS, regardless of their observable and hidden differences in bias inducing parameters (such as race, age, or gender). Any diversion from this definition, also called Equality of Opportunities, is a clear indication of bias and merits further investigation.
The practitioners amongst the readers might notice that achieving something as defined here might be completely impossible given the inherent biases existing in our world. That is true! The world we live in, along with all data collected from occurrences in this world, is subject to much historical and statistical bias. This, indeed, lessens the confidence of one day fully mitigating the impacts of bias on predictive models trained on such “biased” data. However, through the use of various methods, one can try to minimize the impacts of bias. This being the case, the terminology used in the rest of this blog post(s) will shift towards the idea of minimizing the impact of bias rather than fully mitigating it.
Okay! So now that an idea has been brought forth of what bias is and how one could potentially evaluate its existence; If we want to tackle the problem properly, however, we need to know where all these biases originate from.
Understanding the Sources and types
Existing research provides valuable insights into the different types of biases in machine learning. As Mehrabi et. al. (2019) have proceeded to divide biases in machine learning, one can divide biases into 3 major categories. Namely those of:
- Data to Algorithm: a cattegory encomapssing biases that originate from the data itself. Might that be caused through poor data collection, inherent biases existing in the world, etc.
- Algorithm to User: a category focusing on biases that stem from the design and functionality of the algorithms. It includes how algorithms might interpret, weigh, or consider certain data points over others, which can lead to biased outcomes.
- User to Data: pertains to biases that arise from user interaction with the system. The manner in which users input data, their inherent biases, or even their trust in system outputs can influence outcomes.
Whilst the names are indicative of the form of bias, one might still have questions as to the types of biases one might categorize under these umbrella terms. For the enthusiasts among our readers, we’ve provided links to some literature related to this terminology and classification. For the sake of simplicity in this blog post, we will cover a few select biases which are relevant to the situation (Almost all of which are of the category data to algorithm). The specific types of biases are as follows:
- Historical Bias: A type of bias inherent to the data caused by the natural biases existing in the world in different social groups and society in general. It is because of the inherence of this data in the world that it cannot be mitigated through various means of sampling and feature selection.
- Measurement Bias & Representation Bias: These two closely related biases occur when the different subgroups of the dataset contain unequal amounts of “favorable” outcomes. This type of bias can therefore skew the outcome of predictive models
- Algorithmic Bias: Bias purely related to the algorithm in use. As also observed in tests ran (elaborated upon further in the post), this type of bias can have a tremendous effect on the fairness of a given algorithm.
These foundational understandings of bias in machine learning will be utilized in order to tackle the problem more effectively in later posts.
Final Thoughts
In this exploration of bias within artificial intelligence, we’ve illuminated the profound implications it holds in our increasingly AI-driven world. From real-world examples like the Dutch child welfare scandal to the intricate nuances of bias categories and types, it’s evident that recognizing and understanding bias is paramount.
While the challenges posed by biases — whether they be historical, algorithmic, or user-induced — are significant, they are not insurmountable. With a firm grasp on the origins and manifestations of bias, we are better equipped to address them. However, recognition and understanding are just the starting points.
As we move forward in this series, our next focus will be on the tangible tools and frameworks at our disposal. How do we measure the extent of bias in AI models? And more importantly, how do we minimize its impact? These are the pressing questions we’ll delve into next, ensuring that as AI continues to evolve, it does so in a direction that is both fair and performant.
AI’s Unseen Culprit: For Good Measure
Bias blog series: part 2
Introduction
In the previous post in this series, we covered quite a bit of ground as it relates to what bias is and what the different types of it are. We are yet to define, however, a framework through which we can clearly identify the extent to which bias exists in our system; Which is why we are doing so in this post.
Naturally, when correcting for biases in machine learning models, we want to preferably not lose any predictive performance and gain fairness. To also assess the performance of the models, it is crucial to have a set of performance metrics through which one can evaluate how effective a method is for minimizing bias; and to assess the fairness, one can use a series of fairness indicators which utilize various statistical information in order to infer bias.
Therefore, it can be said that we have two components: Fairness and Performance metrics. There is varying literature as to which metrics to utilize given a particular use case, however, most of these studies exceed the scope of these blog posts; it is because of this that we have chosen one metric per category to simplify the process of evaluating the overall performance and fairness of a model. What follows are respective explanations of each metric.
Measure Twice, Cut Once
Performance Metric
The practitioners of the field of AI will be no stranger to the concept of the Area Under the Curve, or AUC for short. It is quite often that we require a comprehensive measure by which we can evaluate the performance of a model, and AUC is just that. Being a value ranging from 0 to 1, with 0 meaning a useless model and 1 indicating a perfect one, AUC is a great comprehensive type of metric, especially when looking to evaluate performance.
More closely examined, AUC is a probability measure, which examines the likelihood of positive class example being placed in proximity to a negative class (where the positive and negative classes are corelating to some value). What this probability tells us is the extent to which our model is capable of telling a negative datapoint apart from a positive example. Additionally, this metric is very valuable because it doesn’t rely on a fixed threshold and evaluates performance across all possible thresholds.
Fairness Metric
When it comes to fairness metrics, the Disparate Impact (DI) is a common choice. This metric measures the ratio between the probabilities of a positive outcome for the unprivileged and privileged groups.
To define it more formally, Disparate Impact is the ratio between the probability of a positive outcome for a datapoint from the sensitive group and the non-sensitive group. This concept can be denoted as:
[Equation]
Where S = s indicates an individual being part of the sensitive/unfavored subgroup.
The ideal DI value is 1, which would indicate that both groups have the same probability of receiving a positive outcome. A value less than 1 indicates a potential bias against the unprivileged group, while a value greater than 1 indicates a potential bias in favor of the unprivileged group. This fairness metric aligns very well with our previously laid out definition of fairness, that is of equality of opportunities; which is why this shall be regarded as our measure of fairness from now on in this blog series.
Just another Tradeoff
Whilst reading the explanation for the two metrics, you might have noticed something: Both corelate to probabilities of certain outcomes. So, one might logically deduce the fact that trying to nudge the value of DI towards 1 in order to ensure fairness would inherently result in a less ideal value for AUC provided the context that it too, is measuring probabilities. And they would be right …
This points towards the fact that the usage of bias minimization techniques does bring with them a loss in accuracy, and as is it stands with current literature, gaining fairness is associated with a trade-off in accuracy. Not to get too philosophical, however, one should note, as previously addressed in this series and by researchers like Hellström et al., that: we live in an inherently biased world full of prejudice and unjust actions; when training models that learn from existing patterns, it is evident that accuracy will be lost to gain fairness. Here a distinction should be made between modelling the world as it is and the world as we want it to be. These methods can be implemented and utilized to reduce the biases in these systems, however, tackling bias at its root requires much greater actions. For now, however, given the increasingly automated state of the world creating fair machines might be a good first step in creating a better future as it relates to equality.
It’s All About Balance
Having discussed the fundamental metrics that help gauge fairness and performance in AI systems, it’s essential to consider the implications of these metrics in real-world scenarios. The interplay between fairness and performance is a topic of significant concern, and understanding it is paramount to building AI systems that are both efficient and equitable.
Recall the concept of representation bias we introduced in the first post of this series. To provide further clarity, consider the visualization below. The outer rectangle represents the entirety of a dataset, while each smaller rectangle within denotes a subset of this dataset.
Given this structure, one must contemplate the potential effects of the disproportionality in subset ratios. How might this disproportionality influence models trained on such a dataset? Delving into these intricacies is crucial, not only for understanding the current challenges faced but also for paving the way to solutions. Before that, however, let us explore the different methods at our disposal.
Previously, we have gone into rather a lot of detail about the ins and outs of bias and evaluative metrics as they relate to fairness and performance. We’ve also covered the fact that gaining fairness, in reality, is a tradeoff of performance. One crucial thing, however, that we are yet to discuss is how one could approach minimizing the discussed bias.
The Concepts Are Few, And Methods Many
As we’ve previously covered, there are various classifications of biases. A logical extension of this is that there are various ways of tackling these different classifications of bias. These methods can be best classified through the simplification of the machine learning pipeline in to three phases: data preparation, training, and evaluation phases. With these different stages it is then evident that there are three types of strategies for tackling bias. These consist of:
- Preprocessing approaches: These techniques focus on modifying the dataset before it’s fed into the model, aiming to remove or reduce biases at the data level. By ensuring the training data is as unbiased as possible, the models derived from it are less likely to propagate these biases. Methods might include re-sampling, re-weighting, or generating synthetic samples for a balanced dataset.
- In-processing approaches: During model training, in-processing methods adjust model parameters or the loss function to minimize bias. These approaches work directly with the learning algorithm, ensuring that fairness constraints are integrated during the learning process, making models less likely to produce biased outcomes.
- Post-processing approaches: After a model is trained, post-processing techniques modify the model’s outputs to achieve fairness. These methods adjust model predictions based on fairness criteria, ensuring that final decisions align more closely with fairness objectives without having to retrain the model.
As almost all the previously covered specific biases (namely Historical, representation and measurement) are of the type that originate from the training data, we will further discuss preprocessing approaches for simplicity of the matter; for enthusiasts, however, various resources have been provided to encourage further studies.
Underrepresented groups == mistreated groups
A common type of bias is associated with that induced through the usage of data inherently containing biases; One of which being representation bias. This bias becomes apparent when different subgroups of a dataset contain unequal amounts of datapoints resulting in unjust evaluation of their status by the predictive models.
Let’s take the famous UCI adults dataset which is known to be biased as it relates to subgroups categorized based on the sensitive attributes gender and race.
Given the two outcomes of high or low income (or rather Positive and Negative), the whole dataset can be divided into 8 distinct subgroups with each containing one unique combination of values from these previously mentioned attributes. The unique values per attribute and with that the unique combinations after preprocessing are as follows:
Gender
- Male [M]
- Female[F]
Race
- White[W]
- Non-White[NW]
Outcome
- Positive [P]
- Negative [N]
Combinations
- Negative Outcome White Males (N-M-W)
- …
- Positive Outcome Non-White Females (P-F-NW)
The counts are respectively for eventual visualizatrions [2056, 8308, 2197, 14608, 168, 1184, 630, 7027]
If one looks at the ratios of datapoints of each of these subgroups to the total dataset one might notice that the majority group being Negative Outcome White Males contains 14608 datapoints and the minority group Positve Outcome Non-White Females contains only 168 datapoints of a dataset of size 36178. With these conditions, when testing (later elaborated upon) is conducted, one can notice a significant disparity between the privileged and unprivileged group reflected in the average Fairness.
This raises the question whether this bias is caused through the imbalance of data between the different subgroups. As it currently stands, the distribution of data between the subgroups is as visualized previously with unequally sized boxes, but what if we were to somehow rebalance the distribution of the datapoints such that all subgroups contained equal amounts of datapoints – similar to what is presented here below. Would that help the matter of fairness? And if so, how could one achieve this rebalancing?
This, is where synthetic data comes into the equation, and is where we will be placing our attention in the upcoming post in this series. In the next post we will proceed to define a testing pipeline through which bias can be minimized all whilst measuring model performance and fairness. In the upcoming post, we will shift our focus from definitions and explanations to practical applications and implications of bias minimization techniques for both practitioners and enthusiasts; so make sure to keep an eye out for our socials to get notified when we post the next part of this series.
AI’s Unseen Culprit: Unraveling the Bias Within
Bias blog series: part 3
Introduction
Previously in this series, we’ve delved deep into defining and understanding bias in its various forms. While we’ve discussed many facets of bias, a comprehensive method to minimize it remains to be addressed.
In this concluding post of the series, we will first propose a testing pipeline to assess the effectiveness of our approach. Then, we’ll introduce a bias-minimization technique that harnesses the power of synthetic data.
The Testing Pipeline
The Methodology
As we’ve previously touched upon, bias arises when individuals from the same Potential Space (PS) are treated differently due to their minority status. To rectify this, we need to measure the disparity in outcomes among different subgroups. For instance, instead of evaluating the general probability of a positive outcome, it’s pivotal to assess the positive rates for both privileged and unprivileged groups separately. This distinction allows us to pinpoint inequalities resulting from biases.
Though we’ve previously discussed specific performance and fairness metrics, it’s essential to emphasize that no single metric perfectly encapsulates bias. Each metric provides a unique perspective. For instance, while the Equal Opportunity Difference (EOD) offers insights into the difference in True Positive Rates (TPR) for privileged and unprivileged groups, the Disparate Impact (DI) focuses on the ratio of favorable outcomes between these groups. Their subtle differences make them collectively crucial for painting a comprehensive picture of bias.
These nuances necessitate a robust testing pipeline that evaluates both performance and fairness. The diagram below illustrates a comprehensive framework for this purpose.
Elaborating on the Testing Pipeline
Effective evaluation requires specific input parameters:
- The unfitted model: a reference to an initialized instance of a model in the sci-kit learn library.
- The train & test datasets: These are used for training the model and evaluating its performance, respectively.
- Privileged specification: a dictionary whose keys are the names of sensitive attributes. Each key’s value is a list indicating the column values classified as privileged and unprivileged.
Post model training, we can then compute standard performance metrics like accuracy, recall, false alarm rate, and precision. However, calculating fairness metrics involves a bit more nuance.
Fairness metric computation can be broken down into three distinct steps:
- Calculating base metrics (True Positives (TP), True Negative (TN), etc.)
- Computing compound metrics (True Positive Rate (TPR), False Discovery Rate (FDR), etc.)
- Deriving fairness metrics (Disparate Impact (DI), Equal Opportunity Difference (EOD), etc.)
Given our focus on evaluating privileged and unprivileged groups separately, it’s essential to compute their metrics distinctively. For this, we loop over all sensitive attributes and separate the test data points for each group. This allows us to calculate confusion matrices for each test dataset subset individually. We store these results in a dictionary for easy reference.
Once we’ve derived these base metrics, we repeat a similar process for the compound metrics. After calculating these for each sensitive attribute subgroup, we integrate them into fairness metrics. The results we present later are averages of fairness metrics, offering a concise understanding of the model’s efficacy.
With this foundation in place, we can transition to presenting a tangible solution to bias.
The Solution
As we’ve highlighted before, bias minimization methods can be broadly categorized into preprocessing, in-processing, and post-processing approaches. We’ve also indicated that imbalances in datasets can lead to inherent biases, exemplified by the Adults UCI dataset. For a deeper understanding, we recommend revisiting our previous post.
Over the years, the challenge of dataset imbalances has spurred numerous solutions. While some suggest resampling subsets, others advocate for under-sampling the majority group. However, these methods might compromise data integrity and model performance. Thus, the quest for a balance between fairness and performance continues. Fortunately, recent technological advancements have opened doors to innovative solutions.
Synthetic Data: A New Contender
The recent surge in data generation and synthesis methods offers an exciting avenue for dataset balancing. Instead of removing data for fairness, we can now generate additional data, minimizing bias while maintaining accuracy. Below is a schematic representation of our proposed bias-minimization methodology.
As depicted, the input for this pipeline consists of:
- The original (biased) dataset with all the raw data.
- An object containing all preprocessing parameters, including columns to be discretized, binarized, removed, and other attributes.
Our process initiates with partial preprocessing of the dataset until a point conducive for smooth data synthesis (stop marker 1). This version of the dataset is stored separately, and a holdout set is separated for testing. Following this, preprocessing continues for both the holdout and training datasets, retaining the states from stop marker 1.
Post-preprocessing, the training dataset undergoes data balancing. Here, the dataset is divided based on provided sensitive attributes (stop marker 2). Statistical analyses determine the required data points for balanced subgroups.
Using the data from stop marker 1, we train the synthetic data engine and generate the necessary data points. This synthetic data undergoes the second phase of preprocessing (folowing the stopmarkers starting with 3) and is similarly divided based on sensitive attributes. By integrating synthetic data, we balance the subgroups, forming a “debiased” and balanced dataset. This dataset, along with the untouched holdout set, is then available for testing.
The Results
Tests using our proposed framework reveal significant efficacy in bias minimization. In some scenarios, we observed up to 2x improvements in the fairness metric DI. In other metrics, the gains were as high as tenfold, all while preserving the AUC score and incurring minimal accuracy loss. Some of these findings are visualized below.
However, there are crucial insights to consider:
- Model specificity: As we’ve discussed, biases can emerge from model selection. Our tests revealed models like Random Forest and Gradient Boosting performing best with default hyperparameters. However, model fine-tuning remains a promising area for further exploration.
- Sensitive attributes matter: During testing, specific combinations of sensitive attributes resulted in significant accuracy losses with negligible fairness improvements. This emphasizes the need for careful attribute selection.
- Model efficacy depends on data and use-case: While our model has demonstrated efficacy across various scenarios, its effectiveness is contingent on the quality and nature of the input data. For instance, if a dataset inherently lacks representation from certain subgroups, bias minimization for those groups becomes infeasible. This reinforces the idea that every step in the machine learning pipeline should be crafted to minimize biases.
Final Thoughts
The results presented here, mark a major breakthrough in dealing with unbalanced data and their associated biases; These are all made possible through leveraging the power of synthetic data, and would have not been possible a mere 10 years ago. It is the case that syntho is a leading company in the generation of synthetic data. It is Syntho’s dedication to exploring new possibilities and it’s willingness to create a better future that has facilitied this research and the results associated with it. If you are interested to explore what Syntho can do for your company, contact us so we can help you leverage the power of Synthetic data in your company today!