The goal of Artificial Intelligence ‘AI’ development was to produce "strong AI," or machines with cognitive abilities comparable to those of humans. American philosopher John Searle coined this phrase in 1980, which is now used extensively in the AI community. AI is the technology that is effectively trained by data analytics to identify patterns on their own and apply them to conclusions and forecasts. This type of learning is called "deep learning," while on a higher level, it is called "machine learning." An algorithm is defined as ‘a preset sequence of computer operations needed by a computing system in order to finish a task or find a solution.’
Despite significant progress in research, the vulnerability associated with learning algorithms remains unresolved, with many contending that these algorithms are vulnerable to the most basic features of the input data. These vulnerabilities in the data act as defects that result in biases in the output. The sources of the defects in the datasets can be classified as either representative data, where the data source may be real but the outcome may still be biased, or unrepresentative data, where the data is biased because the sources from which it is acquired are unreliable.
One of the most crucial components in understanding the rationale behind the results the system generates is the data that is supplied into it. The supplied data may have certain innate biases that could have an impact on the result. So, if the input data is impaired with discrimination, then the output will be defected as well. The issue of algorithmic data and bias, or ‘automation bias’ must be reduced to avoid causing a host of other issues.
It has been observed that artificial intelligence frequently possesses the same prejudices as humans. It is after all a result of the input data that the human agency provides. Yet, somehow, AI being biased poses a greater risk. Social scientists have frequently shown that there are large gender and racial biases in people's opinions. It is an accepted fact that humans are imperfect and at most times, lack objectivity. However a machine is designed to remedy this flaw, and so, is considered to be infallible. Consequently, a biased result, based on a biased dataset, resulting from the inherent human biases is not only a great betrayal, it also poses a great risk of reproducing or strengthening pre-existing prejudices against marginalised groups, including women and workers of colour if considered objective, neutral and infallible.
Examples And Case Studies
The primary cause of the problem is typically the underlying data rather than the algorithm itself. For example, employers may programme the AI to adjudge candidates based on the school, college or university they attended. This tends to favour the candidates belonging more affluent families and discriminate against others [especially, minority groups].
Similarly, Amazon had once tested an algorithm to comb through the resumes of hired candidates to programme its AI to find the top applicants from a pool of fresh candidates. However, given that Silicon Valley has historically been predominately male, consequently, Amazon's AI heavily favoured male candidates. In fact, the bias ran so deep that the AI even undermined those who attended women's universities. Owing to these gender-specific algorithmic biases, Amazon was forced to scrap the use of the AI. 1
More concerning is the example of the Correctional Offender Management Profiling for Alternative Sanctions Algorithm ‘COMPAS’. COMPAS is a recidivism risk algorithm that was developed by a Statistics professor and a Corrections Industry Professional, used by [some] US states. It ranks criminal defendants' likelihood of reoffending based on two dozen factors, such as “criminal histories”, “the crime they committed”, “substance abuse” and "criminal personality." The sentencing decisions for those defendants are then based on this data. Judges might use a specific mathematical logic to assess parole, which produced a "score" of recidivism risk and a related classification: risk of recidivism and risk of violent recidivism. 2
Subsequently, a ProPublica analysis showed that COMPAS drastically overestimates Black recidivism rates while underestimating White recidivism rates, resulting in excessively harsh penalties for offenders of colour. 3 That is to say, that the programme consistently misclassified black offenders as more likely to commit crimes again than white defendants, in a way that may be considered discriminatory given the higher likelihood of a mistake.
Algorithmic systems frequently exhibit discriminatory treatment in similar ways. It has been discovered that the fingerprint recognition algorithm in India's Aadhaar biometric authentication system consistently discriminates against older people and manual labourers. Starting out as a voluntary form of identification, the AADHAAR scheme quickly expanded to be required for various key services in India [access to healthcare facilities, admission in schools, et cetera]. The qualifying requirements for AADHAAR for non-citizens have been modified by the Indian government, which now excludes some groups, making it the proof of Indian citizenship. It is expressly banned for refugees to obtain residency cards without one, which leads to profiling-based discrimination. This was brought before the Hon’ble Supreme Court of India in the case of Justice K.S. Puttaswamy v. Union of India. 4 The AADHAR scheme was held to be constitutionally valid in this 4:1 majority judgement. It was further held that it upheld Articles 14, 15, 19 and 21 of the Constitution and aims at providing a better access to fundamental rights to the weaker and disenfranchised persons.
Discrimination In Law
The laws of most countries bar any discrimination on the basis of sex, caste, creed, race, language, or any other parameter. However, the advent of technology [especially, AI] has resulted in exacerbation of such inherent biases. Given this approach, one would assume that keeping algorithms' inputs and training data free of racial, gender-based and other discrimination would be enough to prevent biases, but this is not the case. The advancement of technology has been so rapid that regulations have failed to keep up. Consequently, codified laws are needed on this issue to protect the rights of the minority groups.
One of the most crucial components in understanding the rationale behind the results the system generates is the data that is supplied into it. The supplied data may have certain innate biases that could have an impact on the result. So, if the input data is impaired with discrimination, then the output will be defected as well. The issue of algorithmic data and bias, or ‘automation bias’ must be reduced to avoid causing a host of other issues.
- Globally
While the idea of regulating AI in terms of bias has not received much attention, yet some jurisdictions have pioneered laws, rules and regulations related to it. The European Union’s General Data Protection Regulation ‘EU GDPR’ grants various rights to Data Subjects. These rights are safeguarded under Articles 12 to 23 of the GDPR. Article 22 addresses automated individual decision-making [including profiling] and constitutes the main area of concern here. As per Article 22, the entire process must be automated for this provision to apply; otherwise, the GDPR regulations will not applicable. At the same time, human oversight is considered necessary to keep the AI and other technological tools in check. The juxtaposition creates a significant jurisdictional problem. 5
Furthermore, the European Commission has enacted the Artificial Intelligence Act [EU AIA]. In summary, Articles 10, 12, and 13 together provide a stronger framework for regulating bias; however, the ambiguity surrounding the definition of data quality, data collection, and effective oversight [human-based] guidelines raises further concerns about how to ascertain whether bias is present in the data samples itself. While the initial proposal was in 2021 with amendments occurring in 2023 as the result of a Compromise between the EU Council, the European Parliament and the European Commission. The Compromise amended provisions related to GPAI [General-Purpose Artificial Intelligence], introduced limitations on the use of biometric data and identification systems, rights of consumers [including right to redressal] and increased the amount of fines. 6
Meanwhile, in the United States of America, the Algorithmic Accountability Act ‘AAA’ was proposed in 2022 to identify and decrease the risks related to social, ethical, and legal issues. AAA advises organisations that use AI systems to implement a variety of doable strategies. AAA aims to exert sector-wide oversight over algorithmic decision-making systems. It emphasises that in order to ameliorate any ethical worries, organisations employing the use of AI must do an effect evaluation of the programmes they use. 7 - Laws In India
India is a large, diverse and dynamic country, resulting in unique challenges when it comes to obtaining training data, which ultimately results in algorithmic biases. It is likely that the training datasets contain biases because of how frequently these biases occur. As a result, creating automated censorship strategies with these datasets as input will inadvertently result in automatic bias.
The Constitution of India guarantees equality and prohibits discrimination in the form of fundamental rights [Part III of the Constitution]. These rights form part of the ‘basic structure’ of the Constitution. 8 Article 14 promises the fundamental right of equality and equal protection to all persons. Article 15 prohibits discrimination against any citizen on the basis only of religion, race, caste, sex, place of birth or any of them. Article 16 promises equality in opportunity [in employment or appointment in Government offices] and Article 21 safeguards the right to life and liberty.
It is pertinent to note that both vertical and horizontal discrimination are forbidden under Article 15 of the Indian Constitution. This further indicates that it is unconstitutional for the State and its people to discriminate against any citizen on the grounds stated in the Article. Therefore, the question that emerges is whether AI can discriminate between two citizens, thereby roping in Article 15 clause [2] into consideration.
In the landmark judgement of State of Kerala v. N.M. Thomas, CJI A.N. Ray opined that, ‘Articles 14, 15 and 16 supplement each other, and that Article 16(1) gives effect to Article 14. Both of these permit reasonable classification as per the object to be achieved. Inherent limitation to the concept of equality is that it is for those who are equals or are similarly circumstanced.’ 9
Furthermore, in the case of Indra Sawhney v. Union of India 10 , HMJ Sawant remarked, “Equality is secured not just by treating equals equally but also by treating un-equals unequally through positive measures to abolish inequality. Equalising measures must use the same tools by which inequality was introduced.“ HMJ Sahai supplemented this by adding that, “Abstract equality is neither the theme nor philosophy of the Constitution and real equality through practical means is the avowed objective. The ethical justification for “reverse discrimination or protective benefits or ameliorative measures” is the need for compensating groups for past injustices and promoting social values. This compensatory principle demands provision of assistance to overcome shortcomings until the point that the disadvantage disappears.“
Although there are currently no laws specifically governing AI in India, the country has launched programmes and recommendations for the responsible development and deployment of AI technology in the recent few years. The ‘National Strategy for Artificial Intelligence #AIForAll’ plan, published in 2018 by the NITI Aayog, included guidelines for AI research and development pertaining to healthcare, agriculture, education, smart cities and infrastructure, and smart mobility and transformation. 11
Following this, the NITI Aayog published ‘Part 1 – Principles for Responsible AI’ in February 2021. 12 This paper, that is broken down into systemic and societal considerations, examines the numerous ethical issues surrounding the implementation of AI solutions in India. While societal issues centre around how automation will affect employment and job development, systemic considerations primarily address the general rules guiding decision-making, beneficiaries' legitimate involvement, and the responsibility of AI judgements. ‘Part 2 – Operationalizing concepts for Responsible AI’ was published in August 2021. 13 It focuses on these operationalizing concepts. The report outlines the steps that the public and private sectors must take in collaboration with research institutes to address regulatory and policy interventions, capacity building, ethical design incentives, and developing frameworks that adhere to pertinent AI standards.
Furthermore, the Digital Personal Data Protection Act ‘DPDPA’ was also enacted in India in 2023. While DPDPA does not include any specific provisions related to AI, it may be read in conjunction with the Information Technology Act, 2000 to address some AI and its algorithmic data-related privacy concerns. Moreover, India is a participant in the Global Partnership on Artificial Intelligence [GPAI] as well.
Solutions
Ideally, there should be no legal issues arising from attempts to define the target variable in a bias-free manner. Yet, there are still, of course, gray areas. However, there are now several ways to enforce fairness restrictions on AI models.
To resolve data issues, designers must carefully examine the accuracy and representativeness of training data, oversample minority groups, and eliminate features that contain human biases. Performance of AI with regard to minority groups [in terms of inclusion, fair consideration] has improved by adding more data points to text classification tasks. Increasing sample input data-sets to include more minority and vulnerable groups significantly aids in ensuring non-discrimination in AI results.
Further, novel approaches to training, such as transfer learning or decoupled classifiers for distinct groups, have shown promising results. Employment of such tools to ensure inclusion of minority groups has resulted in mitigating output discrimination and biases.
Lastly, methods created to tackle the related problem of explainability in AI systems—the challenge of utilising neural networks to explain how a specific prediction or decision was made and which characteristics in the data or elsewhere led to the outcome—can also be useful in detecting and thereby reducing bias
Human biases are inherent. As programmers and the sample data on which the AI algorithms are based are marred by biases, the resultant product is a tainted AI algorithm. The defected input produces defected output, plagued with discrimination and prejudice. This has been observed in the last few years across nations, industries and organisations. As the first step, therefore, it must be accepted that AI results are not necessarily neutrally infallible and that they might as well be biased.
To combat the problem of AI bias, researchers must increase sample size and include more data on minority groups to sensitize AI programmes. They should also devise new and innovative training mechanisms to tackle this issue. In addition to this, the governments need to catch up to the technological advancements and make necessary laws, rules, and regulations to keep AI in check and to ensure that AI bias is curbed and that it does not cause any ill-effects.
