3 Places to Think about Ethics in Tech and AI

We often talk about ethical AI and ethical tech as monolith, but the reality is that ethical issues can sprout in different places. Let's take a look at 3 of those places and what you can do.

Photo by Vlado Paunovic on Unsplash

We often talk about ethics in AI and technology as a monolith - as if the entirety of the system is either ethical or it is not. But that's rarely the case. Sometimes AI and other technology are unethical in their very design. Other times, the software itself isn't explicitly unethical, but the data that we feed into it ends up being unethical. Sometimes, even when the design is ethical and the data is ethical, the way a person or company uses that data is unethical. 

But we don't often call out the nuance. We make a statement that "AI is unethical!" without really calling out where it's unethical. In today's Byte-Sized Ethics, we are going to take a deeper look at the 3 different areas, and how to ethical considerations in each. 

Let's get started. 

Software Design / Model Design

At the level of software design, we look at ethical problems inherent in how the system was programmed, or how the model was adjusted and trained in the case of AI. There's a great example of this in

Sara Wachter-Boettcher

's Book, Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech. In it, she describes the problems of online forms. She talks about challenges with things as simple as entering your legal name, or challenges with selecting your sexual orientation, or even something as blatant as race. 

I talked about my own experience with something similar at the doctor's office in a previous article. Homosexuality was listed as a diagnosis in the electronic medical record software, (where you would typically list things like headache or asthma) instead of with the demographic information. It wasn't my doctor's choice to list it there, he was at the mercy of the software he was using. 

These are examples of ethical problems in the design of the software. My sexual orientation is a demographic, not a medical diagnosis to be cured. When the software was being designed and written, one person, or likely a whole series of people, made a conscious decision to design the software in that particular way. It didn't just magically work that way, someone had to make that choice. 

How to avoid ethical problems in software design 

Involve a diverse set of people early and often in your design process -- and listen to what they have to say. It's important to not invite them to the meeting and then discount or ignore everything they say. If you bring in a diverse set of voices, they'll help point out where your blind spots are and how to be more inclusive, and avoid ethical problems before they ever actually become problems. 

In the realm of AI, there are suites of tools available, like Lime for explainability, Biased Benchmark for QA, and BIG-Bench. 

Photo by Shahadat Rahman on Unsplash

Biased Data

Biased Data gets a lot of air-time - and for good reason! A bad dataset can tank an otherwise wonderful piece of software. There are so many great studies of really bad data, but there are two that I've found especially interesting. In Data Feminism authors Catherine D'Ignazio and Lauren F. Klein do a wonderful job of laying out the challenges with biased data, and spend the book laying out the principles of Data Feminism through lense intersectionality and existing case studies. Best of all - you can read the entirety of Data Feminism online for free at DataFeminism.io and I highly recommend it. 

The second book is Algorithms of Oppression: How Search Engines Reinforce Racism by Dr. Safiya Umoja Noble. In it, amongst a treasure trove of case studies and research, she talks about the now infamous controversy where Google Photos was accidentally identifying black men and women as apes. 

Why was this happening? Because the machine learning model Google was using was trained mostly on folks with light skin color. When it came across a photo BIPOC person, it didn't have any training data to tell it that they were people. 

How to avoid ethical problems with biased data?  

 First, acknowledge to yourself that no matter how well intended, you will never be able to eradicate all bias in your data (and there's a strong case for why you probably don't want that anyway, but that's for another day.) After you get over that hurdle, remember that adage, 'Garbage in, Garbage out.' The single most important thing you can do is ensure you start from good data. You should know how the data was collected, who it was collected from, where it was collected, and when it was collected. If you get garbage data, the best data science team in the world isn't going to be able to eradicate the harmful biases in your data. 

Red team your model. This is a relatively new concept, borrowed from cybersecurity where your team is the "offensive team" against cyberattacks--they are the people you hire to try and break into your systems so that you know where the gaps in your cybersecurity are. You can think of 'red teaming' your AI as a group of people doing everything they can to find the biases and ethically problematic results of the training data. 

How the Software is used 

You can do everything else right from an ethical perspective - you can design an ethically sound program, use high quality-data and red team to find the gaps, and then when you put it in someone else's hands, everything goes to hell. In my very first article on Byte-Sized Ethics, I wrote about Covenant Eyes, a software company that I described as unethical because of the usage of the software, instead of any problem with the software itself. In the article, I talk about how Parole Officers were using Covenant Eyes to monitor folks out on bond for compliance -- even though this usage was against their ToS. 

An Uncomfortable Look at the Use of Surveillance App Covenant Eyes by Law Enforcement

In that, there's nothing unethical about voluntary monitoring for those who want it, despite my distaste for that approach. But using that same software, knowing the limitations and weaknesses of the software, to monitor for law enforcement purposes with very real consequences not just for the monitored but also for their friends and family. 

When folks push back against using AI to identify people in crowds, either by scanning their faces or other means, they are pushing back against the usage of a particular software, not against the software itself. 

Photo by Scott Webb on Unsplash

How to avoid ethical problems in how software is used? 

This is rough because you might not always know that your software is being misused. But you can start by outlining prohibited uses for the software in your ToS or EULA, and then actually enforcing those violations - which will frequently mean walking away from customers and missing out on revenue. This is exactly what Covenant Eyes refuses to do - their greed has superseded their ToS.   

But also recognize that this might also mean not using software that you want because they don't have those prohibited uses in their ToS, or they sell to customers whom you know are abusing their software, and they don't care. 

Closing Thoughts

There are a lot more considerations and a lot more ways to mitigate ethical problems at various levels in your software - but most of them boil down to being proactive, and not reactive. Involve the right, diverse people early in the process and address issues before they become problems. Get good data and red team your models. Write your contracts so that you can prevent the worst uses of your software. 

Being intentional, deliberate, and inquisitive will go a long way to making sure you avoid ethical problems in your software and AI before they become problems.