I’m gong to start with the exciting announcement. Dr. KJ Haywood just launched her book, “Here We Go Again, Except It’s AI: AI Governance: From Analysis to Enterprise Action,” and I’m a contributor to the book! Dr. Haywood is amazing, and I’m excited to get my own copy of the book. Check it out on Amazon today! Dr. Haywood will be launching it in other stores soon, if you prefer not to support Amazon.

Disclosure: While I was a contributor to the book, I have no financial stake in it. I don’t get any kickbacks if you purchase the book. I’m just excited to share Dr. Haywood’s work and my first credited book contribution.

A Short Update

It’s almost 9 months to the day since I last published. There are a lot of reasons for it, but the biggest one is that I fell out of the cadence of writing on a regular schedule, and I struggled to get back into it. Part of it was my day job, part was the shifting state of political and ideological trends, and part was my problem with Substack’s ideology as a platform. That, and I couldn’t decide what I wanted Byte-sized Ethics to actually be.

Despite the gap, I’ve missed writing. I am going to continue Byte-sized Ethics, but I need to re-envision what it is. Those of you who’ve been with me for a while know that I’ve struggled to find a focus and a niche. I believe in the idea of Byte-sized Ethics — exploring the ethics, values, and morals reflected and expressed through our technology and software. The focus on AI Ethics felt too narrow, and I found myself reacting to the hype, not contributing meaningfully to the conversation. But when I expanded the focus, I started asking Where do I stop? The line between what we consider technology and don’t is getting increasingly blurry, so what are my boundaries?

What I’ve been thinking about lately is that I want to explore how to use software, AI, and technology, and still be a good person. It keeps getting harder to determine what companies and software are ‘good’ or aligned to my values, morals, and ethics. If it’s hard for me, someone who thinks about these things all the time, it has to be hard for others, right?

Over the last few months, I’ve also gotten really into Stoicism, thanks to an impulse purchase of a Ryan Holiday book - “Stillness is the Key.” I’ve been thinking about the application of stoicism to AI, software, and technology as well. There could be a really interesting intersection between the two topics.

That’s the direction I’m thinking about - how do I use software and technology and still be a good person? Or at the very least, make informed choices.

What’s Next?

I need to spend more time thinking about where Byte-sized Ethics is going, and then come up with a content approach, and then start writing. I don’t want to put a date on that because there’s a lot of unknown between here and there.

In the interim, I’ll write when I have the opportunity. I have a backlog of topics to cover; I just need to do the actual writing.

That’s all for today, folks. As always, I appreciate you. Thank you for staying with me on this journey and taking the next steps with me.

Before this week’s article, I want to take a moment and celebrate breaking 1,000 subscribers! I am so humbled by the number of people who want to read what I write, and I am so excited to keep writing! Thank you to each of your for your support!

In this vein, I wanted to get your opinion. If you don’t see your option, drop it to me in the comments!

When you last played a video game, did you stop to chat with the merchant NPC about their hopes and dreams? Of course not—because they don't have any. They exist solely to sell you items. This deliberate simplification works perfectly in games but fails catastrophically when applied to real people. Yet this is precisely the challenge facing artificial intelligence: it can only understand humans through proxies—simplified models that flatten our messy, contradictory nature into manageable categories. This fundamental limitation may be AI's greatest blind spot.

Proxies Flatten People

NPCs are useful in video games because they are flattened. They don't have depth, character, wants, or desires. They fit cleanly into the buckets the game developers created for them: merchant, enemy, quest giver. The merchant is just a merchant--her entire existence is wrapped up in that single role. And that's perfect for a game environment.

But real people aren't that simple—our identities are messy and we refuse to stay in pre-defined buckets. Consider stereotypes: do you know anyone who perfectly embodies every characteristic of a stereotype and nothing outside of it? Of course not. A stereotype functions like an NPC—recognizable fundamentally unlike a real person.

When humans are represented as proxies or stereotypes, everything that doesn't fit is ignored. We are flattened—the complexity of identity brushed aside in favor of something easier to understand but ultimately incorrect.

Proxies add in things people aren’t

A proxy doesn't just remove complexity--it adds characteristics we don't possess. If I say that I’m gay, people might assume that I am fashion-conscious. If I say that I’m a nerd, they might assume I’m not physically fit. If I say I’m an athlete, they might assume I’m not interested in intellectual pursuits.

But I'm all of these things...and none of the completely. I am gay but decidedly not fashion-conscious. I'm a nerd who loves physical activity. I'm an athlete who writes about AI ethics. I embody parts of these identities while violating their stereotypes in other ways.

Proxies force zero-sum thinking.

Proxies create a false binary—you either embody all aspects of a category or none. This is precisely how data scientists often approach classification problems: distilling complex realities into simple yes/no decisions that make analysis cleaner but less accurate.

We experience this daily. Think about bizarre targeted advertisements that completely misunderstand your interests. Or consider political discourse where belonging to a party supposedly means supporting every policy position without nuance.

We Confuse the Map for the Territory

The fundamental error is mistaking the proxy for the reality it represents. When some people think "gay man," they immediately assume a collection of stereotypical attributes that may not apply to any individual gay man. The rich universe of individual variation gets ignored because it breaks the proxy.

While humans can update our understanding when confronted with contradictory information, AI cannot. For artificial intelligence, the proxy is the actual thing, not a representation.

The problem for AI with proxies

This is the critical limitation: humans can adapt because we experience the reality behind the proxies. AI cannot—it only ever knows the proxy, never the reality that informed it.

If proxies are inadequate for representing human experience even for humans, how can we expect AI to understand the human experience when all it has to work with are these flawed representations?

This isn't just a technical problem—it's a fundamental constraint. As we increasingly rely on AI to make decisions about people, we must recognize that its understanding will always be limited by the proxies it uses, and that those proxies will never capture the beautiful messiness of human identity.

So how close can we really be to ASI, or even AGI?

Subscribe

I’m trying something a little different today - I’m writing from my mobile because I’m not in front of my computer. My texting prowess is better described as prow-less, so expect some creative grammar and spelling.

After writing about Inworld.AI last week, I spent a little (translation: a lot) time thinking about NPCs and what it would actually be like to have them be more multidimensional. I started asking myself do I even want that as a player? Imagine a nightmare scenario where I say something that pisses off the NPC and then he would refuse to give me the thing I need to progress. That nonsense happens all the time in real life, why do I want that in a game?

The truth is, I don’t. NPCs are one dimensional character for a reason - they are a proxy for a real person, without all the messiness that comes with being a person. It’s beautifully simple. It’s great, the NPC serves its purpose and then I move on. As a stand-in for human interaction, it’s kinda great because people are a lot.

Naturally my internal monologue decided to up the ante and drew a comparison to proxies in data science. In data science, a proxy is a stand in for something the data scientist can’t directly measure. For example, I can’t determine someone’s income directly, but I can look at where they shop to guess about their household income. It’s a simplified, one dimensional representation of something that’s complex.

But here's where it gets interesting: data scientists rarely use just one data point to build a proxy. They're looking at multiple factors to triangulate what they want to know. It's not just where someone shops - it's where they live, what car they drive, whether they have kids, the whole nine yards.

Pretty useful, right? We can infer all kinds of things using proxies. But what happens when you use proxies to create more proxies? Maybe my “do you have kids” data point is actually a proxy made up of a whole bunch of other data points - and so on and so on.

Eventually, the quality goes off the rails because your proxy becomes so far removed from what it's supposed to represent. It's like playing a game of telephone with data - by the end, you're left with something that barely resembles the original message.

Let’s go back to our friendly NPC. We know they are proxies of proxies of people. But they are so far removed from the “source” (aka us, real flesh-and-blood people), that while we can recognize them as the thing they represent, we’d never mistake it for the real deal. The NPC is a proxy of a person, that is a proxy of a type of person that fits into this virtual world, that is a proxy of a designer’s understanding of a real world group of people that they’ve only ever read about, which is yet another proxy … it’s an endless web.

So our NPCs end up being these fascinating patchwork creatures - recognizable as human-ish, but fundamentally incapable of replacing what they represent.

So, then I have to muse: since GenAI is built on the same principle - an amalgamation of proxies cobbled together to simulate consciousness - does that mean it's also inherently limited? Has it become so far removed from what it's trying to represent that we could never mistake it for the real thing?

Maybe that's not such a bad thing after all.

Did you know that for only a few dollars per month, you can help support my work here. If you have the means, please consider subscribing— and as always share with you friends!

Subscribe

Well, it didn't take long for me to miss a week. Sorry about that! I've learned two important lessons: Even if I maintained a habit for 6 months, it takes effort to rebuild it. I've also learned that it is not the best idea to wait until Sunday to write and publish that's my recovery day, and my motivation is low. I will be experimenting with my publication days to determine what works for my readers, so expect some changes in the coming days and weeks.

One of the things I wanted to do differently was to be less of a doom-and-gloom blog and start covering cool ethical* uses of AI. I'm excited to start this trend this week, where I spotlight Inworld.AI and their agentic AI for NPC dialog in video games.

I'm an avid gamer. Specifically, I play a lot of Massively Multiplayer Online Games (MMOs), like World of Warcraft. I love playing games without "winning" scenarios, and I enjoy the community aspects of virtual worlds. But virtual worlds can't only be populated by other human players, the game needs NPCs who provide information, send you on quests, and sell items to make the world feel real and lived in. Those are called Non-Player Characters, or NPCs for short.

The persistent challenge with NPCs is their repetitive dialogue. NPCs say the same things, from a predefined set of responses. For decades, interacting with an NPC involved clicking on it and selecting from a list of predefined interactions. That's remained unchanged for 40 years, give or take, until now.

Thanks to genAI, and a company called Inworld.ai, that's starting to change. Inworld.ai offers agentic AI solutions that enable gamers to have more natural conversations with NPCs, deepening and enhancing game immersion.

Inworld describes several potential uses of agentic AI such as creating unique companions to accompany players in single-player games, and simulating human tactics in single-player games to make it seem like you are playing with people, but without the potential for extreme bigotry and extremism recruiting. Beyond individual NPCs, Inworld says their agentic AI can be adapted to create dynamic environments that respond to the player journey in many different ways such as helping the player through the narrative by creating on-the-fly cues or adjusting difficulty.

Why is this a cool use of AI?

Any gamer will tell you that hearing the same phrase a thousand times gets old. The opportunity to have a real conversation with NPCs, beyond the typical pre-defined list of interactions is really exciting. Beyond individual interactions, a gaming world that responds to you and your unique experience in the game opens up whole new worlds of possibilities that we haven't seen in gaming before.

As a lifelong gamer, I've been excited about the potential for genAI within video games since ChatGPT exploded into public awareness. I'm a little disappointed at the slow adoption of these technologies into games, but given how expensive genAI development can be, I'm not necessarily surprised. I am very excited about the potential that genAI offers for gaming.

Bonus. Content: If you google the phrase, "But then I took an arrow in the knee," you'll see the most iconic and meme'd NPC phrases ever. It's from the game, The Elder Scrolls V: Skyrim, and over a decade later, it's still seared into my mind.

* Ethical Disclaimer: There are two pieces to this disclaimer: First, I define “ethical use of AI” as one that doesn’t materially harm any group of people and solves a problem that we can’t realistically solve using traditional means.

Second, this article isn’t sponsored. Inworld.AI doesn’t have any input on the content of this article, I’m receiving no compensation for this article, and Inworld.AI isn’t aware that I’m writing this ahead of its publication.

This is just me geeking out about a cool thing I found.

Do you have a cool, ethical use of AI you want me to feature? Let me know - drop a comment or send me a message!

Leave a comment

Share

Well, January 2025 is in the books. It might be a book that we'd rather forget about and maybe throw into the fire, but it's done. It took me some time to determine what to include in this round-up. Not only has a lot happened in the last 30 days, but this is also my first round-up in 6 months. There was a lot to sift through.

AI Safety Newsletter

AI Safety Newsletter #46: The Transition

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required…

Read more

a year ago · 20 likes · Corin Katzke and Dan Hendrycks

As much as I want to stay away from the U.S. political ... tomfoolery, it was impossible to ignore all the change last week. I struggled to keep up, but luckily the AI Safety Newsletter was there to help. The US has been thrown into regulatory chaos over the last week, and AISN does a good job of breaking down what's in and out at the Federal level. I'm sure we'll see additional AI shenanigans at the Federal level over the next months and years.

Just over a week ago, no one had ever heard of DeepSeek and now it’s all anyone can talk about in the AI space. After wiping out $1trn in value from the AI market, DeepSeek's emergence onto the world stage has spawned critical conversations from just about everyone. I've grabbed three articles that I found interesting and show the complexity of claims about how much DeepSeek did or didn't spend on developing the model. The first article about DeepSeek is from a Publication also called DeepSeek. As far as I was able to determine, they are not the same entity. But take with a grain of salt just in case.

DeepSeek

🚨 OpenAI vs. DeepSeek: A Landmark Battle Over AI Knowledge Distillation and Intellectual Property 🚨

In a surprising turn of events, OpenAI has publicly accused DeepSeek of illicitly distilling knowledge from its proprietary models. This move not only raises critical questions about the boundaries of AI development but also sets the stage for a potentially groundbreaking legal showdown. Here's what every AI enthusiast and professional needs to know…

Read more

a year ago · 5 likes · 1 comment · Oxy

Second, we have an article from challenging some the claims about cost and performance, and reminding us to be critical of what we hear and the panic and hype.

Tech and Democracy

DeepSeek or DeepFake?

In the high-stakes world of artificial intelligence, geopolitical competition between the United States and China has intensified. The debate over whether AI development should be open or closed is now central to broader discussions about national security, innovation, and …

Read more

a year ago · 1 like · Paulo Carvao

Lastly, we have of expressing similar concerns about merit of DeepSeek’s claims, but taking it step further to also dig into the U.S. market (over?) reaction, the impact of export controls on AI.

The Lee Show

DeepSeek

The recent release of DeepSeek’s new AI model, DeepSeek R1, has sent shockwaves through the equity markets, with NVIDIA (NVDA) and other U.S. technology leaders experiencing a significant sell-off. A…

Read more

a year ago · 3 likes · Lee Bressler

As an AI Governance Enthusiast (if such a thing exists), I was excited to discover 's 25 Practical Tips for AI Governance. Many organizations see AI Governance as an insurmountable problem, and it's relieving to see pragmatic advice. Bonus points for following Patel on LinkedIn because he always posts great content.

Enterprise AI Governance

25 Practical Tips for AI Governance Leaders in 2025

Hey 👋 I’m Oliver Patel, author and creator of Enterprise AI Governance…

Read more

a year ago · 8 likes · Oliver Patel

In the same vein, after getting over the scariness factor, the next major hurdle to getting an AI governance program off the ground is funding. I found this article from from to be helpful. I've gone through the process of pushing the business case for AI governance, and it would have been much easier with a resource like this article.

The Company Ethos - Doing AI Governance

What's your business case for AI governance?

In my previous articles, I explored why high-integrity assurance of AI matters and the failures it can guard against. Organisations like GM Cruise demonstrate how gaps in AI governance can cascade from technical issues into existential business risks. But understanding why governance matters isn't enough — we need to build compelling business cases that…

Read more

a year ago · 1 like · James Kavanagh

If you’d like me to share one of your articles, don’t hesitate to reach out! I’m always open to building my network and talking to fun people!

Last week, I wrote about why you might need to stay on Meta's platforms and some actions you can take to stick it to Meta even as you remain on Facebook, Instagram, Threads, or WhatsApp. This week, I'm tackling how to leave Meta.

Leaving Facebook, Instagram or any of Meta's platforms is hard. Facebook has been in our lives for a long, long time, so it's not always easy to quit even when you want to. It can feel scary to think about social media after Meta. How do you do it, how do you leave and actually stay away? You need an exit strategy.

An exit strategy will help you organize your thoughts, prepare for your post-Meta life, and create a sustainable roadmap to resisting the temptation to return.

There are a few key components of your exit plan that you'll want to be sure you have to set yourself up for success: Timeline, Destination, Communications, Grab your Data, Set yourself up for success, and de-platforming.

Timeline

Your exit strategy needs a timeline and a deadline. You shouldn't immediately deactivate or delete your accounts. You need to determine when you want to deactivate or disable your accounts. The deadline for the exit strategy should be in 2-4 weeks to ensure you maintain momentum. Write down your date, and place it somewhere prominent that you'll see daily.

Decide on a Destination

Next, you need to decide where you are going. There are many options, so you have to identify what's most important for you in a social network. Write out your Top 3 most important features for a social network. Be realistic - no platform will perfectly recreate Meta's ecosystem. You will need to compromise.

As of this writing, most of the alternative social networks exist on the Fediverse which runs an open source protocol to be able to talk to each other. This can be really confusing at first, so I recommend you read up on it here. Here's a list of alternatives you can check out:

• Facebook: MeWe, Tribel, Friendica, Vero, Minds, Diaspora

• Groups/Community: Discord, Reddit, Signal

• Threads: Bluesky, Mastodon

• Instagram: PixelFed, Flashes

• Messenger/WhatsApp: Signal

Leave a comment

Give Notice

Give your networks a heads-up t

hat you are leaving. Write a post to briefly describe that you are leaving, why you are leaving, when you are leaving, and alternative ways to connect with you. 1 to 2 weeks is enough time for most people to see and respond if they want.

Data Preservation

During this time, you can also download your Facebook and Instagram content. These downloads can take time--several hours to several days. Start the process as soon as you can, then review the data when you receive it to make sure you got everything you wanted.

Habit Replacement

This is the hardest part and the most important part. You will find yourself itching to check Facebook, Instagram, or Threads, usually when you are bored and have some downtime. Meta has been a daily habit for many people, and you need to break that habit.

Start by identifying the triggers that have you reaching for your phone scroll Instagram. Then come up with something else to do when you get the urge to doomscroll -- maybe you meditate, read, or send a text. Whatever it is, make it easy and make it rewarding.

Delete or Deactivate

You've decided on your deadline, you planned out your next steps -- where you are going, letting people know you are leaving, you've downloaded your data and come up with a habit replacement. Now is the time to decide: do you delete, or deactivate?

You have to decide at this point whether you want to Deactivate or Delete. If you are unsure about whether you want to come back to Meta products -- maybe if they get less racist and bigoted in the future -- you probably want to Deactivate. If you are committed to not returning or don't mind coming back with a clean slate, you Delete. You might also opt to delete if you are worried about giving in and falling into the temptation. Deleting makes it much harder to get back on the platform and in your networks again.

If you are unsure about Deleting, I recommend Deactivating for right now. You can always come back later and delete, but you can't go from deleting to deactivating.

That's it. Leaving Meta requires intentionality, planning, and self-awareness. By having an exit strategy, you can successfully transition away from these platforms while maintaining your social connections and mental well-being.

Meta has never been welcoming to diversity on their platforms. Sure, they tolerated minorities because advertisers wanted to be able to target them. But they made no effort to entice or incentivize those groups of people. Mark Zuckerberg's leadership has consistently demonstrated questionable ethical standards, with the company's pattern of dismissing serious concerns and maintaining insincere commitments to diversity, equity, and inclusion programs, content moderation, and codes of conduct. In D&D terms, he floats between Lawful Evil and Neutral Evil.

When Zuckerberg rolled back those programs, particularly trans and LGBT+ protections, back for dubious reasons, many people were disappointed, but not surprised. This fits the model that Meta has always operated in. So now that we are here, in this predictable yet disappointing place, what do you do? Your choices boil down to the same two options that you've always had - do you stay, or do you go?

Why are people leaving?

The short answer is that Facebook is becoming even less safe for minorities and it will be harder to determine truth from misinformation and disinformation. The new policies that Meta announced explicitly allow for hate speech under the sheep's skin of "freedom of expression." They instead target "hateful conduct" which allows for speech that is traditionally considered hate speech. In addition, they are discontinuing their fact-checking program in favor of "community notes".

This approach mirrors changes X implemented when Elon Musk took over. X has been on a rapid decline since - both in revenue and users. X has become a hostile, toxic environment with widespread discriminatory content targeting protected classes. Many people expect a similar deterioration on Facebook and Instagram and are leaving the platform.

Reasons to Considering Staying on the Platform

Light of these changes, why would anyone maintain a presence on Meta's platforms? There are a few common reasons:

1. Professional Necessity: Some people depend on Facebook or Instagram for their livelihood. It's not a question of "Do I want to leave Meta products?", it's a question of "Do I want to leave Meta products, or do I want a roof over my head and food to eat?"

2. Unique Connections: These platforms may be the only way to maintain some relationships.

3. Sparks Joy: It just makes you happy. Something about these platforms brings you joy you can't find anywhere else.

For those folks hanging around, here are a few things to keep in mind:

1. Fewer people: You should expect fewer people on Meta's platforms. Fewer people means less engagement. So if you are

2. Unique connections might bail.

3. The new content policy will increase your exposure to harmful and hateful content.

4. Recognize the challenge of distinguishing reliable information from misinformation.

Taking Action While Remaining on the Platform:

Meta makes money off of its advertising and enables targeted ads by looking at your behaviors across the web. You can continue to use Facebook and Instagram, but make your data less valuable to Meta.

The Electronic Frontier Foundation published steps you can take to make your data less valuable to Meta and hurt them in the dollar signs. Check out the link for specifics.

1. Disable the use of targeted advertising. There are several places you need to disable targeted advertising. Make sure you get all of them.

2. Modify Mobile App Usage: Meta generates mountains of data from you through their apps on your mobile devices. The best thing to do is to uninstall the apps. But if you aren't ready to commit to uninstalling the app, limit the app permissions like location for example.

3. Take a Stronger Privacy Stance: Meta tracks you everywhere across the web, whether you know it or not. To make this harder, use a privacy-focused browser like Firefox or Brave. You can also install free browser extensions, like Privacy Badger or uBlock Origins.

Following these guidelines and using these tools, you can significantly reduce their ability to monetize your data. Next week, I'll look at how to leave Facebook and Instagram.

Subscribe now

Share

Well, here we are in 2025. Saying "I fell off the horse with regular writing" doesn't quite capture it. It's more like I fell off, watched the horse gallop a quarter mile away, and stood there dazed, wondering where I was. What can I say? 2024 was one hell of a year.

Subscribe now

For 2025, I'm getting back in the saddle with a more sustainable content calendar. As many of you know, I typically write longer pieces—which I love doing—but staring down 1,000 words can be intimidating. So I'm adjusting my approach:

• One long-form post per month, diving deep into responsible AI, AI governance, and other complex topics that deserve thorough exploration

• One medium-length post monthly, covering AI case studies, notable news stories, and topics that can be effectively addressed in fewer words

• Monthly round-ups highlighting work from other Substackers and relevant sources across the web

• One short post per month exploring the fun, positive sides to the AI revolution.

One thing I want to change from last year: BSE became more doom-and-gloom about AI than I intended. While we need to address AI's challenges, there are also exciting developments worth celebrating. I want to strike a better balance in covering both perspectives.

You can expect the first article under this new format this week. I'm excited to share it with you.

Now, about 2024—what a journey it was. Late 2023 brought what seemed like my dream opportunity: launching an AI Governance and Privacy program at my company as we prepared to spin out from our parent company. After years of working toward this goal, it was finally happening. Then, just months into 2024, plans changed. The spin-out was canceled, my position became redundant, and I was laid off.

I got lucky, though, and I landed a new role as a Director of Product. While not in privacy and AI governance, I remained in cybersecurity. I love my new company, but I still miss those six months in my dream role.

During this professional hiccup, my husband and I decided to act on years of discussions about moving somewhere with better weather. The layoff became our catalyst, and we relocated four states and 600+ miles away. Between losing a job, starting a new one, and making an interstate move, writing naturally took a back seat.

But now, with all that behind us, I'm ready to return to writing. As always, thank you for reading and subscribing. I appreciate each of you, and I'm looking forward to making 2025 a great year together.

What do we mean when we say "artificial intelligence?" A colleague and I recently wrangled with this question while we were trying to update a few legal documents at work. It's not that we don't know what artificial intelligence is - we do. But it's challenging to capture what we mean when say "artificial intelligence" in a few words. As my colleague and I worked through various definitions, I lamented that my dumb calculator would satisfy some of these definitions, and then that my Furby was, for all intents and purposes, a HAL-level AI based on these definitions.

Thus was born the Calculator Test and the Furby Test

Other contenders from my snarky commentary were, in no particular order:

1. The Tamagotchi Test

2. The Roomba Test

3. The "Took an Arrow to the Knee " Test

4. The Super Fancy Bidet Test

5. The robot-mouse cat toy that my cat murdered in 3 seconds flat Test

These tests are dual-purpose. First, they provide you with a framework for evaluating your definition of AI, or any other definition of AI based on real-world, well-known examples from recent-ish history. Second, more importantly, it challenges your framing of how you think about whether your software solution qualifies as AI under various laws, or doesn't. Drawing parallels between a calculator, a Furby, or any other similar style toy or device will allow you to evaluate your own machine-based systems more robustly by challenging your preferred framing.

Who knew that a furry little demonic toy from 1998 would teach me something about Artificial Intelligence policy today?

Why the Calculator and the Furby?

I'm going out on a limb and assuming that everyone knows what a calculator is, but not everyone might know what a Furby is. For anyone who doesn't know what a Furby is, it's a Robotic toy launched in 1998 that claimed to "learn" English from interacting with people, could interact and "learn" from other Furbys, and was actually banned from the NSA over concerns of it leaking Top Secret information. It's a cute, furry owl-hamster-demonic-entity thing that for some reason I absolutely love ... for whatever that says about me.

Honestly, the Calculator Test is pretty utilitarian and uninspired, but it gets the job done. A calculator is one of the most basic functions a computer machine can fulfill - basic math. So, if we can apply a given definition of AI to a calculator and have it reasonably (and this word is doing a lot of heavy lifting here) satisfy the requirements, then the definition is too broad. If it can applied to a calculator, it can be applied to every single piece of software ever conceived, which isn't great. If you satisfy the Calculator test, it's time to back to the drawing board.

The Furby Test is more stringent. The toy responds to inputs and gives the illusion of learning but doesn't actually utilize any modern machine learning techniques as we consider them today. (I'm referring to the 1998 version of the toy, the 2012/2023 versions are more sophisticated). But knowing that it responds to inputs and gives the illusion of making decisions and learning, if your definition of AI satisfies the Furby Test, meaning that your definition would include the Furby, but doesn't satisfy the Calculator test, you are on the right path and likely just need to tweak some things.

Why the Furby? Like I said, I kinda love the stupid little things. But they also represent a really important mix of responding to human and robot interactions and convincingly imitating AI-like functionality. They aren't something that we consider when we talk about artificial intelligence today, but they bear all of the hallmarks of artificial intelligence.

These two tests validate whether a given definition of AI is too broad, and by how much. It works like this: if I read a definition of AI and can reasonably make that definition apply to a calculator, the definition is too broad. If I can read a definition and make it apply to the 1998 Furby, but not the calculator, it might be too broad but it's definitely on the right track.

What is a legal definition?

Before we talk about these tests, let's start with why they can be helpful as we try to navigate legal definitions. A legal definition is how something is defined within a piece of legislation and it determines what's covered in the legislation and what's not. Most pieces of policy will have a definitions section, where they spell out and define all the key terms through the legislation. For example, gambling also has a legal definition, based on the three pillars of "chance, consideration, and prize," and that definition is how some sleazy video games get their commonly understood as gambling tactics to avoid actually being legally defined as gambling. It’s also important to remember that a legal definition will differ from a popular culture definition.

Why does this matter AI? We want to make sure that definition captures only what we intend it to, and not the myriad of AI-like things. If we are sloppy with our definition of AI, we could accidentally have our agreements and laws cover types of software we don't intend, or miss a piece of what we do intend to cover. Like gambling, we don't want folks to be able to skirt the law with AI through sloppy definitions.

A Common AI Definition

So let's try it out. Let's start by taking a common definition of Artificial Intelligence from Coursera, "Artificial intelligence (AI) refers to computer systems capable of performing complex tasks that historically only a human could do, such as reasoning, making decisions, or solving problems."

That's straightforward, right? Let's apply the Calculator Test.

1. Is a calculator a computer system? Yes, I think it is. The definition doesn't give us a lower bound on the capabilities of the system, so I can reasonably say it's a computer system as defined here.

2. Does it perform complex tasks that historically only a human could do? Maybe? We don't know what "complex tasks" actually refer to here, and it could easily be argued that even a dumb calculator is capable of finding the answers to complex mathematical problems that historically required a human to complete.

3. Does its task include reasoning, making decisions, or solving problems? Yes, it does solve math problems for us.

VERDICT: FAILS the Calculator Test because a calculator can reasonably be said to satisfy all 3 criteria of the definition.

Since we know the Calculator qualifies for the definition, the Furby definitely will so there's no need to run the Furby test until we can pass the Calculator test.

Take it Up a Notch with actual legislation

So let's take a look at how some laws have attempted to define artificial intelligence. The US National Artificial Intelligence Initiative Act (15 USC 9401(3)) defines AI as, "a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments. Artificial intelligence systems use machine and human-based inputs to perceive real and virtual environments, abstract such perceptions into models through analysis in an automated manner; and use model inference to formulate options for information or action." This definition has 3 criteria:

1. A machine-based system

2. that can, for a given set of human-defined objectives

3. make predictions, recommendations, or decisions

4. influencing real or virtual environments

5. use machine- and human-based inputs to perceive real and virtual environments,

6. abstract such perceptions into models through analysis in an automated

   manner

7. use model inference to formulate options for information or action

The "model" here refers to the AI model, which is defined in the same Act as "a component of information system that implements AI Technology and uses computational, statistical, or machine-learning techniques to product outputs from a given set of inputs.

The Calculator Test

1. Machine-based System - no guardrails, so yes.

2. that can, for a given set of human-defined objectives - Debateable here, but I'll say yes under the logic that entering the equation into the calculator is a human-defined objective

3. make predictions, recommendations, or decisions - I'm struggling to say that the output for a mathematical equation is a prediction, recommendation, or decision. I'm going to say a calculator doesn't satisfy this requirement

4. influencing real or virtual environments - A calculator is a siloed system, so it doesn't have any influence on a real or a virtual environment.

5. use machine- and human-based inputs to perceive real and virtual environments - This one is difficult because some virtual environments can only be expressed by equations and mathematical formulae. I'm going to say it doesn't satisfy the criteria for this because although there are human-based inputs and virtual environments can be expressed through mathematical means, I don't think the calculator can express those.

6. Since the calculator doesn't satisfy this requirement, it cannot satisfy the following 2 requirements either.

VERDICT: PASS - A calculator does not fit this definition of artificial intelligence.

The Furby Test (woo I'm excited!)

1. Machine-based System - no guardrails, so yes again.

2. that can, for a given set of human-defined objectives - I think the Furby achieves this with flying colors because it is more sophisticated, and its objectives are codified into its literal code

3. make predictions, recommendations, or decisions - This one is iffy. I'm stuck on what we mean by "decisions" here - if it refers to a pre-programmed decision based on inputs, then Furby satisfies this requirement. However, since it doesn't call out inferred or pre-determined decisions, I'll say that Furby satisfies this criterion as well.

4. influencing real or virtual environments - The 1998 Furby had limited perceptive capabilities, as we see from the sensor array between its eyes. Because it could communicate with other Furbys and 'teach them things', I'm going to conclude that's satisfying a virtual virtual environment. Furby satisfies this criterion.

5. use machine- and human-based inputs to perceive real and virtual environments - Weirdly, I think Furby satisfies this criterion with flying colors. But I think it's because of a loophole - the definition doesn't require that machine- and human-based inputs be directly tied to specific perceptions. So while the Furby didn't respond to specific words people said, it did respond to the noises that people made to make it mimic learning.

6. abstract such perceptions into models through analysis in an automated manner - This one is tricksy because we don't really know what the 1998 Furby was doing with the sounds it was processing. It responded to those sounds to some degree in an automated manner. Interactions between two different Furbys required perceptions of the other Furby, and then some processing of the "conversation" that the Furbys were having without any human interaction. I say Furby satisfies this criterion as well.

7. use model inference to formulate options for information or action - This is where I think Furby fails us. Whatever Furby was doing, I doubt it was using machine-learning model interference to formulate options for information or action. I think it probably used more traditional software development techniques to formulate options for information or action. There's a chance here that I could be wrong, but this would be a weak justification. Furby fails to satisfy this criterion.

Verdict: PASS - If it weren't for the specific callouts for model interference to formulate options, the 1998 version would have satisfied this criterion, which would have meant that the definition was still probably too broad.

Make It Your Own

Use this approach when you are trying to come up with your own definitions of Artificial Intelligence if you aren't using the EU AI Act definition or the US National Artificial Intelligence Initiative definition. But more likely, you'll want to use this as a model to apply to your own solutions to determine whether they satisfy the definitions and what that means for your legal obligations.

Either way, I just got to write a whole freakin' article tying Furby to AI legislation. Pretty damn awesome, if I do say so myself.

What about you? What tools or techniques are you using to evaluate AI definitions and your own AI solutions for regulatory compliance?

You might have noticed that things went sideways on Friday, July 19th. We woke up to half the world staring at what are affectionately called BSODs - or Blue Screens of Death. Cybersecurity gigantor Crowdstrike had pushed a "content" update that caused Windows PCs and servers worldwide to throw up the Blue Screen of Death. The BSODs disrupted entire countries, local governments, the whole airline industry, and countless others. As of this writing, over a week later, we still haven't fully recovered from the Crowdstrike Event(TM).

This incident shows us the risk of protecting our businesses and governments from cyber attacks. But more than that, it shows us how minute changes can have huge, dramatic, unintended impacts. This incident is a great wake-up call for the need for AI Governance. As we incorporate AI, especially GenAI, in more and more aspects of our lives and work, we open ourselves up to AI's own Crowdstrike Event.

A Brief Diversion…

Let's look at what likely happened from a cybersecurity industry veteran (me), what it tells us to expect from GenAI, and why we need AI Governance right now, and not in a few years.

As a veteran in cybersecurity product development, I'm intimately familiar with how Endpoint Protection (EPP) systems work. EPP consists of two main components:

1. Cloud-based Backend ("mothership"): This central system issues commands to endpoint agents, receives and processes security data, and alerts security teams to potential threats.

2. Endpoint Agent: A small program installed on user devices that collects data, prevents malicious or unauthorized programs from running, and can even restrict internet access. Recently, more threat identification and rule-based actions have been pushed to these agents, sometimes called "The Edge."

Crowdstrike referred to the recent issue as a "content" release problem. In cybersecurity, this typically means deploying a new rule or prevention technique to agents without updating the main software. It's similar to adding new photos to a photo app without updating the app itself.

Content releases are usually deployed automatically. The mothership sends new rules to agents, which download and implement them without user intervention or awareness. This system is efficient and usually works well—except when it doesn't, as we saw on July 19th.

So What Likely Happened?

At this point, what likely happened is pretty straightforward

1. Crowdstrike authored a new rule and deployed it to the whole world.

2. The rule was faulty and quality assurance and testing didn’t catch it.

3. Because the deployment was a content release, it didn't allow companies to choose when to deploy the rule or even know the deployment was happening.

4. The change went wide quickly, and the damage was done before anyone knew what was happening.

None of this is bad and that doesn’t mean Crowdstrike has bad QA. This is software development, bugs always happen. In endpoint security, it's even more difficult because there are millions of different combinations - it's impossible to test for them all.

It was dramatic, and caused many problems — but things like this happen every day, and in every piece of software just with less dramatic outcomes. This event was a big deal because of the extent of the impact - Crowdstrike is the cybersecurity market leader and everyone uses it. So when Crowdstrike went sideways, “everyone" who uses Crowdstrike went sideways too.

How does this relate to AI?

GenAI is still growing. We see new tools almost daily, and the foundation models are upgrading faster and faster. GenAI, for better or worse, is becoming a big part of our day-to-day lives. It’s not as big as the cybersecurity industry yet, but it's well on its way. OpenAI is a dominant player in the market, not unlike Crowdstrike. Businesses are pulling GenAI-powered tools into their critical systems and processes. GenAI is becoming embedded in a way that cybersecurity will never be. When something goes sideways with GenAI, the impact will be greater.

My educated guess is that we'll see a major failure from AI soon, probably within the next 24 months. I don’t know what it will look like - it could be a breach, and exploit, or the models could collapse. But it will fail eventually. Because this is software development and this is what happens.

GenAI has the perfect mix of lots of hype, lots of hubris, lots of grifters, and few guardrails as companies "race to be the winner of GenAI." That's a recipe for poor decisions, rushed testing and timelines, and risky development practices.

If I had to guess, I would posit that OpenAI will be the likely culprit when everything goes sideways.

How do you prepare for a GenAI Crowdstrike-style Failure?

AI Governance. Whether you create a dedicated role, add the responsibilities to your existing Privacy or GRC team, or engage outside help -- you need to get a handle on your AI usage now, not when the Crowdstrike-style software failure happens. AI Governance helps you:

1. Understand and document where you are using GenAI

2. Document what kinds of GenAI and which foundation models you are using

3. How GenAI is being used

4. What data can GenAI access?

Having these things identified and tracked allows you to make informed, deliberate decisions that account for the risk of a catastrophic failure.

Without an AI Governance program, your company will be on its heels when something bad happens with a GenAI model. You won't know where, or even if it's being used in your company. You will be busy trying to learn about yourself in the most stressful time possible -- trying to understand if you are impacted, then how badly, and then figure out what to do about it. If you have an AI Governance program, the first two points are easy to answer, and you can move straight to fixing the issue.

A few weeks ago I commented on a LinkedIn post from Allie Miller that got a lot of attention. I was surprised, contradicting LinkedIn posts about GenAI hype posts is something I do frequently, but something about this one resonated with people. Allie made a post titled, "Is your company on track with AI?" with a chart.

I commented on the post, arguing that such a chart created a sense of FOMO, and ignored the fundamental question: Does your business need AI? I had a lot of responses to my commentary (unfortunately, none from Allie herself), but more than most of my other posts. I thought, "This would make a great article."

It seems like all of my articles lately are inspired by LinkedIn interactions. Inspiration has come from stranger places I suppose. Using my one-sided interaction with Allie as a starting point, I will explore the reports from Goldman Sachs and others and how they all relate to a fundamental question: Is GenAI Worth it?

I characterize a lot of GenAI as a solution looking for a problem, and as a result, it doesn't provide real value. That's why I find LinkedIn posts like Allie's so frustrating -- it relies on creating a sense of FOMO (Fear of Missing Out) for companies that they are somehow 'behind the pack' without saying why. Posts that rely on FOMO make it seem like GenAI is intrinsically valuable. That just using GenAI creates value. It doesn't. GenAI is a tool that works well in some situations and bad a lot of others. It's akin to saying that I can hammer in a nail with an ice pick. I'm sure I could, but why would I? An ice pick doesn't work as well as a hammer. Many use cases of GenAI are trying to hammer a nail with an ice pick.

For GenAI to be worth it, you have to have a use case where AI provides value that you can't derive from other places. GenAI shines in processing a lot of data with low accuracy. Summarizing long documents, like privacy policies or helping security operations teams detect attacks and provide nuanced details about the attacks.

Additionally, there's a growing body of work that says that humans and GenAI paired together and working generate better outcomes, with less bias than either could alone. But that's expensive as the company has to pay for the employee and the AI to gain modest improvement in outcomes. For many companies, the increase in outcomes won't be worth the additional cost.

That's the big takeaway from my article--despite the hype, there are very few current GenAI use cases where GenAI is the best solution.

It's not a surprise that Goldman Sachs says that AI is too expensive and unreliable. It's unfathomably expensive to create, update, and maintain. Outside of a few big names, no companies are close to breaking even. Investors and VCs have been sinking money into anything with the letters "A" and "I" mushed together in hopes of a big payout, swallowing the hype hook, line, and sinker. Now, we are little more than 2 years into this journey as a society, and the misinformation problems are still there and just as problematic as before. Training AI models still costs astronomical amounts of money -- the cost to train models has increased -- with very little value to show for the investment.

Businesses aren't finding the huge productivity gains they were promised. It's telling that a Google search finds very few cases where AI increased business productivity and a lot of articles talking about how GenAI will increase productivity. The hype says the value is there, but the proof isn't in the pudding.

Does your business need GenAI? No, you don't. You might want it, but the chance that it revolutionizes your business is almost-zero because it is not solving a new problem, it is finding dumber ways to do the same old thing and selling you hype.

When Apple Intelligence was announced, I posted some snark on LinkedIn that it was entirely expected and entirely unneeded. Both in the fact that they announced it all and the fact that launched it more because they felt like they had to instead of solving any customer problems. Typing that out, I thought I should dig more into it, and the deeper I went the more meh I became. The features that I was excited about had very little to do with Apple Intelligence, and core Apple Intelligence use cases are kitschy (and I'm being charitable there).

Will Apple Intelligence change anything?

I'm going to begin at the end and answer the question: Will Apple Intelligence bring GenAI to the masses, making it indispensable in our day-to-day lives? No, and it's not even a question. Apple doesn't know what to do with GenAI anymore than anyone else does. It doesn’t do anything that hasn’t been done before. The greatest innovations from Apple Intelligence are the privacy tech and the improvements to Siri. Everything else is just noise and marketing hype.

Apple Intelligence is the first real attempt by big tech to put an LLM in everyday people's hands. (Technically Google did it first with Gemini, but no one is taking Google's AI efforts seriously.) Apple Intelligence is launching with the standard fare for an LLM - text and image generation.

And yes, it's every bit as kitschy as every other LLM.

Apple Intelligence Fails to Solve Any Meaningful Problems

Despite taking their time to get an AI product to market, Apple made the same mistakes as everyone else peddling AI solutions - the Apple Intelligence features are a solution looking for a problem. Text generation sounds interesting, but who pays attention to tone and grammar when texting or emailing friends and family? In a professional setting privacy and infosec concerns aside, the people who care the most about what they are writing aren't going to trust Apple Intelligence to write it for them. The people who it most benefits are the ones who shouldn't be using it in the first place -- they aren't going to review the output, they are just going to hit send.

Circling words on a page and generating an image based on those words makes for a good marketing story, but when is that useful? I think about the number of times I've been taking notes and said, "You know what would make this better? If I could doodle it out..." and that number is nigh nonexistent. This isn't a problem I, or anyone I know, has ever encountered.

The integration with ChatGPT is a nightmare. You should never use it. Full stop.

The features Apple is marketing sound cool and slick. But they aren't going to have an impact on your day-to-day because they aren't solving any problems you have.

The Impactful Pieces of Apple Intelligence

Contrary to the rest of this nonsense, the Apple Intelligence upgrades to Siri could be really useful. There's a real problem to be solved here in that Siri, for all her advances, can't handle natural language conversation. Apple Intelligence claims to move the needle, allowing Siri to have greater contextual awareness and understand the natural flow of conversations better. If Apple is bringing me closer to being able to yell at my house and issue a complex series of commands like I'm on Janeway's Enterprise, I'm here for it.

The other exciting part of Apple's announcement is one that most people aren't going to notice - the on-device processing and "Private Cloud Compute." Of all the major tech companies peddling half-baked AI solutions, Apple is the only one that's made end-user privacy part of their story. This is on-brand for them, as they made the confidentiality of women's health tracking in Apple Health a major talking point in the wake of the repeal of Roe v. Wade.

On-device processing is exciting because it's a win for privacy--instead of shipping your data off to some cloud system somewhere and hoping that everyone in that chain has good data handling (spoiler: they don't), your phone can handle the bulk of the work. We aren’t sure exactly what’s going to be handled on device, but we know it won’t be everything. Apple is still sending your data off their servers, but claims it's protecting your privacy using its "Private Cloud Compute."

Privacy continues to be a huge problem in AI. If Apple can deliver on its privacy-protecting practices, this could be a huge win for privacy-conscious users and businesses. While OpenAI, Google, and others are relying on hoovering up all your personal information to train the models, Apple's approach is refreshing. If they can pull it off.

I Have Concerns

That's not to say that I don't have concerns. First, no one has validated Apple's privacy claims. Apple Intelligence hasn't been audited to see if holds up its privacy claims, so we are left with Apple promising us it's doing what it's said it's doing. Historically, self-governing like this has not yielded good results for citizens, or companies.

Second, we are talking about a lot of data stored on device for all this processing, and I wonder how secure that data is. Our phones already capture and process unfathomable amounts of data, adding AI processing and storage on the device means even more data. I worry about how secure that is - if someone can hack into your iPhone, can they crack the AI data? I hope that Apple has put in the due diligence, but the Apple Intelligence launch feels rushed and I worry about the corners they cut to make this happen.

Finally, while Apple is making a lot of noise about Private Cloud Compute, we haven't seen it in action yet. As a newer tech, it will attract more adversarial attention and if it doesn't hold up to muster, Apple could end up with egg on its face, and users could end up holding the short end of the stick.

Wrap-Up

At the end of the day, Apple Intelligence is a Me-too move by Apple. They felt like they had to have something GenAI, and they slapped in the standard solutions of text and image generations without clear problems to solve. The most interesting parts of the announcement are about the under-the-hood changes to their infrastructure and Siri.

No, this isn’t a pivotal moment for AI, and Apple Intelligence won’t change anything. It’s another mostly forgettable entrance into a market that’s struggling to show it’s value.

Things are finally calming down enough for me to start writing regularly again. I've missed having the headspace to write weekly. But we're getting back there, and I'm excited about it. It's been quite a while since I've done a round-up, and now seemed like a good time.

is one of the first Substacks that I subscribed to when I joined the platform. I appreciate the balanced approach that Dr. Mollick brings--it's not fawning about the absolute good of AI or decrying it as the end of human civilization. He is more pragmatic. In one of his recent posts, he writes about doing stuff with AI. He starts by covering the fun stuff, like using models to create a song, and then he goes into the more serious usage, mostly in a business setting. One phrase he used stuck out to me, "always invite AI to the table." I'm not sure how I feel about it. It's pragmatic for sure, but also slightly anthropomorphizes AI as a thing that needs to be invited.

Check it out, and let me know how you feel about the phrase "Always invite AI to the table."

One Useful Thing

Doing Stuff with AI: Opinionated Midyear Edition

Every six months or so, I write a guide to doing stuff with AI. A lot has changed since the last guide, while a few important things have stayed the same. It is time for an update. This is usually a serious endeavor, but, heeding the advice of Allie Miller…

Read more

2 years ago · 312 likes · 25 comments · Ethan Mollick

Next up, I found this article from of titled "Understanding the real threat generative AI poses to our job." Merchant takes a more hopeful tone in this piece but highlights the risk that AI poses to creative disciplines. One point he made that I particularly liked--even if AI isn't used in the workplace, the threat of AI can still be used to suppress the working class. We don't often consider the indirect impact of AI, so the idea that just the existence of AI can be used to disenfranchise workers was interesting to me.

What do you think? Do we do enough to consider the impact that AI causes by just existing?

Blood in the Machine

Understanding the real threat generative AI poses to our jobs

Hello, and welcome back to Blood in the Machine: The Newsletter. (As opposed to Blood in the Machine: The Book.) It’s a one-man publication that covers big tech, labor, and AI. It’s all free and public, but if you find this sort of independent tech journalism and criticism valuable, and you’re able, I’d be thrilled if you’d help back the project. Enough…

Read more

2 years ago · 81 likes · 29 comments · Brian Merchant

One thing that Merchant Highlights is that creative jobs are particularly susceptible to replacement by AI. What I loved about this next article is that it makes the exact opposite point. of argues that execs should be the people who are worried about AI, not creatives. His argument centers around the value of the human creative process and intent in the creative process, which AI can't replicate. As the tsunami of AI-generated content hits the internet, people are going to value the unique, human-created art more.

He also uses the perfect phrase to describe AI usage today:

Let’s shoehorn AI into every technological crevice we can find. You know, the way mold grows insidiously in cracks in the pavement, or on the damp walls of your poorly ventilated bathroom.

What do you think? At the end of the day when AI fails to live up to the hype, who will be left holding the bag?

Write More with Simon K Jones

It's not artists who should fear AI

As artists, we’ve been living in a state of perpetual fear and uncertainty since 2022. Sorry, I meant since 1422. Maybe earlier. But that constant sensation of being an endangered species intensified dramatically in 2022 with the arrival of generative AI…

Read more

2 years ago · 141 likes · 67 comments · Simon K Jones

Finally, there's been furor about the (negative) role of AI in the upcoming US elections. Most of this furor has been around the ease of disinformation that can be created to change the narrative. of challenges this narrative, saying much of the doom and gloom hasn't materialized as everyone predicted. There's logic to this, many of these predictions didn't take into account our changing relationship with media as a result of the threat of disinformation that began in 2016 and continues to evolve today. People don’t accepted what they read online as fact as readily as they did before.

Is the use of AI a threat to the US election in the fall, or is it overstated doom and gloom? Is AI just making everything more efficient, as Micah says?

The Connector

Time to Calm Down About AI and Politics

Hello to all my new subscribers! If you’re here because you read the tome I put out last week about how race, class, and identity were playing out in my home congressional district, as incumbent Rep. Jamaal Bowman and challenger George Latimer clash in the wake of the Israel-Palestine rift among Democrats, be forewarned: The Connector is my forum for re…

Read more

2 years ago · 8 likes · 2 comments · Micah L. Sifry

The other day on LinkedIn, I stumbled across a post announcing that tools like NewHire.AI were the future of recruiting and hiring. Granted, some random person on LinkedIn announcing that so-and-so AI usage is the future isn't anything unique. Especially for me - my feed is full of hot AI takes, mostly exceptionally bad AI takes. But the way this particular poster framed the conversation, as if this new AI use case was a boon to job seekers, rubbed me the wrong way. I thought this would be a great case study to look at what seems like a good use of AI, but probably isn't. 

NewHire.AI Use Case

NewHire.AI is a single-page website a single paragraph and a demo video. The page title boldly claims to "Revolutionize Your Hiring with AI." The only content on the page is a single paragraph describing NewHire: 

"Conduct 10-30 minute phone screens at scale using your own AI recruiter. It will call a candidate, ask any job-related questions, answer the candidate's questions about the role and your company, and then send you AI summarized notes with a recording after each call. Start saving hundreds of hours and make better hiring decisions." 

NewHire isn't the only company doing this--there are quite a few companies using AI to solve this particular use case. The idea is this: As a candidate, I apply for a job, and I get my initial phone screen with an AI recruiter who sounds just like a human. We do the phone screen and AI generates a summary that it sends back to someone at the company to decide whether to move forward with me. As a candidate, I get to ask questions and get answers about the company to determine if it's a good fit for me. 

Technical Challenges  

There are a lot of issues with this approach. The first is the most obvious - LLMs hallucinate as a function of the model and are only correct by happenstance, not design. So as a company, you are entrusting a machine that has been shown to hallucinate as much as 50% to represent your company. If a human recruiter lied 50% of the time to candidates, how long would that person have a recruiting job? 

The flip side of the coin is that candidates cannot review and correct the summaries before the AI sends them to the company. The AI could easily lie about a candidate, either favorably or unfavorably, and the candidate would never know about it. They are at the mercy of an AI that frequently makes mistakes to properly represent them, their experiences and qualifications. Anyone who has ever used an LLM to summarize content knows that it never creates perfect content -- it still needs to be reviewed and corrected. Yet NewHire assumes that it can accurately reflect a summarized conversation. But if it makes a mistake, it can hurt candidates - and it makes mistakes frequently. 

Then, because this is an LLM, it's susceptible to adversarial prompting. For the LLM to be able to give a background and history of your company, you are going to have to train it on your company's data. A savvy hacker will know this and can mine the LLM for sensitive information to exploit the company. 

Or I could be a candidate who knows that the company I'm applying to uses NewHire, and I can look up the best way to answer questions or ask questions to get the most favorable summary and recommendation based on the strengths and weaknesses of the model. That erodes the trust in the tool to fulfill its use case. 

Socio-technical Problems 

What originally inspired me to comment, and then write this post was the statement that tools like NewHire are good for job seekers because it means they get to have more, valuable conversations. 

Let's be honest - that's horse shit. 

NewHire's website claims to be able to do "phone screens at scale." Because the cost of doing a phone screen drops with an AI screener, plus multiple phone screens can happen simultaneously, a company could screen many times more applicants than they could before. 

But regardless of how many phone screens there are, there's still only 1 job. So instead of screening 10 or 20 folks for a job, a company screen 100-200 folks for less cost than those 10 to 20. This is great for the company because they can screen candidates for pennies on the dollar. But it sucks for job seekers. 

The logical fallacy in the "getting a phone screen is good for candidates" is that the phone screen is the desired outcome. It's not. As a job seeker, I don't care about having great conversations. I care about getting a job. That's the only metric a job seeker cares about. 

As a job seeker, I don't care about having great conversations. I care about getting a job.

As a candidate , I get overwhelmed with phone screens with AI because the cost is slow. Whereas before only a subset of thge 'best' applicants would get a phone screen, with an AI tool many more applicants 'qualify' for a phone screen, even if I'm not as qualified. So instead of having a small number of 20-30 minute conversations, I now have a huge number of 20-30-minute conversations to complete, but the same number of actual jobs available. So I end up working even harder as more companies want to phone screen me for the same number of positions. NewHire shifts the cost of recruiting from the company to the candidate and declares a good thing for the candidate. 

If a company can't be bothered to dedicate 20 minutes of their time for me to talk with a real person, why should I be willing to invest in them? They've already shown me that they don't value me or the recruiting process, and they expect me to invest my the of time and resources to get a job with them, while they invest nothing. Is that the kind of image a company wants? They value their employees so little they would rather replace them with AI. 

NewHire doesn't directly say it, but they expect you to make hiring decisions based on the summaries their AI recruiters provide. In most cases, the decision is move on the process. I will be amazed if NewHire doesn't plan to provide a recommendation as part of their summaries. If they do, they will risk violating New York's AI Bias law, Colorado's AI Law, California's pending AI legislation, as well as the EU AI Act. 

There's a good chance that even just providing summaries of content without a recommendation can violate the regulation, as the summaries are expected to be used to determine whether to move candidates forward in the process. 

Speaking of regulation, many laws require an opt-out of AI function function that doesn't penalize the user for opting out. A candidate could opt-out of the AI for a actual person. But in that case, nothing stops the company from finding a reason that you aren't in consideration for the role anymore. Even though there will be required opt-outs from speaking with AI, users will still be punished for exercising the right because it's too easy to disqualify a candidate over any perceived 'legal' reason. 

Can They Fix It? 

The use-case of using an LLM to conduct a phone screen is riddled with issues, both technical and socio-technical. But is there a way that this could work while minimizing harm to candidates? Perhaps, but mitigating the harms to individuals lows the value to the company.

1. The biggest thing is candidates need to be able to opt out without penalty. I don't know how to do this, but it needs to be safe to request to speak to a real person. 

2. Candidates need to be able to update and rectify their summaries before they are sent to the company. There's too much risk of the LLM getting something wildly wrong to not give candidates the option to review and correct the summaries. 

3. NewHire needs to strike the "at scale" part of this. There has to be a limited number of candidates that can screened at one time. Anything else results in hundreds of phone screens for just one position. When done across hundreds of positions in many companies, candidates will be drowning in hundreds of low-value phone screens. 

4. Complying with regulations is going to be tough. Companies will need to ensure human-in-the-loop and defensible decision-making. In the strictest sense, providing a summary is not providing an explicit recommendation. But at the same time, the model is going to editorialize as it summarizes the conversation. That editorialization will skew the output of the summary one way or another and trip over regulation. Having a policy and a human-in-the-loop is the only way to mitigate this, but it's still high risk even with that. 

5. Company reputation isn't something you can fix when you using a tool like NewHire. No amount of wordsmithing will do anything to mitigate the reputational impact to the company. Companies will either need to accept the reputational hit and know that implementing this tool will drive away quality candidates or not use the tool at all. 

6. Today, there isn't a good defense against adversarial prompting other than red-teaming and fixing individual issues. The only way to mitigate the risk is to not train it on sensitive company information. But there's still high residual risk and companies need to be prepared to accept the additional cybersecurity risk a tool like this presents. 

There we have it folks. A Byte-sized Case Study on NewHire.AI and the use-case of using LLMs to do the phone screens. Let me know what you think in the comments below! Do you have a tool/use case you'd like me to review? Send them over! 

I saw a post the other day on LinkedIn asking, "Is it worse to have no legal advice, or to have some legal advice from GenAI that could be wrong because it hallucinated?" My immediate reaction was, "Of course, it's worse to have bad legal information than none."

But ... is it? I thought about more, beyond just my initial reaction. I realized that the question is more nuanced than I gave it credit for. There are layers to this question (just like ogres). As I explored the layers, I realized my initial thought might have been too hasty. 

Is it ethical to maybe get the wrong information if your other option is no information at all? I thought it would be interesting to break this down issue this week on Byte-Sized Ethics. 

Note: Throughout the article, I'm going to refer to the bad information provided by an LLM as "misinformation" instead of hallucination, because there's no functional difference in this situation.

Context is King 

I want to start by broadening the question. Is it better to have no information, regardless of the context, than it is to have misinformation that you don't know is misinformation? Broadening the question serves one purpose, it helps us understand that the answer to this question depends on the subject on the subject matter. 

For example, if I ask ChatGPT for a bread recipe and it gives me a bad recipe that calls for 4 pounds of salt, I've lost some ingredients, and sometimes, maybe I lose some face at the family dinner. In the grand scheme, the impact of the bad recipe was pretty small. 

But, the situation changes when we start to think about things that can have a greater impact. For example, if I ask ChatGPT for legal advice, that can have a material negative impact on my life. I'm not going to link again the absurd number of lawyers who have been disciplined by the courts for citing fake cases ChatGPT. 

The same situation applies when looking at other subject areas that have a low tolerance for mistakes. Medical misinformation comes to mind, and yet we can see big companies are already yolo'ing medical research into LLMs (complete with a great example of LinkedIn influencers more in love with their success than the harm their content causes). 

Even medium-risk scenarios are concerning. I was recently laid off, and when I was gearing up for my job search, I tried to use Claude to help me write my resume and match it better to the jobs I was applying for. But Claude wanted to lie to me. He consistently created resumes and content that were patently not true. If I had just used those and not proofread, I would have been in a heap of trouble if I got any interviews from my made-up resumes. (For the record, I corrected all the lies that Claude came up with to make me look better). 

The subject matter, and the context here matter in the answer to this question. 

Who You Are Matters

This is where things get uncomfortable. I'm a white dude - I live a comfortable life, both in the opportunities I'm presen

ted with and the lifestyle those opportunities enable. It is easy to forget to check my privilege. 

For someone who can't afford a lawyer to help with the legal problems, this question isn't actually how I framed it above. Instead, I need to ask, "Is it better to go into court knowing that I know absolutely nothing, or is it better to go in knowing something that may or may not be wrong?" 

From a medical context, my question is instead, "Is it better to have no idea about what my symptoms may indicate because I can't afford to go to the doctor and I have no idea what to do, or to get what might be bad advice but at least have something to do?" For others, using an unedited version of their ChatGPT edited resume might be the difference between being able to move out of a shelter, or not. 

I initially approached the question as someone in my position making decisions for other folks in different socio-economic situations. That was flawed because I had the means to get legal advice. My situation allows the privilege to say that no information is better than misinformation because I don't have as much at stake as someone in a more challenging socio-economic situation. 

Their decision isn't between "Legal advice from a professional, or legal advice from ChatGPT," their decision is actually between "No legal advice at all, and legal advice that might be bad." Their choice is the equivalent of being guaranteed to lose in court or just very likely to lose in court. Or it's the choice of being desperate to go to work so you can make rent and try something, or not and getting evicted. 

The situation of actual people impacted is wildly important. 

Does the Model even Matter? 

We have to remember - it's only happenstance when an LLM provides accurate information. LLMs look for the statistically most likely next word in a sequence of words which can frequently yield accurate information -- but the LLM isn't trying to answer correctly. That's before the randomness is injected into the system just so you get variation in results, making it seem more "real." 

But does that matter? Does the fact that ChatGPT doesn't try to be accurate matter when the other option is a guaranteed bad outcome? Does the patent misrepresentation of the capabilities of AI matter when your choice is limited to guaranteed bad outcome, and just a most likely bad outcome for you? 

I would argue that it doesn't. 

The Verdict 

So is it ethical for LLM models like ChatGPT to give legal and medical advice, even knowing the propensity they have to generate misinformation? I'm going to change my initial assessment and say, yes it is ethical. For me, the choice is different - because I have a choice between "probably good outcomes" and "likely bad outcomes" when choosing to use ChatGPT for this particular use. But for others, their choices are "guaranteed bad" and "very likely bad," which is a very different situation. 

ChatGPT can't know which of the social-economic situations a user is in. I think it's ethical to allow ChatGPT to provide medical and legal advice, even knowing that it might be wrong -- because for a lot of folks, "might be wrong," is still better than "definitely wrong." 

This is a rough issue, what do you think? Do you think using ChatGPT as a citizen in high-risk situations is ethical? Let me know in the comments. 

You know what’s really exciting? Setting a goal or milestone and then achieving it. I recently crossed 500 subscribers (540 at the time of this writing) and wrote my 40th article. To celebrate, I purchased a custom domain - bytesizedethics.io.

I want thank all 540 of you who joined me for this journey so far. I really appreciate your interest in my writing, and I can’t express how great it makes me feel that people want to read what I have to say. Thank you so much for your support.

If you’ve thought about supporting me in the past, but thought the $50 for a year was just too much, I set up a discount to celebrate. You can support me for a year at $35 for a limited time. I would give discounts on the monthly payments, but Substack doesn’t let me know.

As I reflected on the last 10 months of writing, I realized that I had never really explained why the name. While I thought it was a fun and clever name, a conversation with my good friend about the custom domain made me realize that it might be as clear to everyone. So, today's milestone post is all about how I came up with the name.

Why Byte-Sized Ethics?

I had a couple things I wanted to achieve when I started out - I wanted to learn more about responsible tech and responsible AI, and I wanted to help equip other people with this information to be more active participants in the technology that influences so much of our lives.

A byte is one of the building blocks of all software. It's a unit that's so small by today's standards that it's almost meaningless. Almost. We talk about gigabytes (1 billion bytes), terabytes (1 trillion bytes), petabytes (1 quadrillion bytes), or even exabytes (1 quintillion bytes). It's still not the smallest unit - that honor goes to a bit - which is a unit that can have one of two values - 1 or 0. Those bits are then strung together in sequences of 8 to make a byte. Fun fact, that's why every kilobyte, megabyte, and up is divisible by 8.

So from a technology perspective, a byte is the first real unit of meaning. Any expression of ethics, morals, and values starts here. Plus, it's a cool-looking word. I'm not immune to a good-looking word. Plus, it was a homophone for "bite-sized," which I'm sure is no surprise, means easily consumable. So Byte-sized meant two things - both the technology-driven topic and the goal of being easily consumable for most people.

The ethics piece of that name.... well, that's pretty self-evident.

The Domain

When I went to register a new domain, I was torn between the default '.com' and what I thought was a clever usage of the '.io' domain. I thought .io was clever because it's like a bit - using the short-hand "i" for true and "o" for false. I realized though, when I had to explain to a good friend, who is also very nerdy, that might be a little niche even for my audience.

So I did the only logical thing - I bought both the .io and the .com. But the .com redirects to the .io. In the end, I couldn't deny my nerdy side and the stupid jokes that make me happy.

So that's it. That's how I came up with Byte-sized Ethics and the domain I chose. It's really a testament to my own humor and jokes that really I get. But it makes me happy, and makes me excited to write, so it all works out in the end :-)1

Here’s to the next 40 posts, and the next 500 subscribers.

So you've been reading up on GenAI and the 'AI All the Things' trends, and you're getting concerned. It seems like there's a lot of risk, AI is everywhere and it's impossible to get your hands around what's happening, and protect your company, your colleagues, and your customers from AI-induced risk. You aren't alone. Most companies are taking a much more conservative approach than what the "thought leaders" on LinkedIn would have you believe.

So what do you do? You limit the risk by being intentional in all the ways you use AI, and your vendors use AI. 

In this article, I'll walk through high-level tasks you need to take to help reduce risk to your company, your employees, and your customers. First, make sure you have a policy written. Then, review all of your vendors and find out who's pushing AI solutions. Finally, make it part of your company culture. 

I’m happy to help turn these articles into a reality for your organization. If you have questions, problems, or just want to discuss approaches, feel free to reach out to bytesizedethics@substack.com and we’ll setup some time.

Implement an AI Usage Policy 

First, you need to codify that you want to avoid AI usage. I've already written out the aspects your policy should contain. This policy will guide the rest of your approach. If you plan to allow some limited AI usage, this policy becomes even more important for you to list out allowed tools, disallowed tools, and allowed/disallowed usages. 

The policy should give guidance to your company and how (not) to engage with AI and give your colleagues the tools to make responsible decisions, and let them know what could happen if they violate these guidelines. A well-crafted policy is your first line of defense against risky AI usage. 

If you want to limit AI risk as much as possible, your AI policy must include how to respond to vendor AI usage. After governing your employees' use of foundational models like ChatGPT, how your vendors use AI is the greatest AI risk to your company. While you might be doing your due diligence, there's no guarantee your vendors are doing the same. The AI landrush means that a lot of vendors are stapling AI offerings onto their existing products and in many cases, enabling those features for you without your knowledge or consent. It's great (/s). 

Conduct a Vendor Review

Once you've written the policy with a vendor AI usage section, you need to do a vendor review to see which vendors are using AI. This could be a time-consuming process if you have a lot of vendors. The best way to do this is to come up with a questionnaire and require your vendors to complete it. 

Overall, you'll want to know if they are using AI, how they are using it, what data powers it, and how they manage the data they send to the foundational models. Some of the things you'll want to ask: 

1. Does your solution include any AI features, specifically generative AI features like those powered by ChatGPT?

2. How is the AI used? What goals or features does it enable? What does it do in your product? 

3. What foundation model(s) does your feature use (i.e. ChatGPT 4.0, Claude 3.0, etc, In-house)?

4. Can I enable or disable these AI features? 

5. What data is sent to the model? 

6. Can I control what data is sent to the model, by field or record? 

7. Is my data used to train the model? 

   1. What assurances, outside of contractual obligations, do you have to ensure that the model isn't being trained on my data? 

8. What is the retention period of any data I send to you? 

These questions should help quantify the risk you assume by using this vendor, and their AI features. The biggest area of concern you'll have is what data the vendor is sending to the foundation models, if that data is used to train the models, and the data retention. 

There's a good chance your vendor won't be able to answer all the questions. But their inability to answer tells you something about the risk you'd be adopting by using their AI tools. My experience in using a similar questionnaire was that vendors couldn't answer all the questions. I asked what kind of assurances they had, outside of contractual obligation, that the data they sent to the foundational model wasn't being used to train that model. No one has had anything beyond the contract thus far, which is problematic because it's almost impossible to tell if your data has been used to train a model until a breach occurs. 

Once you've gathered all of this information, you can make decisions about whether the use is acceptable for you or not. You should track all of this information somewhere, through a tool like Trustible.ai or Enz.ai. At a minimum, drop all of this information into a spreadsheet for tracking. 

Incorporate it into Your Company Culture

Now you've got a policy, you've done an audit of your vendors and their use of AI, now it's time to make it part of your company culture. This can be a daunting task, especially considering how seductive GenAI can be as a perceived efficiency booster. But there are a few methods to help:

Make your employees review and acknowledge your AI Usage policy.

They will grumble about it, but you went to all that effort to write the policy, you want to make sure that everyone reads it and acknowledges it because you don't want anyone to be able to say, "I didn't know."   

You should make your policy acknowledgment a yearly thing, and keep it posted in an easy-to-find, easily referencable place. You also need to talk about it frequently. It shouldn't be a "I check a box yearly" and never think of it again. AI is changing rapidly, and you want your employees and colleagues to think about how they use AI all the time, not just when they click "I acknowledge". 

Update your MSAs - with customers and with Vendors

AI and GenAI are still getting a lot of attention... even 18 months after OpenAI released ChatGPT 3.0 to the public. With this attention, fueled by hyperbolic and bombastic claims from "experts", your customers will be on high alert for genAI usage, and you should be too. 

Work with your legal department to codify in your MSAs with customers and with your vendors the rules and guidelines around AI and GenAI -- you don't want any surprises and you want to make sure you are clear with your customers about how you'll use GenAI so that they don't have any surprises. 

Incorporate your AI Questionnaire into your vendor-sourcing 

Take that questionnaire you created before and make it part of every vendor evaluation so that you know what you are getting into before you sign the paperwork. Because AI is evolving so fast, you also need to have a 6-month or yearly review with the vendor to ensure that their answers are accurate. 

Bonus Round: 

You might also want to make a public-facing AI usage Notice and guidelines so that it's easy for your customers to see and find. But, you should only do this if you are confident you can abide by what you are saying. If you post a public notice and then don't follow what you said, the FTC could show up at your door. 

There you go, three straightforward ways that you can limit your AI exposure and risk through policy and process. What do you think? Do you have any other ways to help limit your AI risk? 

Do you need some help implementing these recommendations? I'm happy to help - just shoot me an email at bytesizedethics@substack.com

Hi friends. Let me start today with an apology for missing last week. I've mentioned a few times that right now is a tumultuous time for me - vacation, laid off, dealing with a particularly ornery instance of my seasonal affective disorder -- I needed to make some head space for myself. I'm working to get my groove of publishing back, so I'm asking for some grace while I find that cadence again. 

Relatedly, I'm contemplating dropping down to every other week while I get my bearings. What do you think? Is every other work enough? Let me know in the comments. 

Finally, I'd like to thank my first yearly subscriber - my mom. You might say, "Well, of course, your mom subscribed, she's your mom." You aren't wrong, but it still means that she cared enough to consistently read and to throw some money at me. So, thanks Mom - I appreciate it. If you'd like to join her, you know what to do. And I'd appreciate it just as much. Well almost. It's hard to compete with Mom. 

Subscribe now

With that, on to the main event.

Recently, I was having dinner with a friend who told me that he was unsure about AI and what it meant for the future. We spent the next 20 minutes discussing his concerns about AI--what AI was, what AI wasn't, where we should be concerned, and where the hype was overblown. It was a lot of fun and a super productive conversation. Our conversation ended with the mutual understanding that a lot of people just don't 'get' what AI is, how it does things, and its limitations. 

That's a pretty big problem. 

Most folks lack AI Literacy. If you asked the average person how ChatGPT works, you'd get a shrug, and "I don't know. It just does." The average person knows remarkably little about this technology that touches almost every aspect of their lives and has an impact on them every single day. They make assumptions about what it's doing, or worse they just take outputs from AI systems as objectively true. When people don't have literacy in technology this pervasive, they can't make informed, meaningful decisions about the technology. 

What is AI Literacy? 

Ironically, the best definition I found of AI Literacy comes from AI itself. According to Claude3, AI Literacy, "refers to having an understanding of what artificial intelligence (AI) is, how it works, and how it impacts society. It involves being knowledgeable about the capabilities, limitations, and implications of AI systems." Being AI literate is foundational to using AI responsibly. 

Being AI literate is being able to use Claude3, and understand, "All the model did was convert my question into numbers, run some statistics on those numbers to predict the most likely next numbers in the sequence, and then translate those numbers back into words to print on the screen." Now, there's a whole shitton more actually going on here when I posed my question to Claude3, but conceptually, that's what happening. It's not difficult to understand, but it helps me frame the result I get back. 

The ultimate goal of AI literacy is to materially educate folks so that can intentionally and deliberately engage with AI, being aware of the risks and limitations of doing so. If you are AI literate, you can use AI responsibly, and get the benefits of the technology while avoiding the worst of the risks and challenges. 

So my understanding of how Claude works helps me frame my interactions with Claude. Claude didn't understand my question, it just did some algebra. It doesn't mean that Claude is wrong, or that it generated incorrect responses. It just means that I know that Claude can be wrong because it's not programmed to be correct. 

The Broader Vision 

This isn't the first time this kind of thing has come up. In the early 2000s when the internet was just starting to become a part of everyday life, the concept of Digital Literacy came up and as a society, we dropped the ball. We never really got the broad digital literacy we needed to allow people to better engage with the technology that was inundating their lies. 

There are lots of reasons that have led to the increase in misinformation and disinformation. But our social inability to understand the technology that had come to rule our lives opened the door to our current challenges with misinformation and disinformation. 

We could repeat the past if we don't focus on our AI literacy today. I won't be hyperbolic and say it'll be worse than the outcome of the lack of digital literacy. But a lack of AI literacy can make things much harder than they need to be for us going forward. 

If it's so simple, why aren't more people AI literate?

There are a couple of reasons from my perspective - the first is that no one has made a concerted effort to develop an accessible AI Literacy curriculum that anyone with a high school education can understand. The resources just don't exist yet. Teaching yourself requires a lot of motivation, and the ability to translate semi-Technobabble into something that you can understand, which is a pretty high bar to clear. We need folks who take these very head concepts and make them accessible and easy to understand. 

The second is a more tin-foil hat reason, but companies like OpenAI, Microsoft, and the other genAI players have a vested interest in folks not understanding AI. If people don't understand AI, they are easier to (lie) sell to because the companies can make wild claims like they've developed Artificial General Intelligence, and no one knows any better than to call them on it. They can exploit the misunderstanding to boost usage, increase dependence on their tools, and find ways to make more money through misunderstanding. The fewer people question the results of AI, the more ingrained it becomes in our lives, and the easier it is to manipulate how we engage with AI.

The literacy problem isn't limited to just the average person. Many folks with very impressive titles have the same lack of AI literacy. 

For example, I have a friend. They're brilliant, one of the most driven, motivated, and intelligent people I have ever met. But when they post on LinkedIn about AI, it's clear they don't understand the technology. They make claims and predictions about AI that if you know enough about AI, you know aren't plausible or realistic. They use words like "understanding" and AI's ability to "reason," as predictors of future performance. But, knowing how GenAI works, we know it just gives the illusion of understanding and reasoning but it's not doing any of that. 

I know of a VP at another company who told his entire department that if they weren't using ChatGPT they were failing. He told them to use ChatGPT for all of their daily tasks, from writing emails to customers to reviewing draft contracts to keeping track of their quarterly goals. He wanted them to use ChatGPT for everything because, in his mind, it would boost productivity. 

This is problematic for a lot of reasons, but the most important at the moment is that ChatGPT trains the model based on user inputs. Because this VP didn't didn't understand how ChatGPT was working even at a business level, he put his entire company at risk of unintended disclosure.

Finally, there's a frankly embarrassing number of lawyers who've gotten in trouble because they use ChatGPT to write court documents, ChatGPT hallucinated a bunch of cases, and the lawyers got in trouble. It's driven by these lawyers fundamentally not understanding what ChatGPT is doing and how it works. If you were looking for the legal version of Fuck Around and Find Out, this is probably it. 

So what do we do? 

Right now we fight the good fight. Chances are good that if you are reading Byte-Sized Ethics, you have some AI literacy. Your charge is to spread that AI literacy as much as possible. 

• Whenever you see someone using words like "understanding" or "reasoning", correct it. Be *that* person on the comment thread or at the meeting. It might seem like a useless gesture, but it helps change the dialog. 

• Create your own AI elevator story - come up with a quick story to talk about AI that avoids the worst misconceptions. No more than 30 seconds. This will help you prepare for how to talk about AI in a lot of different kinds of scenarios. 

• Read - the best way to learn more about how AI works is just to read. Check out the Book Bytes series here on Byte-Sized Ethics for a good place to start. 

  • I also recommend "The Alignment Problem" by Brian Christian for a great primer on the history of AI and concepts. I haven't written about this one yet, but it's coming. 

• Help inform people about the challenges or risks that come with using AI in some cases. 

• Take AI courses - there are a few good introductions to AI courses on Edx, Coursera, and LinkedIn Learning 

What do you think? Do you think AI literacy is important? And what are some other ways to increase AI literacy among non-technical folks? 

Subscribe now

I won't lie, it took me a while to get through Algorithms of Oppression by Dr. Safiya Omoja Noble. Don't get me wrong, it's a wonderful book and I cannot recommend it highly enough. But it was hard to read in the best possible way. Dr. Noble brings authenticity and emotionality to the issues of representation in search engine results that I wasn't expecting. But the discomfort the book caused me is what made me love it.

What appealed to me personally about her approach was my own background in cultural studies. Tying issues of representation, typically thought of in purely creative fields, to technical representation in such a compelling, evocative way really spoke to the neutral theorist in me.

It's so easy to to treat the search results as empirical, as if they represent reality. What Dr. Noble reminds us is that search engines are made by people, and people are not empirical. Our biases come out in our technology, especially when we aren't paying attention. It is, unironically, the basis for the entire discipline of Responsible Technology.

I can't begin to do justice to Dr. Noble's work for a lot of reasons. But if you want to understand not just the technical implications but also the actual lived experience consequences caused by reinforced stereotypes in technology, there's no better book to read. Bounce over to Amazon, and pick up your own copy. You won't be disappointed. Maybe a little uncomfortable, but not disappointed.

Do you have a suggestion for a book you think I should read? Let me know in the comments! I’m always on the lookout for more great books. Also, please consider subscribing, even for a month. Byte-Sized Ethics is an independent publication, and you support helps me keep it going — and really makes me feel great.

Subscribe now

There's been a lot happening in Vida de Andy of late, so I haven't had much time to write as I would have liked. I found out earlier this week that I was being laid off. I had been expecting that it was coming for about the last 2 weeks or so, but even with that knowledge, it still hit me like a sucker punch I wasn't expecting. 

But, not to worry. I have a few options in the pipeline, and I'm sure I'll be back up and gainfully employed in no time. 

All of that is to say that I thought I would take a light-hearted approach this week. I wanted to find some of the absurd uses of AI out there. And fam, the interwebs did not disappoint. 

A guess of a guess is somehow less of a guess? 

With a name like Parabon Nanolabs, you know that whatever their product is going to high on hype, low on caring about consequences. Parabon Nanolabs is a company that uses your DNA to make a guess at what your face looks like. This isn't a new use of AI and machine learning, but that doesn't make it any less facepalm-worthy. The Wired article talks about a case from 2017 where police used a face generated using Parabon Nanolab's software, and then later ran that ML-generated face through face recognition software. And thought was all totally fine. 

Why is this absurd? 

We can't even conclusively predict eye color using DNA. Yet, this company has the audacity to say it can recreate an entire face? Bruh, really? Even worse, the police then took the ML-generated face and ran it through ML-facial recognition models because what your ML really needs is more ML. It's like ML-inception. Can imagine someone being arrested because some random company created an image that looked like you using the proprietary ML model that they super duper pinky swear is tots accurate, and then police ran this fake image against the already finicky facial recognition models? 

I need AI to help me walk

Who thought my roller skates needed AI? Apparently, Moonwalkers did. They slapped some AI into a pair of roller skates completed with a $1,400 price tag to boot to solve the problem with people ... walking I guess? I fail to see the problem this is solving other than, "How can we make the most absurdly expensive roller skate on the market while also capitalizing on the AI marketing hype bubble? 

Why is this absurd? 

I'm reasonably sure that those for whom walking is an option are not going to shell out $1,400 for a pair of roller skates with AI in them for our "commute." 

Really? AI-ified roller skates?

This company wants to make sure you get that annoying small talk with your coffee... whether you want it or not 

This company, NeuroSpot shows every sign of being absolutely incorrigible about their big brother-esque surveillance technology. I stumbled on this originally by looking at this LinkedIn post that shows the technology being applied in a coffee shop to ensure that employees were talking to customers often enough. If you check out the comments you'll see that NeuroSpot proudly confirms that the video is their software. 

Their website is ... terrifying with the blasé way they present their surveillance technology. They suggest constant surveiling of schools, hospitals, restaurants and private property. Because really, what could go wrong?

Why is this absurd? 

There's so much here that's wrong. It's blatantly privacy-violating across so many jurisdictions. Their models are proprietary, meaning that we don't know how accurate any of this actually is. Not to mention that the things they claim to detect are incredibly hard to identify even for nation-states with virtually unlimited resources 

So these are just 3 examples of absurd uses of AI. I'm sure there are more out there - so let me know. What absurd uses of AI have you seen?