Evaluating the “Goodness” of AI

A lot of folks in my circles that were “AI skeptics” are starting to come around after seeing examples of “agentic” AI in the last month or so (Mind-blowing! Holy shit! Why aren’t more people talking about this?!).

I’ve been spending a fair bit of time playing with various “vibe-coding” tools, and I can’t say with certainty that I fully understand the zealotry quite yet. I find model-context protocol (MCP) really interesting, but the rapidity with which it’s being dropped into automations that have no confirmation step has been alarming to me. I mostly use Claude to dump out the intrusive thoughts in my head and entertain myself with imagining the LLM enmeshing with my own consciousness. I think the hallucinations are the most interesting and annoying part of LLMs.

I can play in Cursor or watch a friend use Lovable to build a game prototype that would have taken me weeks to build in 2010 and immediately understand the appeal. My experience of the tooling so far feels a bit like gambling. Occasionally you get magical results. Sometimes the codegen is chaotic, but it shortcuts you to a prototype and you can do a little cleanup work. We’re quite obsessed with doing more things faster and freeing ourselves up to do “meaningful” work (whatever the fuck that means).

I get it. This stuff is starting to become useful to a lot of people.

What I’ve been really interested in noticing, though, is how we are all—zealots and skeptics alike—evaluating AI based on the question:

Is it good?

But what does “good” mean? For what/whose definition?

• (A) good, as in: performs a task effectively? (it did a “good” job), or
• (B) good, as in: “not bad”, or “does not harm” (it is net “good” for us as beings).

I’m noticing how much the excitement about AI has changed because it’s starting to meet the definition of (A)-Good for many people.

Past couple weeks, I’ve read many such articles and posts (some even with the implied pejorative here):

I would suggest that the skeptics are not missing something. The skeptics continue to evaluate AI in both the (A) and (B) contexts.

It feels to me like we’re losing more critical skepticism, but I think it’s because many of the initial skeptics were actually (A)-Good evaluators. They switched from NO to YES because AI is now meeting their personal definition of good. They were not actually interested in whether the progress does harm, but whether the progress progresses—for them. And in the tech industry, the progress must progress (never mind the productivity industry!)

And, yes, there are hardline (B)-good folks who refuse to engage with the technology at all and mostly consider the scaled harms. Thank goddess for them too.

It’s clear this technology in the wrong hands can do immeasurable harm and can and is used as a tool for disinformation, targeted harm, and even genocide.

I think we need the hardliners. I think we have to accept that products and technologies that upend entire industries and revolutionize productivity are going to “produce” both zealots and haters.

My current evaluation in A/B contexts:

In the (A)-Good context: Not good per se—I think it’s chaotic—and chaotic speed can be beneficial at times, especially for rote tasks or iterative experiments. However, it’s also abjectly terrible a lot more than is being noted.

In the (B)-Good context: If we’re evaluating AI for (B)-Good, it’s pretty awful. We’re consolidating so much of our autonomy in a handful of corporations and maligning sense-making significantly.

In short, at the moment the value of (A)-Good explains but does not excuse what’s happening to (B)-Good.