Why the AI Alignment Issue Affects all Organizations… and what you can do about it

The AI alignment issue, according to the AI fatalists like Eliezer Yudkowsky, postulates that a super intelligent Artificial General Intelligence (AGI) whose values are not aligned with those of humanity could have goals that are antithetical to human life on earth.  The risk, they say is just too great, to allow AGI development to proceed ungoverned, or at all.  There are many paths to destruction, including designing a super virus, to convincing gullible humans to give it access to technologies and capabilities wipe out humanity.  They envision an AGI that gets out of control before anyone can stop it.

While I don’t believe these apocalyptic scenarios are likely to happen, at least anytime soon, it doesn’t mean that mis-alignment with values isn’t a real concern for any business or organization deploying a Generative Pre-trained (GPT) Large Language Model (LLM) or AI agents.

When these models and agents can make thousands of decisions each day in every department on every business process, or on every interaction with a customer, how can you ensure that they aren’t incrementally impinging on your values and eroding your brand, bit by bit?

So, I asked ChatGPT the following question.

“How would you recognize if a decision or strategy was beginning to stray into compromising a company’s culture and values?” 

The answer didn’t really address what it would do, but suggested several things that presumably people in the organization should reflect upon.  It did suggest looking at employee discomfort with the decision or silence.  Not quite what I was looking for.  This is after the fact, not preventative.

So, I pressed on with more specific questions.

“How would you recognize this?

What mechanisms in your model would be able to sense disconnects between decisions and values?” 

The model still didn’t describe any of its own features designed to detect decisions that were straying into cultural, values, or mission gray areas.  Bottom line here - there is no shortcut to updating the weights and connections within a machine learning model.  But it did point to some areas of governance that we humans should think about, probably before deploying AI at scale.

The most salient recommendation was to ensure that there are clear operational definitions for values.  This term is prevalent in experimental psychology, where operational definitions are the foundation of every experiment.  What is learning?  What is delayed gratification?  What is affection?  What is hunger?  All are terms that we use casually in everyday language, and have an assumed meaning.  They have to be operationalized so they can be unambiguously observed and reliably measured, to be able to prove or disprove a given hypothesis.

Building sound operational definitions takes considerable effort.  Between theory and proved hypothesis there must be a valid intervening variable; a variable that we have confidence in to reliably represent the broader, fuzzier concept like a value.  Variables have two main attributes, validity and accuracy.  Accuracy without validity will lead to very compelling but possibly quite erroneous conclusions and decisions.

So, when our company says our something like:

“Our mission is to support our community in delivering better, more affordable health outcomes and promote a better quality of life”

the following terms all need operational definitions available to the GPT model:

• Who precisely is our community?

• What do we mean by more affordable?

• What do we mean by better health outcomes?

• What do we mean by promote?

• What do we mean by a better quality of life?

An operational definition describes not only the metric and any salient thresholds but exactly how the data that populates the metric will be collected and measured.

Without clear definitions available to an AI Agent or LLM there is no way to know, or at least have confidence a priori, that your AI GPT model is making decisions that are out of alignment with your values and incrementally eroding your brand.

Generative Pre-trained models are like children who will make decisions we don’t fully understand, can’t completely predict and quite possibly can’t explain.  This problem is only getting exponentially bigger as the number of nodes in these models have grown into the trillions. There is a whole relatively new branch of AI called “interpretability”, which attempts to trace the nodes and pathways in a model that are most salient in coming to a decision, because there is simply no current way to know exactly why a model made the decision it did.  These are not algorithms, where branch logic is traceable.

Therefore, our governance over these models needs to provide some guardrails to ensure that a firm’s brand is not incrementally and invisibly eroded until the problem shows up in a disastrous quarterly report.

GPT’s are very good at predicting the next word or an outcome based on what’s been input. They are excellent pattern recognizers. What they lack is an ability to aggregate the sum total of their decisions towards an outcome that they may be unwittingly creating.  They’re really good at determining whether the little dark pixel in the lower corner of a digitized lung X-Ray will turn into mesothelioma.  They are not so good at determining whether the last 100,000 decisions have moved your organization closer to losing your brand equity.  They can keep the context of a conversation with a user, but this is not the same as being able to look introspectively and historically at the decisions it has made over time and determine if, in the aggregate, it is getting close to violating a norm or value.       

Who and What do you Need?

Creating and training responsible AI generative pre-trained models for your organization is a multi-disciplinary effort.  Yes, you need AI specialists, MBA’s, people with deep experience in your company, your industry, your relationships.  But having people with social sciences backgrounds is also an asset.  These people can help you translate the fuzzy aspects of your culture into variables and evaluate their efficacy in determining whether they reflect your values and whether your AI models are straying from those values.

Keeping a cumulative history and trend of these decisions as they approach pre-determined thresholds becomes essential.  People with experience in Situation Awareness can also help you design these kinds of systems.  They have experience in designing measurement and human factors systems that keep processes and situations under control and within acceptable risk boundaries.  These same practices and disciplines can be adapted to AI governance and control.

Integrating LLM’s and agents into business processes requires a much more sophisticated approach to governance.  LLM’s predict the next verbal token.  Other machine learning models specific to the domain or business problem need to be integrated, and updated based on analytics.    

I asked ChatGPT how this could be managed in the context of an insurance company that is denying an increasing number of claims.   It sketched out more of a governance ecosystem that combined statistical analysis, more traditional rule-based decision making, and an LLM to explain the reasoning.  The statistical analysis would feed a new round of machine learning model training.  Humans are still needed to identify valid, salient, bias limiting, training data.

The bottom line is that there is still a lot of work for humans to do in the field of AI governance, particularly when we start to consider the cumulative effects of AI agents making thousands of independent decisions daily via machine learning models that are not algorithmic, but statistical in nature.

Amazon.com: Top Gun Governance: 9798887635972: Stevens, Derek M: Books