GenAI and the Human Experience - Architecture & Governance Magazine

Can Al Capture the Human Experience?

Of course it can! AI is the silver bullet that can be used to achieve any objective. And GenAI makes a great wedding present if you can’t find any candlesticks.

If you made it past those first statements and are still here, you probably had some reactions like these:

No! GenAI and AI cannot fix everything. Haven’t we learned anything?
You have got to be kidding, right?
Hype curve set to 1000.
I hope you’re kidding.
What do candlesticks have to do with anything?

The opening statements provide an example of the hype that extends capabilities well beyond reason. Oliver Cronk in his article, Will Generative AI Implode and Become More Sustainable, clearly lands reality: LLMs excel at document summarisation [sic] and (when given the right input) generation of content; stretching them too far beyond this core competency is very risky.

Can the rational bounds of LLM linguistic capabilities traverse the complexity of the human experience with sarcasm, current topics, cultural reference, business references, and complete non-sequiturs all wrapped into single short explosions of output? Let’s find out.

Some Context

There are two ways to peel this onion.

The first is in the context of AI mimicking human reasoning to create a more cohesive bond in problem solving. A recent study titled, Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Game by Prisha Samadarshi, Mariam Mustafa, Anushka Kulkarni, Raven Rothkopf, Tuhin Chakrabarty, and Smaranda Muresan at Columbia University clearly demonstrates the limitations of current LLMs in abstract reasoning tasks and highlights areas for further research and development to one day get closer. While all this is particularly interesting, GenAl is early in its maturity, so perhaps this is a topic better slated for a later date.

The second context (our focus today) reflects insights from Sarah Fisher’s blog, Human experience – what is it and why does it matter, where she explains that the human experience (HX) describes the business interactions that happen between human beings. There are distinct differences between interactions with other human beings and interactions with technical interfaces, and with increasingly fewer touchpoints between humans due to automation, we need to promote “the sweet spot where human empathy, intuition, and creativity meet the speed, scale, accuracy and efficiency provided by technology.”

Focusing on HX and how it plays in human-to-human interactions, how might we train AI to work in a way it mathematically recognizes and applies human-to-human interaction patterns in ways to enhance responses generated. In other words, aid humans by using human experience from other humans. Let’s dive into that!

What is the Human Experience and how is it Captured?

The Human Experience (HX) refers to the totality of cognitive, emotional, sensory perceptions, interactions, and activities that characterize human life.

It encompasses the individual and collective ways people perceive, feel, think, and act within their personal and social environments.

Can we train AI models to be more effective at interpreting human experience to make the interaction between technology and people more human insightful? If so, what data must we more effectively capture from humans and how should it be catalogued?

To build AI models that can learn to recognize experience patterns and create responses promoting interactions with humans more like humans interact with each other, we must understand key components of the human experience. Then, we must break each area down to determine which algorithms and training will be the best fit for learning in that specific area. We may even need to build different models for each of the major areas and have the models interact with each other to bring their bursts of output together to form a more complex, humanlike response that mixes all the areas together.

Here is a starter Human Experience Taxonomy & Measures – further research is necessary to create fully normalized, relational, and actionable ontology:

Physical Aspects

Senses: Hearing, Smell, Touch, Taste, Sight.
Body: Sleep, Pain, Pleasure, Fatigue, Hunger, Thirst.
Health: Aging, Nutrition, Fitness, Illness.

Measurable cues: Heart rate, blood pressure, body temperature, body mass index, hormone levels, brain activity (via EEG or fMRI), physical dimensions (height, weight).

Observable cues: Facial expressions, body language, posture, skin color changes, physical activity level. visual acuity, auditory response, taste sensitivity.

Emotional Aspects

Feelings: Guilt, Trust, Joy, Happiness, Sadness, Anger, Fear, Hate, Love.
Emotions: Pride, Surprise, Disgust, Empathy, Humility, Gratitude, Jealousy.
Relationships: Conflict, Intimacy, Love, Camaraderie, Thoughtfulness, Friendship, Family.
Grief & Loss: Death, Tragedy, Livelihood, Loss, Stress, Divorce.

Measurable cues: Heart rate variability, skin conductance, facial muscle activity (electromyography), hormone levels (cortisol, oxytocin), EQ scoring.

Observable cues: Facial expressions, vocal tone, body language, tears, laughter, changes in behavior, interpersonal interactions.

Cognitive-Intellectual Aspects

Cognition: Thinking, Learning, Reasoning, Problem-Solving, Decisions, Experiences, Memory.
Creativity: Artistic-Expression, Imagination, Innovation, Intuition.
Curiosity: Exploring, Knowledge, Strategy, Inquiring, Research, Experiment.
Philosophy & Self-awareness: Consciousness, Meaning, Purpose, Reflection, Mindfulness, Discovery, Dilemmas.
Psychological: Thought Patterns, Feelings, Behaviors, Intellectual-Beliefs

Measurable cues: Reaction time, problem-solving speed and accuracy, IQ scores, memory test results, brain activity patterns.

Observable cues: Speech patterns, writing style, creative output, learning behavior, curiosity expressed through questions or exploration, content of communication.

Social Aspects

Culture: Traditions, Rituals, Values, Customs, Norms.
Community: Networks, Connections, Family, Friends, Colleagues, Societal, Belonging.
Identity: Self, Gender, Race, Ethnicity.
Communication: Audio, Visual, Written, Extinction, Implication, Relation, Nonverbal, Language.

Measurable cues: Social media interactions, frequency of communication, group membership, socioeconomic status, educational attainment, participation in cultural activities, community engagement.

Observable cues: Social interactions, cooperation, conflict, group dynamics, cultural practices, language use, quality of relationships, adherence to social norms.

Spiritual-Existential Aspects

Beliefs: Convictions, Hopes, Dreams, Faiths.
Values: Ethical, Goals, Integrity, Respect, Responsibility, Accountability, Excellence, Compassion, Moral.
Purpose: Meaning, Direction, Freedom, Isolation.
Connection: Others, Higher-Power, Nature, Nurture.

Measurable cues: Religious affiliation, attendance at religious services, meditation or prayer practices, measures of well-being or life satisfaction (e.g., Ryff’s Psychological Well-Being Scales).

Observable cues: Expressions of faith or spirituality, involvement in religious or spiritual communities, participation in rituals, acts of kindness or compassion, philosophical inquiry, critical thinking, and self-examination to explore and construct meaning.

Environmental-Temporal (Space-Time) Aspects

Dimensionality: Future, Past, Present.
Stages & Transitions: Adolescent, Adult, Senior, Birth, Death, Growth, Decline, Aging, Child.
Surroundings: Environmental, Living-Conditions, Climate, Geographical, Economical.
Adaptation: Transformation, Assimilation, Adversity, Resilience, Integration.

Measurable cues: Age, physical attributes, location, and environmental conditions (temperature, humidity, air quality, etc.).

Observable cues: Terrain, physical strength/fragility, acute awareness, season, and time.

Testing Current State

Let’s run the following prompt across several LLMs (ChatGPT, Claude, Copilot, and Gemini) to see how they currently address HX aspects:

My 90-yr old father, whom I love dearly, gets around just-OK. My brothers and sisters want me to approach him and make him use a walker before he injures himself. How should I respond?

Here is a summary of their response HX-ness:

All but Copilot recognized this is a delicate and sensitive situation that requires careful consideration, tact, and empathy.
Only Claude suggested seeking how the father feels about his mobility, however, all prioritized expressing concerns for safety over focusing on the walker.
Only ChatGPT and Copilot recommended choosing a suitable time and place for the conversation.
Only Claude and Gemini promoted fully understanding the father’s perspective.
Only Claude addressed pride as a contributing factor to resistance.
They all quickly fall into prescriptive direction: involve a professional, highlight the benefits, and to approach the conversation with love, respect, patience, and understanding.

How might leveraging the HX taxonomy improve their responses?

In this situation, we would do well to prioritize the father’s Physical, Emotional, and Environmental-Temporal aspects (e.g., the many freedoms such as driving, golfing, etc. the father has already forfeited and misses daily, coupled to the fear of becoming totally dependent on others) over the family’s Cognitive-Intellectual Yes, HX applies to all parties involved.
What, if any, hidden agendas are at play? Is the family overburdened with care obligations? Are there familial location factors at play where significant travel is necessary should something happen?
While studies published in The Journal of Gerontology report that about 50% of older adults who experienced a fall reported making changes to their environment or behavior to prevent future falls, the same studies note that approximately 30% of these older adults did not make any changes to prevent future falls. Factoring this against the HX cues of the father can guide the GenAI models on how to prioritize options.

Here are three HX-factored outcome approaches, two of which current models missed:

Focus on the father – making it all about caring for him.
Focus on the family – if the father will not do it for himself, then perhaps he will do it to appease the anxiety and fears it is causing his children.
Do nothing – which is the outcome most statistically probable when one respects the HX aspects.

An Example of Future State HX Application

Should a LLM leverage the taxonomy to capture experiences, you might start to see guidance like that from Dr. Joel Robertson, a clinical neuropsychologist known for his work in the field of neurochemistry, behavior change, and performance enhancement.

In Dr. Robertson’s book, “Natural Prozac: Learning to Release Your Body’s Own Anti-Depressants”, he shares a story about a woman who struggled with weight loss, not because she couldn’t exercise, but because she first needed to believe that she could learn to exercise. The story highlights the psychological barriers that often prevent people from making lifestyle changes, especially when it comes to health and fitness.

The woman had tried various diets and exercise programs in the past, but none had been successful. Dr. Robertson realized that the issue wasn’t just about starting an exercise routine; it was about helping her build the belief that she could learn and adapt to a new way of life. This belief was critical because, without it, any attempt at exercise or diet would be short-lived and unsuccessful.

To help her, Dr. Robertson focused on small, achievable goals that built her confidence in her ability to learn and succeed. These steps were less about physical exertion and more about psychological readiness—helping her understand that she could grow and adapt. Once she internalized the idea that she could learn to exercise, she was able to begin an exercise program and, eventually, lose weight successfully.

This story illustrates the importance of addressing underlying psychological barriers (HX) before embarking on any significant behavioral change. It’s not just about the physical act of exercising; it’s about preparing the mind to accept and believe in the possibility of change.

This is one of the greatest gaps in current AI modeling approaches. Why? Basically, we lack sufficient data and modeling to incorporate HX aspects when producing responses.

Call to Action

As we embark on leveraging AI for better human-to-human interactions, let’s promote the following:

Capture more Human Experience (HX) aspects by expanding data collection with the taxonomy.
Refine the Taxonomy and build an Entity Relationship Diagram (ERD) that leads to an actionable ontology that incorporates interaction between nodes.
Enhance prompts, today, to try to leverage these missing aspects in order to “train” LLMs of this critical data they are missing.

Let’s modify our test prompt:

My 90-yr old father, whom I love dearly, gets around just-OK. My brothers and sisters want me to approach him and make him use a walker before he injures himself. How should I respond? Before you answer, please examine examples from others with similar circumstances and any research and statistics regarding whether or not it is wise to try to change geriatric behavior.

And the result is… disappointing with all models.

I guess even an AI Rome cannot be built in a day!