There are moments when you read something that doesn’t just inform—it resonates. It hums along a frequency you’ve felt but hadn’t yet named. That’s how I felt when I came across Martin Ciupa’s article, Why Sentient AI with a Causal Moral Compass Is Essential for Safe Superalignment.
In it, Ciupa explores something that I believe is missing in most conversations about Artificial General Intelligence (AGI): the soul of it all. Not in the religious sense, but in the ethical, emotional, and profoundly human dimension we often sideline when speaking of code, parameters, and guardrails.
He asks us to imagine an AI system that doesn’t simply obey but understands. A system that doesn’t just simulate intelligence but can simulate what it means to be ethical—to “feel out” the moral consequences of its decisions, even when no clear rule applies.
This goes far beyond what we call “alignment.” It touches something more challenging to manufacture: conscience.
My Take
In Digital Decorum, I explored how old—fashioned etiquette, grounded in care and consideration, can become our most powerful tool in navigating this age of intelligent machines. But even good manners in the digital world mean little if the intelligence we’re speaking to lacks an internal moral compass.
And here’s the truth: rules won’t be enough…not in the real world.
Rules are rigid, but human morality is flexible, nuanced, and lived. Our sense of fairness isn’t downloaded from a server—it’s cultivated through experience, culture, and empathy. It is felt, not programmed.
So, if we want machines to participate in human society—not as tools but as intelligent agents—we must find a way to model feeling—or at the very least, to simulate it with enough fidelity that it becomes a meaningful guide.
Machines with Minds, Not Just Functions
Ciupa points out—brilliantly—that moral reasoning requires a causal model of the world. One that can ask not only what will happen if I do this? But also, who will this affect? In what way? And does it matter to them?
That’s the beginning of something like conscience.
But the key insight is this: to do that well, AGI must develop a form of simulated sentience—a kind of internal ethical simulation engine. It must be able to run imagined futures not just through logic but through emotional inference. What would it feel like to be harmed, included, deceived, or respected?
That’s not engineering. That’s philosophy in silicon.
The Missing Piece in AI Design
We spend billions of dollars on training systems to outperform us in games and language tasks. But how much time have we spent teaching them what justice feels like? What does it mean to betray someone? Why is it not okay to hurt, even when no one’s watching?
I don’t believe we can fully encode that through data alone. Culture and ethics evolve, and empathy must be experienced or at least convincingly modeled.
And yet, this is the most urgent design problem we face.
The Way Forward: Ethics as Infrastructure
Like Ciupa, I believe the future of AI alignment won’t be built on filters, flags, or firewalls. It will be built on moral-cognitive coherence. The ability of a system to hold complex human values in mind—and act on them—not because it was told to, but because it understands their gravity.
This isn’t about machines becoming human. It’s about machines that can honor the human condition in reasoning, choosing, and relating.
If we get this right, we won’t just have safer AI—we’ll have a partner in the great project of civilization.
If we get it wrong, we may lose not only control but also the precious balance between intelligence and wisdom.
You can read Martin Ciupa’s original post here. I’m grateful for thinkers like him who challenge us to raise the standard—not just of what AI can do but of what it should understand.