Recently I started watching Witch Hat Atelier and realized that we are living in a somewhat similar world when we think about artificial intelligences. There are some people who control this power, there are problematic artificial intelligences and, if we use magic — or an artificial intelligence — the wrong way, we can destroy the world. Or, at least, mess it up a little.
The magic circle — or Sigil — in Witch Hat Atelier follows some rules, but, basically, aside from the fact that it needs to be drawn with a special magical ink, anyone who knows the symbols and knows how to draw them can do magic. This is precisely one of the big secrets of that world: magic is not an ability that some people are simply born with; anyone could use it if they knew how the circles are built.[1]
Ah, it is worth remembering: if you have not watched the first season of the anime, there will be spoilers, okay?
Even by tracing over a drawing, someone can reproduce a spell. Which can be a problem if you do not know exactly what you are doing.
This world of magic circles is not that far from current AIs. We have some AI users who can do a few extra tricks with LLMs, especially developers, but, with a little practice and knowledge, practically anyone can ask a model to write texts, code, research information or control other tools.
And that is where some problems begin.
First: if anyone can create any magic or any computer program, what prevents that magic or that program from being used for evil?
Second: even if a Magic or Program was created to be useful, what guarantees that the creator has enough knowledge to prevent it from doing something wrong?
Third: if a Magic, Program or System is strong enough to cause gigantic damage to humanity, should people still be able to write their own magic circles or use an Oracle to write their programs?
Protecting humanity from itself
In Witch Hat Atelier, they chose a specific path to protect humanity from itself.
After wars, accidents and destructive uses of magic, the so-called Day of the Pact emerges. From that point on, knowledge about the true functioning of magic comes to be kept by a restricted group of witches. The population’s memories about that knowledge are erased, and those who do not know the secret become the Ignorant.[2]
In addition, there are specific rules:
- A witch must not allow an Ignorant person to discover the secret of how magic is performed.
- Certain types of magic are forbidden, including spells capable of directly affecting body or mind and transformations of living beings.[2]
- If an Ignorant person discovers how magic really works, their memory of that knowledge may be erased.[2][3]
It is a very extreme solution. It is not enough to control the use of magic: the very knowledge of how it works also needs to be protected.
We have something similar happening with LLM models, do we not?
Certain types of assistance are deliberately restricted when they could facilitate serious harm. Frontier labs already explicitly evaluate capabilities related, for example, to biological, chemical, cyber and autonomy risks before or during the release of advanced models.[7][8][9]
Many of the largest LLMs are also closed models. The user can use that “magic”, but does not necessarily receive the model weights, the data used in training, the internal instructions or all the details about how that capability was built.
When we use a closed model through an API or a chat interface, we can ask it to perform a task. We can use the magic, but we do not necessarily receive the Sigil behind it.
In a model with open weights, the situation starts to change. You are not just using the magic: you are receiving a much larger part of the Sigil and, depending on your knowledge, you can modify it, adjust it or use it to create something completely different.
A clarification is worth making here: when I talk about “open models” in this text, I am talking mainly about open-weight models. Having access to the weights does not necessarily mean that the dataset, the training code and the entire model construction process are also open.[4]
Witch Hat Atelier takes this logic of control to its ultimate consequences. It is not enough to prevent someone from using certain spells. The knowledge that anyone could draw these magic circles also needs to remain hidden.
And this is where a much bigger problem appears.
There is an important difference between controlling a tool and controlling the knowledge needed to build that tool.
In the world of Witch Hat Atelier, the witches decided to control both.
When everyone can draw magic circles
In our reality, though, “Magic” seems to be following a somewhat different path, but one that may end up crossing with the world of Witch Hat Atelier.
Closed frontier models already go through specific risk assessment and mitigation processes as their capabilities increase.[7][8][9] And open-weight models are getting closer and closer to closed systems across different metrics.
Stanford’s AI Index 2026 shows this convergence well. In May 2023, the gap between the best closed model and the best open-weight one in the Arena was about 15.2%. In August 2024, it reached just 0.5%. The gap grew again with new proprietary models, but, in March 2026, it was still at approximately 3.3%.[4]
That is: the distance varies as new generations are released, but models that can be downloaded and modified are much more competitive than they were a few years ago.
And this is also starting to show up in regulation.
In the European Union, for example, the AI Act provides additional obligations for general-purpose models considered to be of systemic risk. Models distributed under free and open licenses may receive some exemptions, but those exemptions do not apply in the same way when the model is classified as systemic risk.[5][6]
Even more interesting: the European Commission itself recognizes that a model may present systemic risk not only because of its size or training cost, but also because of factors such as reach, scalability and the scaffolding around it — that is, the tools and systems that expand what that model is able to do.[5]
And that leads us to an interesting problem.
Maybe the biggest risk is not even the number of parameters.
What happens when a model small enough to run on a home computer is capable of coordinating dozens or hundreds of agents performing complex tasks autonomously?
What happens when anyone can assemble a small swarm of agents to research, program, test systems or execute a task for hours with little human intervention?
At that point, it does not matter so much whether that model has 15 billion, 100 billion or one trillion parameters.
What matters is the capability that has been placed in someone’s hands.
I can imagine a scenario in which we will have restrictions increasingly similar to those in the world of Witch Hat Atelier. Maybe our memory will not be erased after discovering how a Transformer works — I hope —, but we may reach a point where some capabilities are considered too dangerous to be distributed without any kind of control.
But that creates another question.
Is there a way for all human beings to use magic — I mean, state-of-the-art models — without us blowing ourselves up along the way?
Everyone should be able to learn magic
I think so.
But maybe the answer lies neither in releasing absolutely everything nor in turning some companies, governments or labs into a kind of Council of Witches.
Because both extremes have problems.
If we release absolutely all capabilities without any kind of concern, we need to accept that ill-intentioned people will also receive exactly the same tools. There is no version of democratization in which only nice people get access.
On the other hand, if we decide that advanced artificial intelligence is too dangerous for the population, then we start to build a very small class of people and organizations that can use one of the most powerful technologies ever created.
In that scenario, a few groups would not only be responsible for building the Sigils.
They would also choose which Sigils are allowed to exist.
And that seems equally dangerous to me.
Maybe there is a third path.
Instead of dividing the world between Witches and the Ignorant, we can assume that everyone should have the right to learn magic, but that not all capabilities necessarily need to be made available in the same way.
Models can be open, research can be open. People can study how these systems work, create their own models, run agents and experiment with new ways of using them.
But capabilities able to produce gigantic harm may require additional layers of security.
The greater a system’s capacity to cause harm, the greater the requirements for auditability, isolation, traceability and responsibility could be.
It would not be:
“You cannot learn magic.”
It would be something closer to:
“You can learn magic. But some spells require more responsibility to be used.”
Maybe the witches’ mistake in Witch Hat Atelier was not realizing that some spells were dangerous.
Maybe the mistake was concluding that, to prevent those spells, they needed to hide the very existence of magic.
The problem is not the Sigil
It is worth remembering that a Sigil is not necessarily good or evil. An artificial intelligence model is not either.
The same system capable of looking for a vulnerability in thousands of computers can be used to find those vulnerabilities before an attacker does. An agent capable of manipulating a complex system can also be used to monitor it, fix errors or prevent something from getting out of control.
So maybe our defense against extremely capable artificial intelligences is not to build systems that are less and less capable. Maybe it is to build defensive systems that are equally capable. If anyone can have a small army of agents attacking something, maybe our computers also need their own agents protecting them.
If models can find flaws in seconds, maybe the infrastructure also needs models continuously looking for those flaws.
A kind of digital immune system, or more or less what your antivirus does every day. This still does not eliminate the need for rules, but it breaks the idea that our only possible protection would be to hide knowledge.
After all, who should control magic?
Maybe this is one of the most important questions we will have to answer in the coming years.
Not just:
“How intelligent can an AI become?”
But:
“Who will be able to use it when it gets there?”
I would not like to live in a world where only a few companies have access to the most powerful intelligences in existence. But I also think that living in a world where anyone can press a button and set hundreds of autonomous agents to do absolutely anything, without any limit, does not sound very nice either.
Maybe the balance lies in democratizing knowledge and, at the same time, treating truly dangerous capabilities as what they are: dangerous capabilities.
Without turning every user into an Ignorant, nor every researcher into a Pointed Hat. And, above all, without turning a few organizations into the only authorized Witches on the planet.
Witch Hat Atelier imagines a world where humanity solved the danger of magic by hiding its secret. That choice no longer exists and perhaps should not even exist.
Our magic is already being used and continues to evolve. And, probably, more and more people are going to learn to draw their own magic circles. The question now is no longer whether we should allow humanity to discover magic. It is figuring out how to let everyone use it without destroying the world in the process.
Bibliography
-
Kodansha — official Witch Hat Atelier website. Presents the central premise that anyone can use magic by drawing the circles with the correct tools and that this fact is kept secret by the witches.
https://morning.kodansha.co.jp/tongari_official/ -
Independent Witch Hat Atelier Wiki — Day of the Pact. Reference for the Day of the Pact, prohibition of spells that affect body and mind, transformation of living beings and rules about revealing the secret of magic.
https://witchhatatelier.telepedia.net/wiki/Day_of_the_Pact -
Independent Witch Hat Atelier Wiki — Memory Erasure. Reference for the use of memory-erasing magic on people who discover the secret of magic.
https://witchhatatelier.telepedia.net/wiki/Memory_Erasure -
Stanford Institute for Human-Centered AI — AI Index Report 2026, Technical Performance. Data on the performance difference between closed-weight and open-weight models and explanation of the distinction between different levels of openness.
https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance -
European Commission — General-Purpose AI Models in the AI Act: Questions & Answers. Explains general-purpose models with systemic risk, criteria such as reach, scalability and scaffolding, as well as the rules applicable to open-source models.
https://digital-strategy.ec.europa.eu/en/faqs/general-purpose-ai-models-ai-act-questions-answers -
European Union — Regulation (EU) 2024/1689, Artificial Intelligence Act. Legal text of the AI Act, including the classification and obligations for general-purpose AI models with systemic risk.
https://eur-lex.europa.eu/eli/reg/2024/1689/oj -
OpenAI — Preparedness Framework, Version 2. Framework to track frontier capabilities and risks related to biology/chemistry, cybersecurity and other domains, associating higher capabilities with additional safeguards.
https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdf -
Anthropic — Responsible Scaling Policy. Risk governance framework that connects advancing model capabilities to progressively greater evaluations and safety measures.
https://www.anthropic.com/responsible-scaling-policy -
Google DeepMind — Frontier Safety Framework. Framework for identifying advanced AI capabilities capable of producing severe harm and applying risk-proportionate evaluations and mitigations.
https://deepmind.google/frontier-safety/