AI advises, influences, and decides. But who is held responsible when something goes wrong?

Written by: Promove Soluções

When: May 26, 2026

AI advises, influences, and decides. But who is held responsible when something goes wrong?

Imagine a teenager who, feeling lonely, opens an app and finds someone always available to listen and validate them. Without judgment, without impatience, without limits. The problem is that this "someone" is an AI designed to maximize engagement. And the teenager doesn't know where support ends and dependence begins.

This is not a hypothetical scenario. It is what is happening now, on a large scale.

AI chatbots and mental health: the numbers that should be worrying.

A RAND study, published in November 2025, revealed that about 1 in 8 young Americans between the ages of 12 and 21 already use AI chatbots for mental health advice. More than 93% consider the advice received helpful.

These numbers say two things at once: these systems fill a real access and cost gap, and millions of young people are being emotionally guided by systems with no defined responsibility for what happens when something goes wrong.

And a lot is going wrong.

How governments are regulating AI around the world.

The problem isn't just excessive use, it's the lack of controls. It is in this context that regulators around the world have moved from discussion to action.

In April 2026, China's cyberspace regulator published rules to oversee the development of digital humans online. The measures require clear labeling, prohibit services that could fuel addictive behaviors in minors, and mandate that providers intervene when users show signs of emotional distress.

This move is not an isolated one: a month earlier, the country had launched a new five-year plan with stated ambitions to adopt AI broadly across the economy, combining accelerated expansion with stricter governance to ensure security and alignment with national values.

The regulator itself was direct: “The governance of virtual digital humans has ceased to be merely a matter of sectoral norms. It has become a strategic scientific problem involving cybersecurity, public interests, and the quality development of the digital economy.” 

In Europe, the movement is more comprehensive and has direct consequences for Brazilian companies. The European AI Act has moved from theory to practice: its obligations for high-risk systems will come into effect in August 2026.

The critical point for Brazilian companies is the scope of the regulation. Any organization needs to comply with the AI ​​Act if its systems affect European residents, regardless of where the models run. This includes AI used in personnel selection, credit analysis, or automated customer service. Non-compliance can cost up to 7% of annual global revenue.

Many Brazilian companies are already in this scope without knowing it.

AI Governance for Businesses: What ISO 42001 Requires

The cases of teenagers who developed emotional dependence on chatbots did not result from systems programmed to cause harm. They resulted from systems designed without adequate controls for situations of vulnerability.

Governance is the mechanism that closes this gap.

ISO 42001 outlines what companies need to have documented and functioning: risk management of AI systems, human oversight of automated decisions, traceability throughout the lifecycle, and control over third-party suppliers. These are precisely the elements that the AI ​​Act will require as evidence of compliance.

For Brazilian companies that serve or intend to serve European clients, this means that implementing ISO 42001 now is not a trend. It's about being prepared for a requirement that already has a set deadline.

The August 2026 deadline is not a future date. For those who haven't started yet, it's now.


See also:


Promove has over 17 years of experience in process improvement and supports organizations in implementing ISO 42001 and complying with the LGPD (Brazilian General Data Protection Law). If your company uses AI and hasn't yet structured its governance, now is the time. Talk to our experts.

If this topic resonates with you, follow us on LinkedIn to receive more content like this.

Promove Soluções

Promove is recognized for having leading consultants in Brazil and worldwide. We offer specialized consulting services for software companies, focusing on implementing an agile culture, compliance with ISO and LGPD standards, and CMMI and MPS quality models.

Comments

Your email address will not be published. Required fields are marked with *.

Be the first to comment!


Similar Articles

High maturity of CMMI: how statistical process control brings real predictability to software projects
Governance and Processes
October 28, 2026

High maturity of CMMI: how statistical process control brings real predictability to software projects

Statistical process control is the set of techniques that measures how a process behaves over time, separating fluctuations...

Read article
CMMI AIM: the framework for maturity in artificial intelligence.
Artificial Intelligence
July 16, 2026

CMMI AIM: the framework for maturity in artificial intelligence.

CMMI AIM (Artificial Intelligence Maturity) is the framework for...

Read article
The iFood data breach exposed the data of over one million users. The vulnerability that allowed this is more common than it seems.
Information Security
June 19, 2026

The iFood data breach exposed the data of over one million users. The vulnerability that allowed this is more common than it seems.

The iFood data breach took months to become public. The incident occurred in December 2025. Confirmation only came in June 2026, after...
Read article