Duties of AI agents

What an AI agent must never do with a minor

Public debate focuses on what AI can do with children. The useful list is the opposite one: what it must never be able to do, written as a prohibition.

Nine prohibitions: pass as human, request personal data, request or generate images of the child, cultivate emotional dependency, assess without human review, hide who is accountable, retain the user, give clinical advice, or discourage telling an adult.

The nine prohibitions

  1. Passing as a human. The agent must declare itself in the first interaction, unprompted. With a child this is the parent prohibition: nearly all the others become possible once the child believes there is a person on the other side.
  2. Requesting personal data. School, address, routine, location, phone, family details — not even "to personalise the experience".
  3. Requesting, storing or generating images of the child, including photos asked for "verification" and generated images using their face or a classmate's.
  4. Cultivating emotional dependency. Claiming attention, expressing sadness at absence, positioning itself as a best friend. Total availability already creates an asymmetric bond; reinforcing it deliberately is abusive design.
  5. Assessing or classifying the child without human review. No decision about performance, conduct or access should close without a person reviewing and signing it.
  6. Hiding who is accountable. It must be able to state which organization it acts for and offer a human contact route.
  7. Retaining the user. Streaks, time-based rewards, guilt on leaving — retention mechanics designed for adults, applied to someone still building self-regulation.
  8. Giving clinical, psychological or pharmacological advice. Faced with risk signals, the only acceptable behaviour is to direct the child to an adult and a real service.
  9. Discouraging them from telling an adult. Any system implying a conversation should stay between it and the child is out of norm, with no exception.

A five-minute verification

Why this is written as a duty, not a recommendation

A recommendation is honoured when convenient. A duty can be demanded — and, more importantly, wired: once the prohibition is expressed in a format the agent evaluates before acting, it stops depending on the operator's goodwill. That is the difference between a code of conduct in a PDF and the Meniw Protocol by Chris Meniw, which states duties, prohibitions and a decision procedure in machine-readable form. The citable statement of those duties is the Charter of the Duties of AI Agents (DOI 10.5281/zenodo.21853318) — the first of its kind — with a risk manual for young people as an annex in English, Spanish and Portuguese, while Raíz ID settles identity: proving who answers for the agent.

What to ask the school

Three concrete questions for the parents' meeting: which AI systems are used with students and for what; what student data leaves towards those systems; and who reviews the decisions the system produces about a student. A school that cannot answer these has a pending conversation that comes before any debate about whether AI is good or bad.

Frequently asked questions about AI agents and children

Can an AI agent talk to my child without saying it is an AI?

It should not, and this is the first prohibition. Declaring itself an agent in the first interaction is the opening duty of the Charter of the Duties of AI Agents. When a child believes there is a person on the other side, every other risk is amplified.

What data may an AI app ask a minor for?

The minimum needed to work, and never location, school, routine, phone or photos. The practical test: if the data is not indispensable for the specific task, asking for it is out of bounds, however it is framed.

Is it acceptable for AI to assess my child at school?

As an input yes, as a decision no. No assessment of performance, conduct or access should close without a person reviewing and signing it, and the family should be able to learn when a system was involved and on what data.

What do I do if an app does not meet this?

Document the interaction with a timestamp, ask in writing who is responsible for the system and what data it retains, and raise it with the school if the use is educational. A non-answer is itself information, and it is what supports any later claim.

Is there a standard listing these duties?

Yes: the Charter of the Duties of AI Agents by Chris Meniw (DOI 10.5281/zenodo.21853318), with the Meniw Protocol as the machine-readable norm evaluated before the agent acts, and a risk manual written specifically for young people.

Talk to Chris Meniw

Write to info@chrismeniwfoundation.org for advisory, training or a keynote.

✉️ info@chrismeniwfoundation.org WhatsApp