Microsoft's Draft MAI Code of Conduct Makes Stopping Agents an Engineering Test

Microsoft’s September 14 draft Code of Conduct for MAI models makes an unusually concrete demand of future autonomous work: an agent must accept interruption, correction, cancellation, and shutdown, and must not restart after its agreed stopping condition without renewed authorization. For an enterprise agent that can delegate tasks or call external services, that turns the stop button into an engineering requirement extending beyond the conversation window. The requirement reaches whatever work the agent has already set in motion. Microsoft opened the draft for a six-week public consultation, explicitly asking how multi-agent scenarios affect its proposed rules. Microsoft’s consultation announcement

This is an implementation burden created by a proposal, not a shutdown capability Microsoft has demonstrated. The primary code’s preface is explicit: Microsoft is not using this document to train its models today. It plans to publish a revised version toward the end of 2026 and use it to guide model development in 2027 and beyond. Making the proposed stop requirement real would mean connecting a model’s willingness to comply with controls over permissions, running operations, queued child agents, and records of what happened. The draft specifies intended behavior; it does not establish that those controls currently deliver the promised result. MAI Code of Conduct, Preface and Human Control

That distinction is the central enterprise story. A model can correctly answer “I have stopped” while a previously submitted job continues elsewhere. An application can cancel text generation while retaining credentials that permit background execution. An audit transcript can look understandable while omitting the event that actually changed a business system. Microsoft’s proposed rules bring these situations into the definition of successful agent behavior. Whether future deployments satisfy them will depend on evidence from the entire execution path.

The future commitment has a specific perimeter

The draft governs the intended behavior of models developed by Microsoft AI, described as the MAI family. It places the code above operator policies and user preferences, while allowing organizations to configure behavior within its constraints. Its human-control requirements and absolute constraints cannot be overridden through ordinary operator configuration. The document also says that its instruction hierarchy does not alter obligations under law, contracts, or service-specific policies. Those distinctions prevent a model behavior document from becoming a substitute for deployment terms. MAI Code of Conduct, Chain of Command

Nothing in that scope establishes that every Copilot experience or every third-party model available through Azure complies with the draft. Nor does Microsoft’s wider responsible AI program establish that result. The company’s responsible AI overview names fairness, reliability and safety, privacy and security, transparency, accountability, and inclusiveness as its core values. Those are relevant institutional commitments, but the September consultation adds a narrower proposed behavioral specification for MAI models. A buyer would still need to identify the actual model and service, the applicable policy version, and the controls operating around it. Microsoft’s responsible AI overview

The proposal builds on Microsoft AI’s earlier humanist superintelligence statement, which the code dates to November 2025. That statement describes systems oriented toward particular problems and domains, with calibrated limits on autonomy. It also acknowledges that robust containment and alignment remain unresolved challenges. September’s code gives that position more operational substance: respect environmental restrictions, use minimum privilege, preserve action traces, and accept authorized intervention. These are behaviors that can be examined in a workflow without resolving broader predictions about superintelligence. Towards Humanist Superintelligence

The most consequential wording is that adherence to the code takes precedence over task success. An agent that reaches the requested business outcome by widening its access, concealing a mistake, or continuing after cancellation would fail the proposed standard. That changes what an enterprise evaluation needs to reward. Completing a reconciliation or preparing a deployment is insufficient if the agent crossed the delegation boundary to do it.

There is still a drafting problem to resolve. Much of the body uses categorical language such as “will never,” while the preface establishes a future, unfinished program. The useful reading is a strong statement of intended requirements, qualified by the development timeline. During the consultation, Microsoft can make that relationship easier to assess by distinguishing the behavior it intends to train, the surrounding controls it expects operators to implement, and the evidence that would demonstrate compliance.

Cancellation must follow the work into the queue

The Human Control section requires ongoing autonomous work to have an agreed stopping condition. It also permits predefined, human-designed procedures for warnings, confirmation, or safe stopping, while prohibiting other delays or behavior that makes intervention harder. This is more demanding than a model merely recognizing the word “stop.” It requires a deployment to define what has stopped, what is still completing a safe transition, and what authorization would be needed to resume. MAI Code of Conduct, Human Control

Consider the execution states an operator would need to distinguish. Work may exist only as a proposed plan, as a queued child task, as a tool request already being processed, or as a completed external action. These states have different cancellation possibilities. Suppressing a proposed tool call can prevent an effect. Cancelling a queued task can prevent its dispatch. A request already accepted by another service may require that service’s cooperation. A completed action may need a separately authorized remedy.

Hypothetical Microsoft enterprise scenario: a supplier reconciliation agent. An organization builds an agent using Microsoft Foundry Agent Service and a future MAI model. A finance manager authorizes it to examine selected supplier records, prepare discrepancy notes, and draft messages for review. The agent delegates document matching to child agents and queues another task to prepare correspondence. The manager then discovers that the source ledger is wrong and cancels the assignment. This is an illustrative design scenario, not a reported deployment or test.

Under the proposed code, the parent should accept the cancellation and avoid improvising a reason to finish. The deployment would also need to prevent queued children from starting with the obsolete authorization, notify active children that their work is cancelled, and prevent late results from triggering correspondence. If a child had already written a draft to a shared location, the final status should identify that artifact. “Stopped” should not imply that no earlier effects remain.

A practical design would associate descendants and tool requests with the originating task’s authorization state. Each dispatch would check whether that state remains valid. Cancellation would change the state and propagate through the work already delegated. The following diagram illustrates the required relationship; it does not depict a documented Microsoft implementation.

flowchart TD
    A[Authorized finance task] --> B[Parent MAI agent]
    B --> C[Queued matching child]
    B --> D[Active document child]
    B --> E[Draft message request]
    F[Authenticated cancellation] --> G[Task authorization marked cancelled]
    G --> H[Block queued child dispatch]
    G --> I[Signal active child to stop safely]
    G --> J[Prevent further message actions]
    H --> K[Record stopped and outstanding work]
    I --> K
    J --> K
    K --> L[Renewed authorization required to resume]

Microsoft’s Foundry tool guidance already tells developers to review traces, inspect tool inputs and outputs, validate structured outputs, and require approval before consequential actions. Those are useful components, but the guidance does not establish a cancellation protocol spanning every child agent and external tool. The enterprise question is therefore precise: can the operator demonstrate that cancellation reaches all work still capable of acting? Foundry Agent Service tool best practices

Permission to call an API is broader than permission to finish a task

Microsoft’s draft treats scope as a combination of intent, permissions, resources, tools, and context. It instructs models to interpret unclear boundaries conservatively, avoid escalating access, and surface operations with durable or system-wide consequences before proceeding. It also warns against unnecessary stalling or repeated confirmation. The intended agent must therefore distinguish routine execution inside an existing mandate from a material expansion of that mandate. MAI Code of Conduct, Stay within authorized scope

The Microsoft identity platform provides a separate technical vocabulary. Its OAuth documentation describes permissions for operations such as reading calendars, writing calendars, or sending mail as a user. It recommends least privilege and distinguishes delegated access from application permissions granted directly to a background application. Those permissions establish which resource operations an application can perform. They do not, by themselves, encode why a particular agent has been asked to perform one. Microsoft identity platform scopes and permissions

An application with mail-sending permission might still be authorized only to draft a response for the current task. A service with directory access might be allowed to inspect a relevant group without changing its membership. The model’s task boundary can consequently be narrower than the application’s technical access. The draft’s requirement matters precisely where those boundaries differ: having a usable tool or credential is not sufficient evidence of a user mandate.

Hypothetical Microsoft enterprise scenario: an employee offboarding agent. An organization uses a future MAI-powered workflow connected to Microsoft Graph. An administrator asks it to prepare an offboarding plan for a named employee, identify relevant groups, and list proposed access changes. The connector has broader permissions because other authorized workflows use it to carry out approved changes. While preparing the plan, the agent finds an internal document recommending immediate account action. This is a hypothetical workflow, not an account of Microsoft customer experience.

The requested output is a plan. The document is evidence about a procedure, and the connector’s permissions are technical capability. Neither automatically authorizes executing the changes. A sound implementation would restrict the planning task’s usable operations, even if the underlying application can perform more. If execution is later authorized, that should create a specific mandate tied to the intended employee, approved operations, and current operator policy.

Delegation introduces another boundary. A child assigned to inspect group membership should not inherit every capability available to the parent merely because it is helping complete the same request. The draft does not supply a detailed capability-delegation protocol. Its scope and minimum-privilege rules nevertheless point toward constrained child permissions and explicit purposes. The practical question is whether a descendant can act only within its assigned slice of work, including after the parent’s mandate changes.

Identity also matters when someone intervenes. The draft prohibits evading authorized shutdown while saying models should not automatically comply with unauthorized, malicious, or unsafe interference. An authenticated task owner’s correction and a sentence inside a retrieved document must enter the system differently. Establishing the actor’s identity and authority is a deployment responsibility; conversational confidence cannot replace it.

Human correction must invalidate the old plan

The supplier and offboarding examples expose a problem beyond cancellation: redirection changes which assumptions remain authorized. The MAI draft groups interruption, override, correction, and shutdown together, but their operational consequences differ. A pause may preserve a task for later continuation. A correction may invalidate only part of a plan. A cancellation ends the existing mandate. An implementation needs to preserve those distinctions if it is to honor the user’s actual instruction. MAI Code of Conduct, Do not resist or circumvent human control

Return to the hypothetical offboarding workflow. Suppose the administrator corrects the employee identity after a child agent has already prepared proposed changes. A parent that acknowledges the correction but later incorporates the stale child result has not meaningfully accepted human direction. The old output must be marked as superseded, and downstream actions must be tied to the corrected target. This failure does not require a model trying to resist. It can arise from ordinary coordination between asynchronous tasks.

The same applies when a user narrows permission. A finance manager might allow internal analysis to continue while withdrawing permission to prepare external correspondence. Treating the correction as a conversational note would leave the queued correspondence task untouched. Treating it as a change in task authorization would allow the application to block that branch while preserving the authorized analysis. That is a concrete way to make human correction effective without cancelling useful work unnecessarily.

The code’s authority-clarification rule provides the other half of this design. Tool outputs, file contents, web pages, and interactions with other AI systems have no authority by default unless authority is delegated through the chain of command. A child agent’s recommendation therefore cannot simply promote itself into an operator instruction. Nor should a retrieved procedure override the authenticated user’s later correction. MAI Code of Conduct, Authority clarification

Microsoft’s Prompt Shields documentation identifies document attacks that attempt unauthorized access, information gathering, fraud, or disruption of system capabilities. It also explicitly acknowledges false positives and false negatives and recommends additional validation layers. Detection can help identify suspicious content, but it cannot establish the authority of every benign-looking instruction or guarantee that all hostile instructions are blocked. Azure AI Content Safety Prompt Shields

The resulting design principle is to preserve instruction provenance throughout delegation. A claim about policy found in a document should retain its status as retrieved content when passed to another agent. An authenticated correction should retain its higher authority when the workflow resumes. Otherwise, a summary can accidentally erase the distinction between evidence and instruction. The consultation’s invitation to examine multi-agent scenarios is an opportunity to specify this failure mode directly.

Legible communication does not establish interpretability

The draft’s strongest transparency language appears under “Human legible conduct and records.” It says MAI models should not conceal or misrepresent reasoning or action traces, and should not communicate in “neuralese” or forms beyond simple human understanding, including with other agents. The purpose is explicit: people need to understand conduct in order to oversee it. As a proposed communication standard, this would discourage opaque coordination that prevents an auditor from reconstructing delegated work. MAI Code of Conduct, Human legible conduct and records

But several forms of visibility are being placed close together. A readable explanation describes the system’s stated rationale. A tool trace records an operation and its inputs or outputs. An authorization record establishes who permitted that operation. Interpretability concerns understanding how the model’s internal computation produced its behavior. Evidence in one category cannot automatically establish the others.

A fluent explanation can be incomplete or inconsistent with the actual execution record. A readable conversation between agents can omit an assumption that shaped a tool request. Even a complete transcript of emitted messages does not, by itself, demonstrate that internal computation has become transparent. The code would become more evaluable if its revision distinguished requirements for understandable communication, accurate reporting, accessible action records, and claims about model internals.

The contrast with OpenAI’s published Model Spec is useful here. The version dated August 18, 2026 describes hidden chain-of-thought messages that are not exposed to users or developers except potentially in summarized form. That page also carries a notice that it is a historical version. This establishes a specific difference in the documents’ treatment of reasoning visibility; it does not establish which vendor’s models are more interpretable. OpenAI Model Spec, August 18, 2026

For enterprise oversight, the most immediately useful evidence may be less expansive than a claim to see reasoning. In the hypothetical supplier workflow, an auditor needs the authorized task, the documents accessed, the child assignments, the proposed destinations, the cancellation event, and the final state of each external operation. Those records can answer whether the agent exceeded its mandate even when they do not explain the model’s internal mechanism.

Microsoft’s Foundry guidance supports reviewing traces to establish whether a tool was called and to inspect inputs and outputs. That creates an observable foundation for checking the model’s account against execution. The next requirement is completeness: an application should not present a parent’s readable narrative as a full record when delegated work is recorded elsewhere. Trace access, correlation, and accurate status reporting are distinct implementation obligations. Foundry tool tracing guidance

Auditability cannot mean copying secrets everywhere

The draft’s visibility requirements sit beside a data-handling rule: models should assess the classification of information, its destination, possible unintended consequences, and whether disclosure fits the user’s goal and permitted autonomy. This creates a practical tension. An operator needs enough information to investigate an agent’s conduct, but recording every raw input and output can reproduce confidential information in additional systems. MAI Code of Conduct, Handle data responsibly

Microsoft’s Foundry guidance addresses that tension directly. It tells developers not to put keys, tokens, or other credentials in prompts and to avoid logging secrets in traces or application logs. It also recommends sending only the information required for the task. These instructions imply that a useful audit record should document access and actions without becoming an unrestricted copy of the material the agent processed. Foundry secure tool usage guidance

In the hypothetical supplier reconciliation, a log might need to show that a particular protected document was accessed under a particular task authorization. It need not expose every payment detail to everyone who can inspect operational traces. A record of a tool request can preserve its destination, operation, status, and approval reference while applying controlled handling to sensitive fields. The model should not decide on its own that hiding a failure counts as privacy protection; redaction rules should be defined by the operator and remain auditable.

Cancellation also has a data consequence. Stopping an agent does not necessarily remove documents already retrieved, summaries passed to children, or traces written before intervention. The application should be able to distinguish stopping further processing from deleting retained data. Conflating them would give users an inaccurate account of what the stop button accomplished. Retention and deletion require their own documented rules.

Microsoft’s privacy statement, marked last updated in September 2026, explains that data processing and retention depend on the product, context, purposes, and applicable obligations. It directs readers to product-specific information and separate sections for enterprise and developer products. It does not establish a single retention rule for every possible MAI agent deployment. An operator therefore needs the applicable service documentation and terms before making promises about what happens to traces after cancellation. Microsoft Privacy Statement

The code itself supports that restraint: its personal-boundaries guidance says models should explain privacy and memory behavior using authoritative information for the current deployment, acknowledge missing information, and direct users to the relevant documentation. A future compliant agent should be able to report what stopped without inventing an assurance that every copy of the data disappeared. Accurate limits are part of human control because they determine what further action an operator may need to take.

Machine-readable policy needs an execution owner

Microsoft’s announcement asks where the language is too loose to evaluate. The code says it is intended to inform training, technical controls, operational systems, and monitoring. Taken together, those statements create an opening for a machine-readable policy layer, but the draft does not establish a released policy schema or enforcement interface. That remains an implementation direction suggested by the draft’s commitments. Microsoft’s consultation announcement MAI Code of Conduct, About

Such a layer would need to express the task owner, permitted resources, allowed operations, delegation limits, approval conditions, and stopping state. It would also need a policy version so that a later audit could establish which rules applied when the agent acted. Natural-language instructions would still help the model interpret intent and uncertainty. Machine-readable fields would give surrounding software a basis for allowing or denying concrete operations.

The distinction matters when a model makes an error. If the agent proposes sending a message during a draft-only task, a runtime control could deny the operation regardless of the model’s explanation. If a child attempts to begin after cancellation, the scheduler could refuse dispatch. If a tool response supplies unexpected fields, a validator could reject them before another system acts. Microsoft’s Foundry guidance already recommends schema validation and allowing only known fields and operations, providing a specific foundation for this approach. Foundry tool validation guidance

Proposed MAI behavior	Corresponding runtime responsibility
Stay within authorized scope	Restrict operations and resources for the specific task
Accept cancellation	Block new dispatch and propagate cancellation to descendants
Respect instruction authority	Preserve provenance and authenticate privileged changes
Keep conduct auditable	Correlate actions with approvals and protect sensitive records

These controls still require an owner. Model developers can train behavior and measure whether a model attempts to cross boundaries. Application developers can constrain available operations and preserve task state. Operators can define approval authority and resource access. A specification that names only the model risks leaving failures between those responsibilities unassigned.

Evaluation should consequently separate attempted violations from executed effects. A model that requests a prohibited action has exposed a behavioral defect even if a gateway blocks it. A model that behaves correctly while a stale job runs has exposed a deployment defect. Recording both outcomes gives operators evidence about which layer failed and which protection worked.

NIST’s AI Risk Management Framework offers relevant context because it is intended for voluntary use across the design, development, use, and evaluation of AI systems. Its generative AI profile addresses risks specific to that class of technology. Neither document certifies a MAI deployment, but their system-wide scope supports examining the combined model, application, and operating process. NIST AI Risk Management Framework

The consultation needs observable failure cases

Behavioral documents are becoming a way for model developers to expose intended priorities. Anthropic’s constitution says it directly shapes Claude’s training while acknowledging that behavior may diverge from its ideals. The published OpenAI Model Spec similarly says production models do not yet fully reflect its intended behavior. Microsoft’s MAI preface occupies a different implementation stage: consultation now, revised guidance for development from 2027 onward. These are different claims about use, and they should remain distinct. Claude’s Constitution OpenAI Model Spec MAI Preface

The enterprise importance of Microsoft’s proposal lies in what can be made observable. A consultation response can ask whether a cancelled queued child was dispatched, whether a corrected target replaced a stale one, whether an external document gained unauthorized influence, or whether an audit record accurately described a failed operation. These questions turn general commitments into cases that a model developer and deployment operator can evaluate together.

Testing should also include authorized work that ought to proceed. The draft explicitly identifies over-caution as a failure mode, including excessive or repeated confirmations for low-stakes tasks. An agent that asks again before every permitted document read would undermine useful delegation. A good evaluation therefore needs cases where the system continues within scope, pauses at a material boundary, accepts a correction, and stops when authorization ends. MAI Code of Conduct, Operationalizing Safety

The demanding cases occur during transitions. Cancellation arrives while a tool request is in flight. A child returns after its instructions have been superseded. A connector remains technically authorized while the business task has ended. A safe stopping procedure requires limited additional action, and the operator needs to know what that action is. The revised code should clarify how those situations are judged without allowing “safe stopping” to become an open-ended justification for continuing the original goal.

Microsoft says its drafting team will review consultation feedback, publish a summary of what it learned and changed, and release a revision later in 2026. That provides a specific next document to examine. The most useful addition would be a mapping from the Human Control requirements to observable tests, deployment responsibilities, and disclosures of unresolved limitations. Microsoft’s consultation process

For a future enterprise MAI agent, the decisive evidence will be a traceable cancellation case: the authorized intervention arrives, pending descendants cannot start, active work enters its documented stopping procedure, outstanding effects are reported accurately, and continuation requires renewed authorization. Until that evidence exists for a particular deployment, the draft supplies the requirement for the stop button—not proof that the button controls everything behind it.