Ropes & Gray’s AI Diligence Workflow Shows Where Legal Agents Need Human Judgment

Transactional diligence is a bad place for theatrical autonomy. A deal team may have thousands of contracts, little time, and no tolerance for a missed change-of-control clause. Ropes & Gray’s October 29 account of using OpenAI models for transactional diligence is interesting because it places AI inside that pressure cooker. The useful system is not a robot lawyer making a closing decision. It is a workflow that finds clauses, extracts structured facts, identifies uncertainty, and sends the right questions to lawyers before the deadline.

The agent should produce a queue, not a verdict

A defensible architecture starts with bounded tasks. Convert each contract into text with page and paragraph coordinates. Ask the model to extract named fields—counterparty, renewal term, assignment language, consent requirement, governing law—and require a quoted span for every answer. Then run deterministic checks for missing fields, inconsistent dates, and clauses that exceed the deal’s risk threshold. The output is a queue of findings with evidence, not a green light. This design gives the lawyer something to audit. If the model says consent is required, the reviewer can jump directly to the clause. If it says the contract has no change-of-control provision, the system can show the passages it searched and the confidence boundary. Evidence-linked extraction is slower than a free-form summary, but it turns a fluent answer into a reviewable work product. In legal services, reviewability is part of accuracy. The queue also makes escalation explicit. High-risk clauses, low-quality scans, ambiguous definitions, and documents outside the trained playbook should route to a senior reviewer. A useful agent knows when to stop. That is less impressive in a demo than autonomous completion, but it is the behavior that keeps a deal team from treating uncertainty as absence.

Privilege and confidentiality become system properties

A law firm cannot bolt confidentiality onto an agent after deployment. Document routing, model tenancy, retention, logging, and access control all determine whether the workflow is acceptable. Matter identifiers should be separated from general analytics. Prompts and outputs need retention rules. Administrators should be able to prove which users accessed a document and which model processed it. The American Bar Association’s guidance on generative AI puts professional duties around competence, confidentiality, communication, and supervision into the same conversation as technical configuration. The risk is not limited to an outside provider. Internal search indexes, observability tools, browser extensions, and copied spreadsheets can widen the audience for a confidential deal. A system that sends extracted clauses to a general-purpose analytics workspace may be compliant at the model boundary and unsafe at the workflow boundary. Threat modeling must follow the data through every queue, cache, and export. Firms should publish a model register for legal workflows. It should name the provider, model version, allowed matter types, retention setting, evaluation suite, and human owner. When a model changes, the firm should rerun the tests that matter to diligence: missed clauses, fabricated citations, date extraction, definition scope, and behavior on poor scans. ‘The provider updated the model’ is not a risk assessment.

Evaluation must look like a deal room

Generic benchmark scores will not answer whether a diligence agent is safe. A firm needs a private evaluation set drawn from redacted agreements and annotated by lawyers. It should include ordinary contracts and adversarial cases: a clause hidden in an exhibit, a definition that changes the meaning of assignment, a scanned page with a missing character, and a provision that appears harmless until read with a schedule. Measure recall for material findings, citation accuracy, abstention quality, and reviewer time—not just answer similarity. The baseline matters. If junior reviewers currently spend six hours on a document set and miss three material provisions, an AI workflow should be compared with that process, not with an imaginary perfect lawyer. The goal may be to reduce first-pass time while holding missed-finding rates flat. In another matter, the right outcome may be slower review because the documents are unusually novel. Evaluation should reflect the decision, not reward speed in isolation. A deal team should also test human factors. Does a polished summary cause reviewers to skim evidence? Do lawyers override warnings because the queue is too long? Are uncertainty labels understandable under deadline pressure? An agent can improve extraction and still damage review if it creates false confidence. The interface is part of the control system.

The deployment lesson is narrower and more useful

Ropes & Gray’s workflow points toward a practical future for professional services: agents will handle document-scale inspection while licensed professionals own interpretation, exceptions, and advice. That future will not be achieved by adding a chatbot to a document folder. It requires task boundaries, evidence links, matter isolation, evaluations, and a workflow that makes escalation normal. Buyers should ask vendors for error taxonomies rather than glossy time savings. Which clauses are missed? How often does the system cite the wrong page? What happens when the document is multilingual, handwritten, or poorly scanned? Can a client export the audit trail? What does the provider do when the model changes? The answers reveal whether the product is an agentic workflow or a summary box wearing an agent label. The legal agent that wins will be the one a senior lawyer can interrogate at 11:45 p.m. before signing. It will show the source, the uncertainty, the rule that triggered escalation, and the human who resolved it. In diligence, trust is not the absence of automation. It is automation that leaves a path back to the document.

The details that decide production value

Document quality is the first hidden variable. Contracts arrive as native PDFs, image scans, email attachments, spreadsheets, and exhibits with inconsistent numbering. OCR can drop a “not,” merge columns, or misread a date. A diligence agent should classify extraction quality and route weak documents to a human before drawing conclusions. Confidence in the language model cannot repair missing text upstream.

Definitions deserve special handling. A change-of-control clause may refer to a defined term that appears dozens of pages earlier, and an assignment restriction may be modified by a schedule. Retrieval must preserve document structure, cross-references, and page coordinates. A summary that quotes the operative sentence without its definition can be more dangerous than no summary because it looks complete.

Playbooks should be explicit about materiality. One buyer may care about customer-consent requirements above a threshold; another may treat the same clause as routine. Encode the rule as configuration owned by the deal team, not as an unstated model preference. When the threshold changes, the system should show which findings change and why.

Privilege boundaries need matter-aware storage. A model may help with one transaction but must not allow retrieval across unrelated matters. Indexes, embeddings, temporary files, and logs all need tenant isolation. Deleting a document from the visible workspace is insufficient if a vector index or debugging trace retains its text. Security reviews should enumerate every copy.

Human review can be measured. Record which findings were accepted, corrected, dismissed, or escalated, and why. This creates a feedback set for the playbook without turning every lawyer into an unpaid annotator. The firm can learn whether errors come from extraction, retrieval, interpretation, or interface design. Each category needs a different fix.

Conflict checks should remain outside the model’s discretion. An agent can collect names and entities, but deterministic systems should run conflicts against the firm’s authoritative database. The same principle applies to closing conditions and client identity. Language is useful for finding candidates; controlled systems should make the decision.

False negatives deserve priority over fluent false positives. A long list of low-value findings wastes review time, but a missed termination right can alter a deal. Evaluations should weight errors according to consequence. The model may be allowed to abstain often if abstention sends a document to a capable reviewer. That is a rational trade, not a product failure.

Vendor case studies should disclose the baseline. Was the previous workflow manual review, keyword search, or an existing contract platform? How many reviewers participated? Were savings measured across the full matter or only a selected sample? Without those details, a reported acceleration cannot be translated to another firm.

Agent plans should be bounded by tool permissions. The extraction agent may read a matter folder and write a finding object, while a separate service handles exports or client-facing reports. Narrow tools make it harder for a prompt injection in a contract to cause an unrelated action. Every tool call should validate arguments independently of the model.

Change management matters because legal teams are risk-sensitive. Start with internal, low-consequence documents and let lawyers see evidence links before exposing the system to client deliverables. Publish a short usage policy that says when an AI-assisted finding requires additional review. Training should include failure examples, not only successful demos.

An agent can improve consistency without replacing expertise. A structured extraction schema forces the team to agree on what counts as a finding. That agreement is valuable even when a human completes the work. The model becomes a catalyst for a better playbook rather than a substitute for the lawyer who owns the judgment.

Clients will ask about use, and firms should answer plainly. Explain which tasks were assisted, what a lawyer reviewed, how confidential data was handled, and whether the model provider retained prompts. Concealing assistance is more damaging than describing a controlled workflow. Transparency lets the client assess the process against its own risk appetite.

Cross-document reasoning is where the system becomes genuinely useful and genuinely risky. A model can compare renewal dates or identify inconsistent definitions, but it may also invent a relationship between documents. Require the comparison to list source pairs and quote the fields that differ. Reviewers should be able to inspect both originals before accepting a synthesized finding.

After closing, the evaluation set should not disappear. Anonymized outcomes can show which findings mattered, which warnings were noise, and which provisions were missed. That feedback improves future matters and gives the firm evidence for governance reviews. Continuous evaluation is the difference between a pilot and an operating capability.

The professional-services market will reward agents that fit existing accountability. A system that saves an hour but creates an unexplained exception is not ready for sensitive work. A system that turns thousands of pages into a traceable review queue may be less flashy and more valuable. The winning product will respect the document, the deadline, and the person whose signature carries the risk.

Operating cases worth testing

An extraction schema should preserve negative findings. “No consent required” is a conclusion that needs evidence, not an empty cell. Store the passages searched and the rule used to reach the result. This makes a negative answer auditable and prevents blank data from being mistaken for a clean contract.

Deal documents often contain superseded amendments. The agent must identify effective dates and order the versions before comparing terms. A current clause can be hidden in a later amendment, while an old agreement may still govern a subset of customers. Version awareness is a basic diligence requirement, not an advanced feature.

Multilingual matters deserve dedicated evaluation. Translation can change legal nuance, especially around obligation, discretion, and conditions. The system should preserve the original language next to any translation and route consequential findings to a reviewer who can assess both. A fluent English summary is not proof of semantic equivalence.

Confidentiality also covers prompts and outputs that mention a client indirectly. Redaction should remove names only when the workflow remains useful; otherwise it may create false context. Matter-aware access and provider controls are safer than assuming a few string replacements make sensitive text anonymous.

When a reviewer corrects a finding, the system should not silently retrain on privileged content. Feedback pipelines need their own approval and retention policy. A useful lesson from one matter may be generalized as a rule or test case without exporting the underlying client document.

Closing checklists are a natural handoff point. The agent can show unresolved findings, missing documents, and pending consents, while the deal lead decides whether the package is complete. This keeps automation connected to the transaction’s actual control point rather than ending at a summary report.

Procurement teams should compare vendors on evidence handling. Ask whether citations survive export, whether page coordinates work after OCR, and whether a client can retrieve logs during an audit. A model leaderboard says little about these operational details, yet they determine whether the tool fits a law firm.

Latency has a legal workflow dimension. A result that arrives after the review meeting may be useless even if it is accurate. Measure queue time, not only model response time. Batch processing, priority routing, and visible progress can matter more than shaving a second from generation.

Governance should define prohibited uses. A diligence agent may assist review but not send a client opinion, make a privilege determination, or approve a closing condition without human authorization. Clear prohibitions reduce ambiguity for users and provide a basis for monitoring.

The safest claim is specific: the system reduces a defined review burden under a defined playbook and leaves evidence for a lawyer. That claim is less dramatic than “AI automates diligence,” but it is testable. Testable claims are what allow a firm to expand responsibly.

Final checks before a workflow goes live

A reviewer dashboard should show workload as well as findings. If one lawyer receives a hundred low-confidence alerts while another receives five high-risk exceptions, the system is distributing attention badly. Queue design is a professional-responsibility issue because it shapes what humans are likely to notice before a deadline.

Client reporting should preserve the distinction between extracted fact and legal interpretation. A report can say that a clause contains a consent requirement and separately record the lawyer’s view of materiality. Mixing the two makes later review harder and encourages users to treat a model label as advice.

Firms should test the workflow after a model provider changes rate limits or retention settings. Production risk includes service behavior, not only answer quality. A technically accurate model that becomes unavailable during signing creates a different operational failure, so fallback procedures should be rehearsed.

Comparing tools on a common redacted corpus is useful, but a single leaderboard should not decide procurement. Workflow fit, evidence export, access isolation, and reviewer adoption may matter more than a small difference in extraction accuracy. Legal technology is a system purchase, not a model purchase.

The most credible claim is that the agent makes review more inspectable. It forces findings into a schema, preserves source passages, and makes exceptions visible. That is a meaningful improvement even when every material conclusion remains with a lawyer.

Additional evidence boundaries

Diligence work is also a coordination problem. Specialists review employment, privacy, intellectual property, finance, and commercial contracts, then a deal lead assembles the risk picture. An agent can normalize findings across workstreams, but it should preserve the specialist’s source and qualification. Flattening every issue into one confidence score hides the expertise that makes the review useful.

Law firms should define an evidence retention schedule before the first upload. Keeping everything forever creates security and discovery risk; deleting too quickly prevents investigation and client reporting. The schedule should cover originals, OCR output, embeddings, prompts, findings, exports, and evaluation copies. Matter closure is a real system event, not merely a folder rename.

A careful pilot can therefore demonstrate more than speed. It can show that the firm has a repeatable way to bound tasks, inspect evidence, protect matters, and learn from corrections. Those are the capabilities clients will care about when the novelty of an AI assistant has disappeared.

Evidence readers can inspect

The primary announcement is the starting point, not the whole record. The links below are the direct documents and standards used to check dates, product descriptions, governance assumptions, and the boundaries of the claims in this article. They are included so a reader can inspect the evidence rather than accept a polished case study on authority.

https://openai.com/index/ropes-gray-ai-diligence — direct source or governing technical document relevant to the claim above.
https://www.ropesgray.com/en/insights — direct source or governing technical document relevant to the claim above.
https://www.americanbar.org/groups/professional_responsibility/resources/ethics-opinions/ethics-opinion-512/ — direct source or governing technical document relevant to the claim above.
https://www.americanbar.org/groups/professional_responsibility/publications/ethics_opinions/ — direct source or governing technical document relevant to the claim above.
https://www.nist.gov/itl/ai-risk-management-framework — direct source or governing technical document relevant to the claim above.
https://www.nist.gov/itl/ai-risk-management-framework/ai-rmf-playbook — direct source or governing technical document relevant to the claim above.
https://www.sec.gov/edgar.shtml — direct source or governing technical document relevant to the claim above.
https://www.law.cornell.edu/rules/frcp/rule_26 — direct source or governing technical document relevant to the claim above.
https://www.ftc.gov/business-guidance/blog/2023/02/keep-your-ai-claims-check — direct source or governing technical document relevant to the claim above.
https://platform.openai.com/docs/guides/agents — direct source or governing technical document relevant to the claim above.

The operating test

A serious deployment should be judged after the novelty wears off. Keep the original baseline, record exceptions, review failures, and publish what changed in the workflow. That discipline is more valuable than a promise that the model will remove judgment. The people responsible for the result should be able to explain the source, the decision, the uncertainty, and the next action without hiding behind the interface.