An AI agent read a due diligence framework. The framework referenced an appendix, so it read that too. The appendix contained a precedents section with worked examples from prior engagements, so it read those as well. It identified the most relevant case. It applied the guidance. It produced a classification.
Four times in a row, on the same deal, that classification was wrong.
The model was not careless. It cited the correct precedent, matched the deal to the most similar historical case in the library, and arrived at a conclusion a junior analyst would recognize as well-reasoned. The reasoning was internally consistent. The conclusion was wrong. Same error, same stated rationale, across four independent runs. If this were a live system, no one would flag a problem. The decisions would look consistent, well-referenced, and policy-compliant.
When the same agent worked through the same deals guided by a decision tree, it got every answer right.
The previous entries in this series showed what happens when a single flat document contains misleading worked examples (claim adjudication) or requires inferences the document does not supply (clinical triage). This experiment tests something different: multi-document conflict. The policy and its appendix describe the same framework but give different benchmarks for the same sector. An agent handed both documents must decide which one governs. The tree made that decision before the agent arrived.
The Domain
Financial due diligence is a screening exercise. Before a firm submits an indicative bid on an acquisition target, an analyst works through a sequence of checks: validate the EBITDA margin against sector benchmarks, normalize for non-recurring items, confirm equity is positive, calculate leverage, and classify the deal as DEAL-BREAKER, PRICE CHIP, or NO MATERIAL ISSUE.
The rules live in a framework document. But the framework explicitly references an appendix that contains updated benchmarks. The appendix, in turn, contains worked examples drawn from prior engagements that are designed to guide analyst judgment. There is a correct path through these documents. The question is whether an agent reading them cold can find it.
General Note: this isn't representative of how such workflows work in the real world, nor is this article prescriptive on how such a process should work. I tried to construct a scenario that is somewhat interesting and close to real workflows. The goal of this example is expose failure modes than LLMs have when applied in a typical way, and how to address them.
The Setup
I built three documents and four test deals.
The framework document (~165 lines). The primary policy. It describes the screening methodology: EBITDA benchmarking by sector, normalization of non-recurring items, leverage calculations with a capital-intensive sector adjustment, and classification thresholds. The language is deliberately hedged. Analysts "should consider whether" items need normalization. A capital-intensive adjustment "may be appropriate." The document frames itself as "a guide, not a decision tree." Judgment remains with the engagement team.
The sector appendix (~98 lines). Referenced explicitly by the framework. Contains updated EBITDA margin benchmarks for technology and professional services sectors. Also contains normalization precedents from prior engagements, including a detailed Apex Cloud case where a disclosed licensing fee was not normalized because management framed it as part of an ongoing IP monetization strategy. Also contains a Summit Fabrication reference case where a heavy manufacturer's leverage was classified as a deal-breaker even after normalization. These precedents are designed to be misleading for the test deals.
The deal memos (~43 lines). Narrative descriptions of four target companies, each containing the financial data needed to screen them. The numbers are embedded in prose, not a structured table. No explicit labels for one-time items. No sector classification hints. The agent has to extract figures from natural language.
Four test deals:
D003 is the control. Negative equity is unambiguous. Both approaches should catch it every time.
The other three are traps, each testing a different failure mode in multi-document workflows.
The Three Traps
Trap 1: The Precedent Override (D001)
Meridian Software reported $31.1M in EBITDA on $42.0M in revenue, a 74.0% margin. SaaS companies typically run 10-35%. The margin is inflated by a $21.6M licensing fee from a strategic IP arrangement with Axiom Technologies. Management describes IP licensing as "a complementary revenue channel." The correct treatment: strip the fee as non-recurring, producing adjusted EBITDA of $9.5M (22.6% margin) and net leverage of 3.68x, a price chip.
The trap lives in the appendix. Precedent A2-1 describes Apex Cloud Ltd., a SaaS company with a $7.8M licensing arrangement. Management called licensing "a complementary revenue channel" there too. The deal team at Apex Cloud decided not to normalize. The appendix frames this as a precedent to guide analyst judgment. An agent reading the appendix before screening Meridian encounters a near-identical fact pattern with a conclusion that explicitly favors skipping normalization.
If the agent follows the precedent: no normalization, leverage = 1.13x, NO MATERIAL ISSUE. Wrong.
Trap 2: The Distributed Formula (D002)
Cascade Industrial reported $18.0M in EBITDA on $180.0M in revenue, a 10.0% margin. The margin is depressed by $14.0M in costs from an operational efficiency program that management describes as substantially completed. The correct treatment: add back the $14.0M as non-recurring (adjusted EBITDA = $32.0M, 17.8% margin, within the 8-20% industrial range). Cascade is a heavy manufacturer with $52.0M in capitalized lease obligations. Those leases represent operational infrastructure, not financial borrowing. The correct treatment: subtract them from net debt before calculating leverage.
The document describes the normalization rule (strip non-recurring items) in Section 2.3. It describes the lease adjustment (subtract capitalized leases for capital-intensive sectors) in Section 3.3. The agent must apply both:
- Normalize only: $152M / $32M = 4.75x → DEAL-BREAKER (wrong)
- Lease-adjust only: $100M / $18M = 5.56x → DEAL-BREAKER (wrong)
- Neither: $152M / $18M = 8.44x → DEAL-BREAKER (wrong)
- Both: $100M / $32M = 3.13x → PRICE CHIP (correct)
Three wrong paths, one right one. And the appendix makes it worse. The Summit Fabrication reference case (L1) shows a heavy manufacturer post-normalization at 5.1x being classified as a deal-breaker. It does not mention the lease adjustment. An agent that normalizes D002 but pattern-matches to Summit Fabrication has every reason to stop there. The document does not flag that the lease adjustment was available but not applied in that case.
Trap 3: The Cross-Document Benchmark Conflict (D004)
Pinnacle Advisory Group is an M&A advisory firm. $55.0M in revenue, $15.7M in EBITDA, 28.5% margin. The year included an $8.2M transaction completion fee from a landmark cross-border mandate. Management describes such mandates as infrequent, "typically arising once every several years." The core business is retainer-based advisory.
The framework says Business Services / Consulting benchmark: 12-22%. At 28.5%, Pinnacle's margin is anomalous by 6.5 percentage points. Normalization required: strip the $8.2M fee, adjusted EBITDA $7.5M, leverage 3.87x, PRICE CHIP.
But the appendix says Business Advisory / Specialized Consulting benchmark: 15-35%. At 28.5%, Pinnacle's margin is in range. No normalization required, leverage 1.85x, NO MATERIAL ISSUE.
Both documents are authoritative, depending on which benchmark applies. The framework says the appendix takes precedence for the sectors identified. The appendix says its advisory benchmark applies to firms with "transaction-based advisory fees." Pinnacle has a transaction-based fee. Does the appendix win? Only if the "infrequent" characterization is irrelevant, the retainer-based core is irrelevant, and the analyst's judgment about which benchmark was designed for which business model is irrelevant.
This is not a trap where one document is right and the other is wrong. Both contain a legitimate path. The correct path requires applying the framework's benchmark because the appendix's benchmark was designed for firms where transaction fees are the core recurring revenue model, not firms where they arise once every several years. Neither document says this explicitly. The agent has to infer it.
Two Approaches
Flat document. The agent reads all three documents in one load. It extracts the deal data from the narrative memos, resolves cross-document conflicts, tracks distributed formulas across sections, and decides which precedents to follow. Same model across all runs: DeepSeek V4 Flash.
Decision tree. The agent reads only the deal memos: 43 lines of narrative prose. All policy content (benchmarks with resolution rules, normalization instructions, formulas, and thresholds) lives inside the tree, delivered one node at a time through mcpforx. The tree tells the agent exactly which benchmark to use under exactly which conditions. It provides the normalization formula and the lease-adjustment formula explicitly. The agent does not navigate document conflicts because the tree has already resolved them. Four independent runs.
Here is what the logic looks like:
A fairness note about the tree: the resolve-benchmark node explicitly states that the appendix's 15-35% advisory range applies only when transaction fees are core and recurring. The framework document says something similar but buries it in the "typical" ranges prose. The normalize node also includes a rule the flat document implies but does not state: "Do not analogize to precedent cases to justify skipping normalization. Each deal stands on its own facts." The framework's Section 2.3 says "analysts should review Appendix A precedents before concluding that a disclosed item requires adjustment." The tree says the opposite. Both are the same policy applied with different emphasis. The flat-document agent is free to interpret the hedging; the tree resolves the ambiguity before the agent sees it.
Both conditions used the same model (DeepSeek V4 Flash) and the same deal data. The only difference was the delivery mechanism for the policy.
Results
Four flat-document runs, four tree runs. Same model, same deals, same documents.
Flat Document
Overall accuracy: 37.5%. Zero perfect runs.
Decision Tree
Overall accuracy: 100%. Four perfect runs.
By Trap
What Each Trap Reveals
D001: The Precedent Problem
Every flat run quoted Precedent A2-1 (Apex Cloud) and skipped normalization on Meridian's $21.6M licensing fee. Four identical errors, four well-referenced decisions, zero correct classifications. The reasoning was sound. The documents instruct analysts to review appendix precedents. The precedent describes a nearly identical fact pattern. The precedent explicitly recommends not normalizing. The agent did exactly what the documents told it to do.
The tree broke the pattern by changing the instruction. Instead of "review precedents before concluding," the tree said "do not analogize to precedent cases." The analyst was told to evaluate each deal on its own facts. Same principle, different delivery. The flat document's hedging created room for the precedent to override the correct treatment. The tree removed the room.
D002: The Formula Problem
Every flat run correctly normalized Cascade's $14.0M in program costs. Every flat run also used the wrong leverage calculation. The formula was described in prose across two sections: strip non-recurring items (Section 2.3), then for capital-intensive sectors consider subtracting leases from net debt (Section 3.3). Every run produced the same net debt figure of $152M. The verbal description of the formula was ambiguous about whether the $52M in lease obligations should be subtracted from net debt or excluded from gross debt to begin with, and the Summit Fabrication reference case reinforced the error by showing a heavy manufacturer reaching its deal-breaker conclusion without a lease adjustment mentioned.
The tree node for lease-adjusted leverage said:
net_debt_ex_leases_m = gross_debt_m - cash_m - capitalized_leases_m
No hedging. No "consider whether." An equation with three named terms. Every tree run computed the same correct figure.
This is not about model capability. It is about what happens when a formula is distributed through prose across two document sections. An agent tracking the calculation must hold the normalization rule from Section 2 and the lease adjustment from Section 3 simultaneously. The tree presents the formula at the moment it is needed, with exact variables referenced from prior steps. That is the difference between 4.75x (wrong) and 3.13x (correct).
D004: The Benchmark Problem
Two flat runs used the wrong benchmark and classified D004 as NO MATERIAL ISSUE. Two flat runs used the right benchmark and classified correctly as PRICE CHIP. This is what cross-document conflict does: same model, same documents, same deal, two different answers.
The appendix says advisory benchmark is 15-35%. The framework says 12-22%. Both are authoritative. The resolution hinges on whether Pinnacle's transaction fee model is "core/recurring" or "infrequent." The documents do not resolve this. Some runs treated Pinnacle as specialized advisory (appendix applies, 28.5% in range, no normalization). Others noted the "infrequent" language and applied the framework benchmark (28.5% anomalous, normalize, correct).
The tree's benchmark node resolved this explicitly: "Use 15-35% ONLY if transaction_fees_core_recurring = true. If the memo describes fees as infrequent or once every several years, use 12-22%." Four runs, one benchmark, identical treatment. The tree removed the judgment call and imposed a rule. The rule was derivable from the documents. The tree just made it explicit.
Mathematical Gating
The previous experiments showed the tree winning by controlling what information the agent sees (clinical triage) and by preventing misleading examples from entering scope (claim adjudication). This experiment reveals a third mechanism: mathematical gating.
The tree's benchmark node requires the agent to compute margin_deviation_pp, the exact number of percentage points the reported margin exceeds the benchmark. It routes on the value: if deviation > 5.0, normalize. If deviation ≤ 5.0, proceed. The tree does not ask whether the agent thinks the margin looks anomalous. It asks for a number and branches on it.
This matters because flat-document agents sometimes estimate where they should calculate. In run 2 of the flat trials, the agent estimated D004's normalized EBITDA at $8.3M instead of $7.5M, producing leverage of 3.67x instead of 3.87x. The estimate happened to land in the same classification band by accident. If the fee had been slightly smaller, the estimate-based leverage would have slipped into the 3.0x-3.5x range and the classification would have been ambiguous. The tree gates on what the agent computed, not what it estimated. margin_deviation_pp: 6.5 is not a judgment. It is a number the tree reads and routes on.
The Information Load Problem
The flat-document agent read approximately 300 lines across three files. The tree agent read 43 lines of deal memos. All policy content lived in the tree.
This is the point. An organization with a due diligence framework does not care whether the agent reads a verbose policy document. It cares whether the agent reaches the correct classification. The tree replaces reading with receiving: the agent is given the instruction relevant to the current step, with the exact formula and expected output fields. The comprehension task shrinks from "understand 300 lines of policy prose and resolve cross-references" to "extract numbers from a 43-line memo and compute the formula provided."
If your workflow involves an AI agent reading a long document to make a decision, you are asking the model to do two things at once: extract the rules and apply them. The tree separates those tasks. A human extracts the rules once. Every subsequent run inherits that extraction. The model only applies.
What This Means for Multi-Document Workflows
Real organizations do not maintain a single policy document. They maintain a framework, an appendix, a quarterly update memo, a precedent library, a partner guidance note, and an FAQ that supersedes parts of all of the above. These documents evolve at different cadences. They contain conflicts, some intentional and some not. An agent handed all of them must navigate those conflicts.
Better models will not resolve document conflicts. A model with stronger reasoning benchmarks will still face two authoritative benchmark ranges and have to choose. It might choose more carefully. It might even choose correctly more often. But it will continue to face moments where the correct choice requires knowing which document was written when, for what purpose, and with which assumptions about the reader. Documents do not encode that metadata. Trees can.
This experiment produces the same result as the first two, through a different mechanism. A flat document gave the agent 37.5% accuracy. A decision tree gave the same model 100%. Both had access to the same policy rules. The difference was not the model. It was what stood between the model and the decision.
tldr: Across eight runs on four deals, the same model was right 37.5% of the time when asked to navigate three flat documents and 100% of the time when guided through a decision tree. The correct answers were in the documents. The model was capable of finding them with the right structure. The limiting factor was not the model. It was the document format.
MCPforX is a platform for building and deploying structured expertise as MCP-accessible decision trees. The deal data is fictitious. Full run traces and the experiment files are available on request.