Poor data documentation sinks a lot of applications that attempt to use large language models for automated data analysis. Anyone who has tried to ask an LLM questions about a database with vague column names and missing descriptions has seen the failure mode. The model guesses. The guesses are sometimes clever, often plausible, and occasionally completely wrong.
This problem has quickly become one of the most frequently cited reasons that LLMs "don't work with data."
The diagnosis is not wrong. Poor documentation does break a lot of these systems.
But the conclusion people draw from that diagnosis is often mistaken.
Most teams still think about the problem in a strictly linear way:
First, clean the data. Second, document the data properly. Only then should you consider using AI tools.
This sequence is familiar to anyone who has spent time in analytics or data engineering. It reflects decades of good advice. Analysts have been trained to believe that everything must be orderly before analysis begins. The warehouse must be pristine. The documentation must be thorough. Only then can real work start.
That mindset made sense when the only tools available were human analysts and static software.
It makes much less sense now.
The better mental model is not linear. It is iterative.
And the teams that adopt that model early will move much faster than those still waiting to "clean house" before they start.
The Old Mental Model of Data Work
For a long time, the traditional analytics workflow looked something like this:
First you ingest the data. Then you normalize and clean it. Then you document tables, fields, and definitions. After that, analysts begin asking questions.
This sequence was not just a best practice. It was a necessity.
Human analysts cannot efficiently work with poorly described datasets. If a column named VALUE might mean dollars, thousands of dollars, or something else entirely, analysis becomes a guessing game. A single incorrect assumption can invalidate an entire report.
So organizations built processes around preventing that problem.
Data dictionaries were written. Metadata catalogs were maintained. Governance programs were created to ensure definitions stayed consistent.
None of this was wasted effort. In fact, many organizations still struggle precisely because they skipped these steps.
But the traditional process had a hidden side effect. It trained everyone to believe that documentation must come before analysis.
That assumption is now quietly breaking.
The Reality of Most Data Environments
In practice, most organizations operate in an environment that looks nothing like the idealized workflow.
The data exists. The documentation does not.
Columns have names like VAL, AMT, or CODE_3. Tables are inherited from systems built years earlier. Internal notes exist somewhere in Slack or Teams threads, Jira tickets, or forgotten internal directory folders.
Sometimes the only reliable documentation is tribal knowledge held by one analyst who joined the company eight years ago.
This situation is not rare. It is the default.
And it is exactly the environment where many teams assume LLMs will fail.
To be fair, if you simply connect a language model to an undocumented warehouse and start asking questions, it often will fail. The model has no context. It cannot reliably infer semantics from ambiguous field names.
The interesting development is that LLMs do not only answer questions about data.
They can also help construct the documentation itself.
Once that becomes clear, the linear workflow starts to look unnecessary.
Documentation Can Be Generated
The shift happens when the model is allowed to inspect the environment rather than simply query it.
Instead of asking: "What are the top holdings in this dataset?"
You ask the model to investigate the structure first.
It can examine table schemas. It can sample rows. It can search related documentation. It can cross reference external sources.
From there it can begin generating something most datasets lack: a coherent data dictionary.
This process looks closer to investigative work than traditional querying.
A model might start by inspecting tables and noticing that a dataset includes fields such as ACCESSION_NUMBER, CIK, and PERIOD_OF_REPORT. With a small amount of external context, it can determine that these correspond to SEC Form 13F filings, which institutional investment managers submit quarterly to disclose equity holdings. [1]
From there, column meanings become clearer. PORTFOLIO_VALUE represents the total market value of reported securities. HOLDING_COUNT indicates the number of individual positions. Fields like PUT_CALL describe options positions.
The model can assemble these pieces into usable documentation even if the database itself originally contained almost none.
And once that documentation exists, the model's ability to answer questions improves dramatically.
I also don't want to overstate this. Even though documentation can be generated automatically, it still needs to be reviewed by knowledgeable humans before putting in prod. Review is far less of a drudgery than putting it together column by column in the first place.
A Practical Example
Recently I ran an experiment using VerbaGPT. I had it load six quarters of SEC 13F filing data (over 20 million rows) in a Snowflake dataset.
The dataset itself was structured but minimally documented. It included tables like:
- FILINGS
- HOLDINGS
- SECURITIES
Column names were present, but descriptions were sparse or missing. Exactly the sort of dataset many teams assume is unusable for LLM-driven analysis.
Instead of immediately querying the data, the model was asked to perform a different task:
"Review the database and update the data notes so that future queries can benefit from clearer descriptions of variables, tables, and relationships."
The system began by inspecting the schema and sampling the data.
It looked at fields like ACCESSION_NUMBER, CIK, FILING_DATE, PERIOD_OF_REPORT, and PORTFOLIO_VALUE.
It examined categorical values to infer meaning. For example, SUBMISSION_TYPE included entries such as 13F-HR, 13F-NT, and amendment variants. That matched known filing types from the SEC reporting system. [2]
It also analyzed the holdings table, which contained more than 16 million rows of individual securities positions.
From this investigation, the system generated a complete set of data notes:
- A high level description of the dataset
- Table level documentation
- Field level definitions
- Valid categorical values
- Known unit changes in historical fields
- Example analytical questions
One detail illustrates why this process matters. The VALUE column in 13F data changed units in 2023. Historically the value was reported in thousands of dollars. Newer filings report values in actual dollars. Without documenting that distinction, analyses that span both time periods will quietly produce incorrect results.
The model detected and documented this nuance automatically.
Once that metadata existed, subsequent questions about the dataset produced far more reliable answers.
The interesting part is not that documentation improved the analysis. Everyone already knows that.
The interesting part is that the documentation was generated by the same AI system that needed it.
The Feedback Loop
What emerges from this approach is a feedback loop rather than a sequence.
Instead of: Data → Documentation → Analysis
The process becomes something closer to: Data → AI exploration → Documentation → Better AI analysis → Improved documentation
Each pass improves the environment.
The first round of metadata might be rough. Later passes refine it. Analysts can edit or approve definitions. Over time the system converges toward a well documented dataset.
Importantly, the work begins immediately.
No one has to pause progress until the warehouse reaches some mythical "fully documented" state.
Why This Matters
There is a subtle but important shift here.
Historically, documentation was a prerequisite for analysis. That made it easy to postpone indefinitely. Documentation is valuable but rarely urgent. When deadlines approach, it is the first thing teams skip.
Now documentation can be produced as a side effect of exploration.
Instead of asking analysts to pause their work and write metadata, the system generates it while trying to understand the data itself.
This reduces one of the largest bottlenecks in analytics environments.
It also changes who can work with data.
A well instrumented AI system can help a non specialist understand datasets that previously required months of institutional knowledge.
That does not eliminate the need for experts. In fact, domain expertise becomes even more important for validating and refining the automatically generated definitions.
But it dramatically lowers the barrier to getting started.
The Analysts Who Adapt First
None of this means the old principles are wrong.
Clean data still matters. Documentation still matters. Governance still matters.
The mistake is assuming those things must be completed before AI tools become useful.
The analysts who succeed in the next phase of the field will treat these processes as collaborative with AI rather than preparatory for it.
They will let the tools explore messy datasets. They will let the models propose definitions. They will refine those definitions over time.
In other words, they will stop waiting for the house to be perfectly clean before turning on the lights.
And in many cases, the lights will help them clean the house faster.
Data Notes in VerbaGPT before and after the LLM went to work:
Before - we have an empty schema, which is still useful for the LLM to write better SQL code, but context is missing:
After the LLM updated the notes, we have rich context that will benefit subsequent queries.
Footnotes
- U.S. Securities and Exchange Commission, "Form 13F Data Sets." https://www.sec.gov/data-research/sec-markets-data/form-13f-data-sets
- U.S. Securities and Exchange Commission, Form 13F Data Sets README. https://www.sec.gov/files/form_13f_readme.pdf