AI Data Automation with
Hybrid LLM & Rule-Based Pipelines

Every business has at least one workflow where the data is too messy for a simple script and too critical for a pure-AI approach. Vendor price files arrive in dozens of formats — different column names, inconsistent units, missing fields. Retail sales data sits in a database but nobody has built the logic to surface which categories are leaving margin on the table. The common thread: the data exists, but it is not operational.

The pattern we keep returning to is hybrid AI data automation — use an LLM where fuzzy interpretation genuinely helps (mapping column headers that humans named inconsistently, generating narrative explanations of analytical results) and deterministic code where correctness is non-negotiable (validating data types against a 120-column schema, calculating markup anomalies from historical baselines). The LLM handles the squishy parts; Python, Pydantic, Zod and SQL handle the rest. A human-approval layer sits in between so operators can trust the pipeline before anything touches a production system.

We applied this pattern to two very different problems for two different clients. Here is how each one worked.

LLM for interpretation, code for validation — never the reverse

The first architectural decision on the vendor onboarding project was where to draw the line between AI and deterministic code. Column mapping is inherently fuzzy — one supplier calls a field “Unit Cost”, another calls it “Price Per Item (excl. tax)”, a third just writes “$”. An LLM handles that ambiguity well. But validating that the mapped data conforms to a 120-column Acumatica schema is a boolean problem with no room for creativity. We learned early that mixing these responsibilities — letting the LLM both map and validate — produces results that are 80% correct and impossible to debug. The hybrid split is not a philosophical preference; it is the only architecture that survives production.

Few-shot mapping beats fine-tuning for ERP schema alignment

We considered fine-tuning a model on the client’s historical vendor files but rejected it for two reasons. First, the schema changes — the Acumatica template gets new columns every quarter, and a fine-tuned model would need retraining each time. Second, few-shot prompting with the full schema in context gives the model exactly the same reference frame as a human operator reading the template definition. The system prompt is engineered against the 120-column schema with worked examples, and the mapping accuracy improves iteratively as we harden against golden datasets. The eval harness (pytest + great-expectations) catches regressions automatically.

Human-in-the-loop is a trust feature, not a fallback

The n8n orchestration pipeline routes every AI-generated mapping through a human-approval gate before data reaches Acumatica. This was not a concession to imperfect AI — it was an explicit design choice. Operators who do not trust the pipeline will not use it, and operators who cannot see what the AI decided will never trust it. The approval layer shows exactly which columns were mapped, which values were transformed, and which rows were flagged. The target is 80% automation on typical cases, with the remaining edge cases routed to a transparent approval queue. The result: onboarding compressed from weeks to hours, with operator confidence intact.

Rule-based analytics first, LLM narrative later — discipline before spend

The retail markup project had a clear temptation: feed sales data into a language model and ask it to generate insights. We deliberately deferred the LLM. The first version runs entirely on rule-based analytics — combined-signal trend logic, anomaly detection against historical baselines, scenario forecasting with confidence intervals. Auto-reports are generated from rule templates that cite concrete numbers, not fabricated explanations. The architecture exposes a narrative-layer hook so a model-generated text swap is a single deployment when eval justifies the cost. The rule-based output is the baseline any future model has to beat — discipline before spend.

Eval harnesses are non-negotiable for production AI

Both projects include measurable eval frameworks. The vendor onboarding pipeline runs against golden datasets with pytest and great-expectations, measuring mapping accuracy across embedder, schema version, and file format. The retail analytics engine validates anomaly detection precision against labelled historical data. Without a measurable retrieval-quality matrix, every “improvement” is a guess. Half of what looks good in demos regresses against the production data. The eval harness is cheap to set up and impossible to operate without once you ship.

Speed-to-prototype as a kill mechanism

The retail markup PoC was delivered in one week. This was not a corner-cutting exercise — it was a deliberate strategy. Most AI ideas die cheap when you can ship a working proof of concept in days and test it against real data. The expensive ones are the ideas that did not get killed early because nobody could test them. The PoC — explainable analytics, interactive forecasting, production-grade data layer — is currently in client validation. The LLM narrative sits parked behind an architectural hook, ready to deploy when the rule-based baseline proves the concept is worth the model cost.

Related use cases: Fleet Driver Management & Compliance Platform • AI Medical Imaging Diagnostic Platform with CNN & Explainable AI