Not every problem wants a transformer. Ranking, fraud scores, churn, forecasting — gradient boosting and boring tabular models still crush latency and cost on structured data. LLMs shine at language, tools, and messy workflows where the input is prose and the output is action.
The mistake we see constantly: an “AI roadmap” that is only chat UIs, while the highest-ROI ML — the scores that already drive revenue — sits in a notebook nobody deploys. The opposite mistake: a data science team shipping batch scores with no product surface, while the business asks for a copilot because that is what they read about.
The blend we like
- Classical model for the score — fast, cheap, explainable with SHAP or feature importances when regulators ask.
- LLM agent for explanation, triage, and next actions — the layer humans actually interact with.
- Shared feature store or event stream so both stay honest. The agent should not invent a churn probability the model never computed.
This is Embed territory when the agent wraps an existing score. It is Forge territory when you are building the score and the surface together from scratch.
Week-by-week shipping cadence
We are not talking research quarters. A typical Forge + Embed blend:
- Week 1: Define the decision the product makes. Pull historical labels. Baseline with logistic regression or XGBoost — something you can train in an afternoon.
- Week 2: Ship the score behind an API. Add monitoring: distribution drift, null rates, latency. No UI yet.
- Week 3: Wrap with an LLM layer — explain the score, suggest next action, route to human when confidence is low.
- Week 4: Evals, audit trail, kill switch. The boring production stuff that separates a demo from something you leave running.
If week one does not produce a baseline metric, you have a data problem, not a model problem. Fix that before buying GPU hours or agent frameworks.
When to skip classical ML
Skip the tabular stack when:
- The input is unstructured text or images and the output is language — classification maybe, but not scoring on fifty engineered features.
- You have no labelled history and no proxy metric. “We will collect labels later” means later never comes.
- The decision is low volume and high stakes — human review with LLM assist beats an auto-score nobody trusts.
Everything else — ranking, fraud, lead scoring, demand forecasting — classical ML first. Add the LLM where it earns its token cost.
What fails in practice
Two failure modes, equally common. First: the LLM hallucinates a reason for a score it never received — fix with strict tool wiring, never free-text numbers. Second: the classical model ships but nobody maintains it — fix with the same CI and monitoring you would demand for any API.
If your roadmap is only notebooks or only chat, you are leaving money on the table. We do both — in weeks — because the product usually needs both.