AI for data engineers: what actually works in 2026

What AI does well

Writing and refactoring pipeline code. Cursor and Claude Code handle dbt models, Airflow DAGs, and ingestion boilerplate as well as they handle any backend code, and data engineers who live in terminals adopt the agent workflow fastest. GitHub Copilot is a strong alternative if your team uses JetBrains or Neovim instead of VS Code. Observed gain: a new dbt model with tests and documentation in minutes instead of an hour. Caveat: code that runs is not code that is correct, and nothing in the loop checks the numbers.

SQL drafting and dialect translation. Claude, Copilot, and dbt Copilot draft window functions, pivot logic, and Snowflake-to-BigQuery translations reliably. Caveat: the last 10 percent is exactly the dangerous part: null handling, join grain, and timezone semantics are where the confident drafts go wrong.

Bulk refactors. Renaming a column across forty dbt models, adding a tenant filter everywhere, or migrating a DAG framework is the delegation sweet spot: mechanical, verifiable by compile, and boring. Caveat: review the diff for the one model where the rename collides with an alias.

Documentation and lineage. Generating model descriptions, column docs, and yaml from existing SQL is the most reliable win on this list: pure secretarial load, and staleness is the only real risk.

Where it fails

Silent semantic errors. The defining failure of AI in this profession: wrong SQL that executes. A concrete example: asked to join orders to shipments, a model produced a query that double-counted orders with split shipments. It ran, it passed the not-null tests, and it overstated revenue until a finance reconciliation caught it two weeks later. No error, no warning, plausible numbers. This is why failure transparency scores 2.

Warehouse cost optimization. Asked to make a query cheaper, models confidently rewrite it into something that scans more data. They do not see the query plan, the clustering, or the bill, and they optimize for looking optimized.

Schema recall. Models invent column names that almost exist (customer_id on a table that has account_id) and hallucinate join paths on schemas too large to fit in context. The bigger the warehouse, the worse the guessing.

My take

My take (August 2026, Bernat Sampera)

The SERP for this query is listicles and course upsells, because the honest answer is unglamorous: AI made data engineers faster at writing pipelines and no better at trusting them. The gain is real. Most of my pipeline code now starts as a delegated task, and the compile-and-test loop catches the shallow mistakes. The danger is the deep ones: SQL is the one language where wrong output looks exactly like right output, and every failure that matters in this job is silent. The verdict is falsifiable: take a month of AI-written transformations and reconcile them against a known-good baseline. If your reconciliation failure rate is zero, I am wrong and you can skip the validation step. Mine was not zero. The practical rule I have settled on: delegate the code, never delegate the definition of correct, and budget the time you saved writing SQL into testing it.

What I would build

The unsolved problem in every failure above is context, not code. The model does not know your join grains, your metric definitions, or which table is the source of truth for revenue, so it guesses, and its guesses compile. What I would build is a persistent semantic layer for AI-assisted data work: the schema with its real meanings, the approved join paths, the metric definitions with their edge cases, stored where every coding agent reads them before writing a query and validates against them after. The pieces exist today (dbt semantic models, data contracts, catalog tools), but none of them sit in the loop of the tools that write the SQL. Until they do, the workaround is a curated context file per warehouse, pasted into every session that touches money.