← All Kits · All Guides

How to Check a Chart an AI Built Before You Put Your Name on It

Michael Nocito · Updated August 2026 · Every number on this page was worked before it was published

A generated chart arrives fully formatted. Titles, gridlines, a sensible colour scheme, labels in the right places. All the visual signals that used to mean somebody had finished the analysis are now free, and they arrive before any of the analysis has been checked.

What you do: run seven checks in a fixed order, cheapest first, and stop early when one fails. The first three take about two minutes between them and catch most of what actually goes wrong.

The short version. Check what is not on the chart before you check what is.

Check 1: the row count

How many rows went into the chart, and how many are in the source? Not roughly. Exactly.

SELECT COUNT(*) FROM orders;                 -- 41,208
-- and the same filter the chart used
SELECT COUNT(*) FROM orders WHERE ...;       -- 38,904

If those differ, something was excluded, and the interesting question is what. Dropped rows are almost never random: they are the nulls, the ones with a date the parser could not read, or the category that was spelled differently. Every one of those is a group with something in common, which is exactly what makes their absence change the answer.

Check 2: the filter you did not ask for

Read the code, not the chart. Look for anything that removes rows: a WHERE, a dropna(), a date range, a head(), a LIMIT, a "top 10" that was applied before the total was computed.

The one to watch hardest is dropna. It is frequently added to make the code run, and it silently deletes the rows whose absence you would most want to know about. A chart of average order value that quietly excluded every order with no region is not an average order value.

Check 3: the aggregation

The average-of-averages case is worth spelling out because it looks so reasonable. Region averages of 100, 200 and 300 do not average to 200 unless the three regions have identical row counts. The correct figure is total value over total rows, and the difference is often several percent, which is enough to move a decision.

Check 4: the axis

Does the y axis start at zero, and should it? Bars must; lines usually should not. Charting libraries frequently pick a non-zero base automatically for bar charts, because the default is chosen to fill the plot area rather than to be honest. The rule is in should the axis start at zero, and this is the single most common formatting failure in generated charts.

Check 5: the categories that are not shown

Count the distinct categories in the data and count the bars. A chart with the top ten of forty regions is fine if it says so and if there is an other bar. Without either, a reader takes it as the whole picture.

Same for time: does the x axis have a gap? Months with no rows are usually absent rather than zero, so a line chart joins March straight to May and slopes gently through a month where nothing happened.

Check 6: the labels and the units

1. Are the units stated, and are they what you think? Thousands and units mixed on one axis is common when the source columns differ.
2. Is the currency single? A sum over mixed currencies is a number with no meaning.
3. Do the date labels say what period they cover: month start, month end, or week commencing?
4. Is anything cumulative that is presented as periodic? See up and to the right.

Check 7: the claim in the title

Generated titles overstate, because a confident sentence reads better than a careful one. Two failures to look for: a causal claim over correlational data, and a superlative the chart cannot support. "Marketing spend drives ticket volume" over two rising lines is the error worked in correlation vs causation. "Our best quarter ever" over four quarters of data is the other.

The one-minute version

How to apply this to your own work

1. Pick one group, any group, and recompute its number by hand from the source. One row of agreement is worth more than a page of plausible output.
2. Ask for the row count and the list of filters as part of the request, not afterwards. Stating them changes what gets produced.
3. Keep the code that made the chart next to the chart. A chart whose query cannot be found cannot be checked later, when somebody queries it.
4. Add a footnote with the source, the date and the exclusions. If you cannot write that footnote, you have not verified the chart.
5. Sign it only when you could defend every number without opening anything.

The one habit to keep

Ask what would have to be true for this chart to be wrong, then check that specific thing. It is a faster route to the error than re-reading the code from the top, and it is the habit that separates review from admiration.

Could you rebuild the last generated chart you shared, from the source, without help?