MAPE or Absolute Calorie Error? Choosing the Right Accuracy Metric for a Nutrition-App Benchmark

Every accuracy figure in this category is a summary statistic, and summary statistics are choices. Before BiteBench publishes a number we have to decide what “off by this much” means, and that decision changes the leaderboard. This note documents the decision. It ranks nothing and recommends nothing; it is the measurement layer that sits underneath our testing methodology.

The two candidate metrics are mean absolute percentage error (MAPE) and absolute error in kilocalories (MAE). They look like cosmetic variants of each other. They are not. On the same raw residuals, they can produce different orderings of the same set of tools, and a reader who does not know which one was used cannot interpret the result.

The two metrics, stated plainly

For each test meal i, let r_i be the weighed reference energy value and e_i the app’s estimate.

Absolute error (MAE) averages the raw magnitude of the miss, in the original unit:

MAE = (1/n) × Σ |e_i − r_i| → reported in kcal/meal

Mean absolute percentage error (MAPE) normalises each miss by the size of the meal before averaging:

MAPE = (100/n) × Σ |e_i − r_i| / r_i → reported in %

The two differ in exactly one operation — the division by r_i — and every practical consequence follows from it.

Why MAPE is the right default for cross-app comparison

A benchmark meal set is heterogeneous by design. Ours spans single-ingredient whole foods in the low hundreds of kilocalories through composite home-cooked plates well over a thousand. Under MAE, the meals at the top of that range dominate the average: a tool that is excellent on breakfasts and poor on dinners scores worse than a tool with the reverse profile, purely because dinners are bigger. The metric ends up describing the meal set as much as the tool.

Dividing by the reference removes that. Three properties follow, and all three matter for a benchmark:

Scale invariance. A 10% miss is a 10% miss whether the plate is 300 kcal or 1,300 kcal. Tools become comparable across meals of different sizes.

Composition robustness. If two labs weight their meal sets differently — more restaurant plates here, more whole foods there — MAPE is less sensitive to that difference than MAE, because the size of the meals is already divided out. It does not make the two labs’ numbers interchangeable, but it narrows the gap that composition alone can open.

Aggregation across strata. Because MAPE is unitless, a per-cuisine or per-complexity breakdown can be read down the column and compared row to row. A stratified MAE table cannot be read that way without also carrying the mean meal size of each stratum.

That is the case for MAPE as the primary metric. It is a strong case, and it is why the dietary-assessment literature has largely settled on it. It is not a case for reporting MAPE alone.

The four ways MAPE lies

1. Small denominators. MAPE is unstable when the reference value is small and undefined when it is zero. A 20 kcal miss on a 60 kcal cup of black coffee with a splash of milk is a 33% error; the same 20 kcal miss on a 900 kcal plate is 2.2%. If a meal set contains a meaningful tail of low-energy items, MAPE will be driven by that tail, and a tool’s headline number becomes a statement about how it handles snacks. The fix is a pre-specified inclusion floor on reference energy, applied identically to every tool, and disclosed. BiteBench excludes reference items below 50 kcal from the primary MAPE and reports them as a separate low-energy subset.

2. Asymmetry. Percentage error is bounded below and unbounded above. An estimate of zero against a 500 kcal reference is a 100% error, and that is the worst an underestimate can score. An estimate of 2,000 kcal against the same reference is a 300% error. So MAPE structurally penalises overestimation harder than underestimation of equal absolute size. That is a real problem in this category, because self-report and estimation tools tend to under-count — the direction MAPE is most forgiving of. A tool with a systematic downward bias can post an attractive MAPE.

3. Non-additivity. Per-meal MAPE does not aggregate to daily MAPE. Four meals each off by 6% in the same direction give a daily error near 6%; four meals each off by 6% in alternating directions can give a daily error near zero. Because MAPE discards the sign, it cannot distinguish those two cases — and for a user tracking a daily calorie budget, they are completely different products. Daily-total error has to be computed separately from the signed residuals; it cannot be inferred from the per-meal figure.

4. Sign blindness. The absolute value in the numerator is doing real work, and it destroys information. Two tools with identical MAPE can have opposite bias profiles: one scattering symmetrically around the truth, one consistently low. The first is noisy but unbiased; the second is precise but wrong, and its error will not cancel out over a week. MAPE alone cannot tell them apart. Mean signed percentage error, reported alongside, can.

When absolute kilocalorie error is the better answer

MAE comes back into its own the moment the question shifts from “which tool is better” to “what does this cost me.”

Clinical and self-management thresholds in this domain are absolute, not proportional. A carbohydrate-counting window relevant to insulin dosing is expressed in grams. An energy-balance target for supervised weight loss is expressed in kilocalories per day. A reader who wants to know whether a tool is good enough for their purpose needs the answer in the unit their purpose is denominated in — and a percentage cannot be converted into that unit without knowing the meal size it was computed over.

The Initiative’s own methodology work reaches the same conclusion from the statistical side, recommending paired reporting with a pre-specified primary metric tied to intended use (Dietary Assessment Initiative, MAPE versus absolute kilocalorie error, 2025). We follow that recommendation: MAPE is our primary metric because our primary purpose is comparison, and per-meal and per-day kilocalorie error are reported alongside it because our readers’ purposes are absolute.

Report a distribution, not a point

A single mean is a poor description of a skewed distribution, and per-meal error in this category is skewed. Ours is right-tailed in every cycle: most meals land close, a small number land badly, and the mean sits above the median. Four numbers describe that far better than one:

• MAPE — the headline mean, for comparability with the published literature.
• Median absolute percentage error (MdAPE) — the typical meal, insensitive to the tail.
• Share of meals within ±10% — the practical reliability question, and the one readers actually ask.
• Mean signed percentage error — the bias term MAPE discards.

Bland-Altman limits of agreement belong in the same family and are the standard graphical treatment for method-comparison work in this field (Bland & Altman, 1986; see also the Initiative’s reporting conventions paper on the errors that recur in its dietary application). A benchmark that publishes a mean without an interval, or an interval without a bias term, is under-reporting.

Symmetric MAPE (sMAPE) is sometimes proposed as a repair for the asymmetry problem. It helps with the bound, but it introduces its own interpretive difficulty — the denominator changes meaning — and it is rare enough in this literature that using it as a headline would make our figures incomparable with everyone else’s. We compute it as a robustness check and do not lead with it.

None of it counts without the reference

Every metric above is a function of the residuals, and the residuals are only as good as the ground truth. This is where most accuracy claims in the consumer category actually fail — not at the statistics, but one step earlier.

A weighed-food reference protocol has to specify, before data collection:

1. Scale calibration. Resolution, calibration against certified reference weights, and a drift tolerance checked within each session.
2. Ingredient decomposition. How mixed dishes are broken into weighable components, and how undeclared components (oil absorbed in a pan, dressing already on a salad) are handled.
3. Cooked-weight reconciliation. Cooking losses recorded, not assumed, for every component above a stated mass threshold.
4. A versioned database lookup. A named nutrient database with a pinned version, a pre-specified priority order across its sub-databases, and a documented fallback for unresolved items. We use USDA FoodData Central; the version matters because entries are revised between releases.
5. Inter-rater reliability. The ingredient-identification step is a judgement call, so it needs an agreement statistic across raters, reported.
6. Blinding and logger training. Loggers who know which tool is expected to win will produce it. Equal training time per tool, and no visibility into interim results.
7. Pre-specified exclusion rules. What happens when a tool refuses an entry, times out, or returns an obviously nonsensical value. Excluding a tool’s failures inflates its score; the rule has to be fixed in advance and applied identically.
8. Intervals, not points. Confidence intervals on every published figure, and a stated minimum subgroup size below which no subgroup figure is published.

The Initiative’s protocol paper covers items 1 through 5 in more operational detail than we do here and is the document we benchmarked our own procedure against (A protocol for weighed-food reference meal construction, 2025).

The mixed-dish problem sits inside every one of these numbers

One caveat deserves separating out, because it constrains what any metric can tell you. Portion estimation for mixed dishes — stews, curries, composite bowls, anything whose components are not separable on inspection — is the dominant error source in this category, and mixed-dish error typically runs well above single-item error within the same study. Shared plates are worse again: when two people eat from one dish, the reference itself becomes ambiguous, because the quantity attributable to each person is not observable from the plate.

The consequence for metric choice is direct. A whole-set MAPE computed over a meal set light on mixed dishes will be optimistic relative to real-world use, and no amount of statistical care downstream repairs a reference set that does not look like how people eat. This is why we report mixed-dish and shared-plate strata separately rather than folding them into the headline, and why we treat a benchmark that does not disclose its composition as uninterpretable rather than merely incomplete. The field has been explicit that this problem is unsolved (Mixed-dish portion estimation, 2025).

Why this note exists

Because most accuracy claims in this category are a single unqualified percentage, and a single unqualified percentage is not a measurement. It is missing the metric definition, the reference standard, the meal-set composition, the exclusion rules, the interval, and the bias term — six things that each independently determine the number. Our inclusion criteria for what counts as an evaluable claim exist for that reason, and our editorial standards commit us to publishing the same six things about our own figures.

The short version, for anyone building or reading one of these comparisons: pre-specify the primary metric before collecting data, publish MAPE and kilocalorie error together, report the median and the interval and the signed bias alongside the mean, disclose the meal-set composition, and state the exclusion rules. A benchmark that does all six can be argued with productively. One that does none of them cannot be argued with at all, which is a different thing from being right.

Citations

1. Hyndman, R.J. & Koehler, A.B. (2006). “Another look at measures of forecast accuracy.” International Journal of Forecasting, 22(4), 679–688.
2. Makridakis, S. (1993). “Accuracy measures: theoretical and practical concerns.” International Journal of Forecasting, 9(4), 527–529.
3. Willmott, C.J. & Matsuura, K. (2005). “Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance.” Climate Research, 30, 79–82.
4. Bland, J.M. & Altman, D.G. (1986). “Statistical methods for assessing agreement between two methods of clinical measurement.” The Lancet, 327(8476), 307–310.
5. Schoeller, D.A. (1990). “How accurate is self-reported dietary energy intake?” Nutrition Reviews, 48(10), 373–379.
6. Subar, A.F., Freedman, L.S., Tooze, J.A., et al. (2015). “Addressing current criticism regarding the value of self-report dietary data.” The Journal of Nutrition, 145(12), 2639–2645.
7. Mean absolute percentage error versus absolute kilocalorie error in dietary assessment validation. Dietary Assessment Initiative, 2025.
8. A protocol for weighed-food reference meal construction. Dietary Assessment Initiative, 2025.
9. Mixed-dish portion estimation: the unsolved problem at the centre of consumer dietary assessment. Dietary Assessment Initiative, 2025.
10. Bland-Altman analysis for dietary assessment validation. Dietary Assessment Initiative, 2024.
11. USDA FoodData Central. National Agricultural Library.

Filed by the BiteBench data desk, August 6, 2026. This note describes measurement practice and will be revised when our reporting template changes.