Validation

Validation is a critical concept when determining whether a dietary assessment instrument is suitable for a particular research question. This concept involves several aspects: evaluating the [glossary term:] validity of an instrument, understanding characteristics of those who [glossary term:] misreport on an instrument, and considering what type of [glossary term:] measurement error is present in the instrument.

Evaluating Validity

The validity of the food records has been evaluated in several ways (see Key Concepts About Validation). The strongest class of validity studies is based on objective [glossary term:] recovery biomarkers.

  • Salient features of studies with recovery biomarkers:
    • Recovery biomarkers are ideal for validation because the intake of the dietary component is reflected by the [glossary term:] biomarker in a relatively constant and known manner. Recovery biomarkers thus provide unbiased estimates of [glossary term:] true intake.
    • Known recovery biomarkers are [glossary term:] doubly labeled water (DLW) for energy intake, urinary nitrogen for protein intake, urinary potassium for potassium intake, and urinary sodium for sodium intake.
    • Studies using recovery biomarkers generally have been on small, highly selective groups, because of the expense of carrying out such studies. No data are available for children.
  • Results of food record evaluation studies using recovery biomarkers:
    • Food records underestimate true intakes of energy and protein; underestimates range widely from 4% to 37%, depending on the particular study sample [5-6].
    • In the largest published study that included food records (the Nutrition and Physical Activity Assessment Study, which included 450 postmenopausal women from the Women’s Health Initiative), food records underestimated true energy intake by about 20% and protein intake by about 4%, and overestimated protein density by about 17% [14].
    • For sodium, there is a tendency for underreporting in the range of 10% to 20% [7-8].
    • For potassium, underreporting is uncommon, with most studies indicating over-reporting in the range of 12% to 20% [7-8].
    • Because of a lack of [glossary term:] recovery biomarkers for nutrients other than energy, protein, potassium and sodium, little is known about [glossary term:] misreporting on other dietary components.
  • Other notes about recovery biomarkers:
    • The use of DLW to reflect energy intake is based on the assumption that weight status does not change during the measurement period. Because of [glossary term:] reactivity, study participants may eat less than normal during the measurement period (Learn More about Reactivity). Differences between reported intake on the food record and the DLW measure of energy intake can thus be from both undereating and [glossary term:] underreporting [9]. Both phenomena influence the ability of food records to reflect usual intakes, and undereating may invalidate the assumption about weight stability.

A second class of validity studies relies on independent and unobtrusive observation of the eating behaviors being recorded in the food record (Learn More about Observation and Feeding Studies).

  • Salient features and results of observation studies:
    • In an observational study, one or more trained staff unobtrusively observe individuals during a meal while noting the foods and portions consumed. The observer may have access to a planned menu, weighed portions given to participants, and/or plate waste.
    • Studies generally have been on small, highly selective groups (e.g., nursing home residents, children in school meal settings), and few in number, principally because of the dearth of appropriate study settings. In one study of 266 adult free-living volunteers who completed a 7-day food record, researchers found that compared to their energy needs to maintain their weight, energy intake was underreported an average of 18% [10].
    • Recently developed technology-assisted food record methods are being evaluated using observation. Plate waste observation studies may be used to validate the analysis of image-based food records or to evaluate the reporting of those recording their diet [11-12].

The third class of studies, which includes most of the validation studies, examines food record performance relative to other self-report instruments, such as the 24HR.

  • Salient features and results of comparative validation studies:
    • Comparative validation studies administer two or more self-report dietary instruments to the same population, and often for the same or overlapping time periods.
    • Comparative validation is imperfect, as no self-report instrument represents true intake. Although individual comparative validity studies may be useful, for example, to learn whether two different instruments produce comparable results, no overall judgment about 24HR validity can be made from this type of study.
    • Another weakness of relative validation is that errors in the two instruments are likely to be [glossary term:] correlated, which typically results in an overstatement of their agreement.

Food records can be presented in many different ways, but research comparing different types of food records in the same respondents has been limited. The little evidence available indicates no clear performance difference between weighed and estimated written records [13] . In addition, little evaluative information is available comparing technology-assisted food record methods with other forms of food records, although some work is underway (Learn More about Technology in Dietary Assessment).

Understanding Misreporting

Misreporting on dietary assessment instruments can occur either by [glossary term:] overreporting and underreporting of intakes. Knowledge of who is likely to misreport, and in which direction, is useful in interpreting the food record results (Learn More about Misreporting).

Many studies have examined misreporting, looking at a variety of characteristics. Underreporting of energy is more common than overreporting in the United States, but this is not universal in all countries. Studies using recovery biomarkers have reported that respondents with higher body mass index (BMI) and women consistently underreport energy [14-15]. (Learn More about Reactivity and Learn More about Social Desirability).

Considering Measurement Error

Measurement error refers to the difference between the true value of a parameter, such as true energy intake, and the value obtained from a particular measure, for example, energy reported on a food record (see Key Concepts About Measurement Error). There are two types of measurement error:

Food record data are affected by [glossary term:] day-to-day variation. Although day-to-day variation does not reflect "error" in reporting intake for a given day, it is considered to be part of within-person random error from the perspective of estimating [glossary term:] usual dietary intake distributions using data for a small number of days. Because consecutive days of report are correlated, multiple non-consecutive administrations of [glossary term:] n-day food records are needed to account for day-to-day variation. If non-consecutive multiple n-day food records are collected, food record data are assumed to contain more within-person random than systematic error. This type of error can be corrected with statistical modeling.

The Nutrition and Physical Activity Assessment Study [14] found that food records (and 24HR) better capture truth (i.e., had less bias) as measured by biomarkers for energy and protein than did an FFQ. However, bias in food record data may still be substantial because of [glossary term:] person-specific biases that relate to personal characteristics of respondents or individual limitations in completing food records accurately. These biases include errors in recording foods consumed, reactivity, and coding errors due to incomplete details provided by the respondent.