Key Concepts about Validation

Note: Because Validation and Measurement Error are closely related concepts in dietary assessment using self-report instruments and are equally important to being able to conduct research using these instruments, we suggest that you read both Key Concepts sections of the Primer in tandem. You may also wish to work with a statistician in applying the concepts described here.

Validation in dietary assessment is conducted to determine how accurately self-report instruments measure [glossary term:] true intakes. Two main types of designs are used in validation studies:

• The first type assesses validation for a specific number of days and collects reference measures such as direct observation, feeding studies (Learn More about Observation and Feeding Studies), or [glossary term:] recovery biomarkers (Learn More about Biomarkers), for a time period exactly consistent with each [glossary term:] self-reported intake day. The results of this type of study provide calculations of differences in true versus reported intakes of nutrients and food groups, proportion of foods and drinks accurately reported and omitted, and [glossary term:] correlation coefficients.
• The second type, which is more common, assesses how well reported intakes match true usual intakes (Learn More about Usual Dietary Intake) and collects reference measures such as recovery biomarkers or imperfect (also referred to as less-biased) self-report dietary assessment instruments for a time period not exactly consistent with each self-reported intake day. The results of this type of [glossary term:] validation study are summarized by characteristics that include overall [glossary term:] bias, correlation coefficients between self-reported and true usual intakes (often referred to as validation coefficients), and [glossary term:] attenuation factors. Bias is the difference between average reported intake and average true intake, at the group level. Correlation coefficients between self-reported and true usual intakes are related to the loss of [glossary term:] power to detect diet-health [glossary term:] outcome relationships and are therefore useful for estimating the sample size of dietary studies. Attenuation factors represent bias in the estimated effect of self-reported dietary components on a health outcome (Learn More about Regression Calibration) [1].

A key goal of all validation research is to use an unbiased reference measure or instrument that captures true intake without systematic error. Known unbiased reference measures include recovery biomarkers and data from feeding studies or direct observation, but few such measures are available and/or feasible [2]. Therefore, dietary assessment instruments are often evaluated using as a reference another self-report instrument thought to capture diet with systematic bias, but with less bias than the instrument being evaluated (the main instrument). We refer to such instruments as imperfect references. For example, 24-hour dietary recalls (24HRs) are an example of an imperfect [glossary term:] reference instrument used for validating food frequency questionnaires (FFQs).