There are a lot of methodological problems in nutritional research.
Inaccurate Reporting – 24-h recall
Twenty-four hour food recall consists of interviews of people to remember what they ate in the past 24 hours. These sorts of surveys are known to be inaccurate. From (Dietary methods research in the third National Health and Nutrition Examination Survey: underreporting of energy intake. R R Briefel, C T Sempos, M A McDowell, S Chien, K Alaimo. The American Journal of Clinical Nutrition, Volume 65, Issue 4, 1 April 1997, Pages 1203S–1209S) demonstrates the inaccuracy in food logs:
Assessment of diet is a critical component of the third National Health and Nutrition Examination Survey (NHANES III), which was designed to describe the health and nutritional status of the US population. We analyzed data collected with the primary dietary assessment instrument used in NHANES III, the 24-h recall, for 7769 nonpregnant adults aged > or = 20 y to investigate underreporting of total energy intake. Underreporting was addressed by computing a ratio of energy intake (EI) to estimated basal metabolic rate (BMRest). EI:BMRest was 1.47 for men and 1.26 for nonpregnant women; a population level of 1.55 is expected for a sedentary population. About 18% of the men and 28% of the women were classified as underreporters. Underreporting of energy intake was highest in women and persons who were older, overweight, or trying to lose weight. Underreporting varied according to smoking status, level of education, physical activity, and the day of the week the 24-h recall covered. Additionally, underreporting was associated with diets lower in fat (P < 0.01) and alcohol (P < 0.01 in women) when expressed as a percentage of total energy intake.
From the text of the study:
About 10% of men and 8% of women reported that their
intake for the recall day was much less than usual, whereas
3% reported that it was much more.
A unique study provides evidence of the same (Int J Obes (Lond). 2010 Mar;34(3):437-45. doi: 10.1038/ijo.2009.251. Epub 2009 Dec 15. Inaccuracies in food and physical activity diaries of obese subjects: complementary evidence from doubly labeled water and co-twin assessments. Pietiläinen KH1, Korkeila M, Bogl LH, Westerterp KR, Yki-Järvinen H, Kaprio J, Rissanen A.).
Non-obese co-twins consistently reported that their obese twin siblings ate more food overall, consumed less healthy foods and exercised less than the non-obese co-twins do. However, no differences in energy intake (9.6+/-1.0 MJ per day vs 9.8+/-1.1 MJ per day, respectively) in the food diaries or in the mean PA level (1.74+/-0.02 vs 1.79+/-0.04, respectively) in the PA diaries were found between obese and non-obese co-twins. A considerable underreporting of energy intake (3.2+/-1.1 MJ per day, P=0.036) and overreporting of PA (1.8+/-0.8 MJ per day, P=0.049) was observed in the obese, but not in the non-obese co-twins.
Good studies limit the variables to the smallest extend possible. Ideally, they’d only change the one subject of interest. For instance, there are studies comparing ad lib low carb to calorie restricted high carb diets. The two variables (low vs high carb) and (ad lib vs calorie restricted) are then confounded in the results. Did the Low Cab diet do better because it was lower carb or because people could eat as much as they want? That particular study showed that that people doing the low carb diet actually ate less calories than the high carb diet people.
Counselling and Food Surveys
Other studies provide counselling and then later ask the people what they ate. Is this testing the effects of a counselling session and people’s compliance? Or is it testing the dietary effect? There’s ways like measuring Respiratory Exchange Rate (RER) to determine the proportion of carbs to fat that the person ate in the last day or so. Without this sort of verification how can the researchers know that the person complied?
These might be a better study of how ineffective a short training session is on getting people to change their dietary habits.