ORIGINAL

 

Reproducibility and relative validity of a semi-quantitative food-frequency questionnaire in an adult population of Rosario, Argentina

Reproducibilidad y validación relativa de un cuestionario de frecuencia de consumo de alimentos semicuantitativo en población adulta de Rosario, Argentina

 

 

María Elisa Zapata^{a, *}; Romina Buffarini^b; Nadia Lingiardi^a and Ana Luiza Gonçalves-Soares^b

^a Nutrition School, University of Latinoamerican Education Center, Argentina.
^b Post-Graduate Program in Epidemiology, Federal University of Pelotas, Brazil.

Funding: Bank of Santander (PhD Programme Santander-Iberoamérica, PhD 314 fellowship).

Correspondence

 

 

---

ABSTRACT

Introduction: Dietary assessment of nutrients and food groups by food frequency questionnaire needs to be validated in each population. The objective of this cross-sectional study was to evaluate the reproducibility and relative validity of a semi-quantitative food frequency questionnaire among adults of Rosario, Argentina.
Material and Methods: Two food frequency questionnaires and four 24-hour dietary recalls were applied in a sample of 88 adults. Reproducibility of food frequency questionnaire was estimated by correlation coefficients, and validity was assessed comparing the second food frequency questionnaire and the average of the 24-hour dietary recalls using cross-classification and Bland-Altman analyses.
Results: Pearson and intraclass correlation coefficients showed higher correlation for food groups than for energy and nutrients. More than 65% of the subjects were classified into the same quintile or into the adjacent quintile by both food frequency questionnaires. Bland-Altman plots showed reasonably acceptable agreement between the second food frequency questionnaire and 24-hour dietary recalls, especially for the main Argentinian foods groups (meat, cereal, dairy products, vegetables and fruits).
Conclusions: Overall, food frequency questionnaire showed acceptable reproducibility and relative validity for assessment main food groups and nutrient intakes for adult population in Rosario.

Key words: Diet; Diet surveys; Reproducibility of Results; Epidemiology; Adult; Argentina.

---

RESUMEN

Introducción: La evaluación del consumo de nutrientes y grupos de alimentos mediante cuestionarios de frecuencia de consumo requiere ser validada en cada población. El objetivo de este estudio transversal fue evaluar la reproductibilidad y la validez relativa de un cuestionario semicuantitativo de frecuencia de consumo en población adulta de Rosario, Argentina.
Material y Métodos: Dos cuestionarios de frecuencia de consumo y cuatro recordatorios de 24 horas fueron aplicados en una muestra de 88 adultos. La reproductibilidad del cuestionario de frecuencia de consumo fue estimada mediante el coeficiente de correlación y la validez fue evaluada comparando el segundo cuestionario de frecuencia de consumo y el promedio de los recordatorios de 24 horas mediante clasificación cruzada y análisis de Bland-Altman.
Resultados: Los coeficientes de Pearson e intraclase mostraron mayor correlación para los grupos de alimentos que para energía y nutrientes. Más del 65% de los individuos fue clasificado en el mismo quintil o en el adyacente por ambos cuestionarios de frecuencia de consumo. El gráfico de Bland-Altman mostró una razonable concordancia entre el segundo cuestionario de frecuencia de consumo y recordatorios de 24 horas, especialmente para los principales grupos de alimentos de la dieta Argentina (carne, cereales, lácteos, vegetales y frutas).
Conclusiones: En general, el cuestionario de frecuencia de consumo presentó una reproductibilidad aceptable y validez relativa para evaluar los principales grupos de alimentos y la ingesta de nutrientes en la población adulta de Rosario.

Palabras clave: Dieta; Encuestas alimentarias; Reproducibilidad de los resultados; Epidemiología; Adulto; Argentina.

---

 

Introduction

During the 1980s and 1990s, food-frequency questionnaires (FFQs) have become a key research tool in nutrition epidemiology¹. The FFQ is often the most feasible method for collecting data on regular dietary intake of large population samples. The quicker administration and processing, and subsequently lower costs and participant burden than alternative methods such as diet history or repeated 24-hour dietary recalls (24-HRs) are some advantages of the FFQ², as well as the ability to rank individuals according to dietary intake³. However, dietary assessment of nutrients and food groups by FFQ needs to be validated.

Currently, there is no gold standard for the validation of dietary intake. The basic requirement for validation is that the errors of the reference method have to be independent of the tested method. The major sources of errors in FFQs include memory, interpretation of questions, perception of portion sizes, and the restricted food list. Diet records have the least correlated errors with FFQs⁴, but also give great subject burden, decrease the response rate, and may even change subjects' diet. Multiple 24-HRs are widely considered as an alternative method to diet records. A critical review regarding validation of FFQs has shown that FFQs are validated against repeated 24-HRs in 75% of studies⁵.

In this study, the nutrient and food group intakes assessed by FFQ were compared to data obtained from four 24-HRs. The overall goal of this study was to evaluate the reproducibility and relative validity of the FFQ adapted from Balearic Island Nutrition Research Questionnaire, that will be used in the study of nutritional status and eating habits of the adult population of Rosario, Argentina.

 

Material and Methods

This cross-sectional study was conducted in the urban area of Rosario, Argentina, from November 2011 to August 2012. Participants were recruited by convenience sampling at the University of the Latin American Education Center. Individuals aged 18 to 65 years were eligible to participate of the study (n = 88). Roughly 60% of the sample was female, the mean age was 33.8 (SD 12.5) years, and the mean BMI was 25.2 kg/m² (SD 6.3). The majority of them had secondary educational level (54.7%) and 40.7% had university level, 69.8% were single, 72.1% were non-smoker, and 67.4% did some leisure-time physical activity at least once a week.

Two FFQs and four 24-HRs were administered. The 66 fooditem FFQ was developed considering foods and portion size of Dietary Guidelines for Argentinean Population⁶. Participants had to recall the number of times each food item was consumed per day, per week, per month or never during the past year of the amount of all foods consumed. The first FFQ (FFQ1) was administered at the same time of the first 24-HRs, and the second FFQ (FFQ2) was administered ten months later, together with the assessment of the last 24-HRs (Figure 1). The four 24-HRs were administered at baseline and every 3 months during the study period, covering three weekdays and one weekend day. Both methods, FFQ and 24-HRs, were administered face to face by trained interviewers, using visual model of food. Energy and nutrient intake data were analyzed using software SARA version 1.2.12. The main source of information used was the Argenfoods table.

 

 

Food intake obtained from 24-HRs and from FFQs was grouped into 20 food groups (dairy products; eggs, meat; meat products; fish or seafood; vegetables; potatoes; fruits; nuts; legumes; cereals; olive oils or olives; pastries, cakes or sweets; seed oils; other fats; sauces; soft drinks; tea or coffee; alcoholic beverages; and snacks).

First, mean, standard deviation (SD) and median were calculated for both FFQs and 24-HRs. The mean daily intake of the four 24-HRs was used as representative of all 24-HRs. The significance of the differences of food groups and nutrients intake between FFQs and the average of the four 24-HRs, and between FFQ1 and FFQ2 were determined using Wilcoxon signed rank test.

Then, the data were transformed (log10) to optimize the normality of the distribution. Pearson product-moment correlation coefficients (PPMCC) and intraclass correlation coefficients (ICC) of log-transformed food and nutrient intakes were calculated, and energy-adjusted nutrient intakes were calculated using the residual method⁴. To correct for within-individual error in the measurement of the 24-HRs, the observed correlation coefficient was multiplied by the de-attenuation factor (1+(σ²_w /σ²_b )/n)^0.5. The within- and between-individual variance components were determined by a random-effect model with the recorded intake as the dependent variable and subject identification number as the independent variable⁷.

The FFQ2, instead of FFQ1, was chosen to explore relative validity because it covered the same time period (i.e. previous year) as the four 24-hour recalls used as reference. Quintiles of intake were also calculated and the degree of gross misclassification in FFQ2 in relation to the 24-HRs was evaluated using contingency tables. To graphically check the agreement between the two methods in terms of absolute intake, the analysis proposed by Bland and Altman was used⁸. All statistical analyses were performed using SPSS version 19.0 (SPSS Inc., Chicago, United States) and STATA version 12.1 (StataCorp, College Station, Texas, USA).

The study protocol was approved by the Faculty of Chemistry of the University of the Latin American Education Center and the Secretary of Public Health of Rosario. Before joining the study, all participants agreed and provided written informed consent form.

 

Results

Descriptive statistics of intake are presented in Table 1. Wilcoxon signed-rank test showed significant lower consumption of dairy products, fish or seafood, vegetables, olive oils or olives, seed oils, and alcoholic beverages in FFQ2 than in FFQ1. Lower consumption of vegetables and cereals was also observed, however the consumption of dairy products and fruits was higher in FFQ2 than in 24-HRs. FFQ2 underestimated the consumption of energy, carbohydrate, fat, PUFA, folate, vitamin A, sodium and potassium, and overestimate the intake of fiber, niacin, vitamin B12, vitamin C, calcium, phosphorous and zinc compared to 24-HRs.

Table 2 shows the reproducibility of the FFQ measurements (FFQ1 vs. FFQ2) in terms of the PPMCC and ICC. For food groups, PPMCC ranged from 0.08 to 0.84, and in 15 out of 20 food groups it was above 0.54 (mean 0.56, SD 0.16). ICC ranged from 0.14 to 0.90, and in 13 out of 20 food groups it was above 0.71 (mean 0.70, SD 0.16). For energy and nutrient intakes, the PPMCC ranged from 0.43 to 0.67 (mean 0.53, SD 0.07) and from 0.29 to 0.74 (mean 0.53, SD 0.11) for unadjusted and energy-adjusted data, respectively. ICC ranged from 0.56 to 0.77 (mean 0.67, SD 0.06) and from 0.35 to 0.81 (mean 0.62, SD 0.11) for unadjusted and energy-adjusted data, respectively.

The percentages of gross misclassification are shown in Table 3. For the five mainly consumed food groups (i.e. dairy products, meat, vegetables, fruits, and cereals), among 69.8% to 83.7% of the individuals in FFQ2 were classified in the same or in the adjacent quintile of the 24-HRs, and 40.7% to 72.1% for energy and nutrient intake. The overall extreme misclassification was always lower than 10.0%, except for vitamin C, which was 18.6%.

Figures 2 and 3 present the differences between FFQ2 and 24-HRs. The Bland-Altman plots showed a general underestimation of the consumption of food groups, and overestimation of nutrient intake by the FFQ. Energy, carbohydrate and fat intake were underestimated, although the mean differences were close to zero. Small differences were also observed for underestimated groups.

The mean error in intake assessment by FFQ does not change with different values of dietary intake for cereals, iron, sodium, potassium, thiamin, riboflavin and vitamin C. On the other hand, a systematic variation in agreement between the two methods was observed. As the intakes of some nutrients increases, the FFQ showed an overestimation, as the case of SFA, MUFA, riboflavin and niacin; and an underestimation of cholesterol and PUFA consumption. For dairy products, fruits, seed oils, olives, tea, and coffee, water and soft drinks low intakes were underestimated whereas high intakes were overestimated by the FFQ; for eggs, fish, legumes, potatoes, alcoholic drinks, pastries, snacks, and other fats, low intakes were overestimated and high intakes were underestimated by the FFQ (data not shown).

 

Discussion

This study examined the validity and reproducibility of the FFQ modified from Balearic Island Nutrition Research FFQ to assess the food group and nutrient consumption of an adult population of Rosario (Argentina). The results suggest that the FFQ has a good reproducible estimation and a reasonable validity for nutrient intake and the main Argentinean food groups.

In Latin and Central America many studies have been conducted to assess the reproducibility and validity of FFQs, but only three published studies for adult population were found in Argentina. Navarro et al. conducted a study in Cordoba with individuals aged 23 to 80y to validate a FFQ specific for adults with cancer⁹. Dehghan et al. validated a FFQ with 116 women and 40 men aged 52.7±9.5y living in rural and urban area of Rosario¹⁰, and Elorriaga et al. assessed 157 individuals, aged 21 to 74 years old of Argentina, Uruguay and Chile¹¹. However, these studies assessed only nutrients or selected food groups. The energy and macronutrient intakes observed in this study were similar to that obtained by Dehghan et al. through 24-hour recall¹⁰. On the other hand, Elorriaga et al.¹¹ obtained lower values of energy and nutrients intakes, possibly because of the sampling selection of the study, which was done in primary-care clinics. In their study, the mean age of the sample was higher, which could have resulted in a lower mean intake compared with our results¹¹.

We found that the mean and median daily intakes of energy, nutrients and food groups obtained from FFQ1 were in general higher than that obtained from FFQ2. This could have happened because in the second administration of the FFQ the participants could quantify their food intake more precisely due to the familiarity with the FFQ methodology. These findings are consistent with results of other studies in similar^10,11 and different populations^12-19.

The FFQ tended to overestimate the intake of nutrients and some food group items compared to the 24-HRs, but also underestimated some macronutrients, such as carbohydrates and fat. An overestimation of the FFQ in relation to other dietary methods is a common problem, as reported by other authors^{10,12-15,17,19}, probably due to the use of long lists of foods and difficulties in estimating an accurate frequency of food consumption, as well as an overestimation of the portion sizes⁵. The underestimation of items like meat products, vegetables, and cereals could have happened because these foods are most consumed in the main meals, and this fact is better identified in the 24-HRs.

The reproducibility of the FFQ was acceptable. Cade et al.⁵ suggested a threshold of 0.4 and Masson et al.²⁰ suggested threshold above 0.5 as an acceptable reproducibility. For almost all food groups Pearson correlation coefficient was higher than 0.4 and the ICC was higher than 0.5. The highest correlations coefficients were found for those food groups more frequently consumed, like dairy products, meats, meat products and fruits. The lowest correlations coefficients were seen for tea and coffee. As the recall period of FFQ was the year before the interview, some foods more likely to be consumed at specific periods of the year (e.g. ice cream in the summer and hot beverages in the winter) or more likely to be available in food supply²¹ could be affected by recent past memory, increasing the recall²². This could have happened for tea and coffee, both of them mostly consumed during the winter, period of assessment of the last FFQ.

Higher correlation coefficients for season-dependent foods might have been observed if the two FFQs were administered in the same season.

Two main strengths were identified in this study. To our knowledge, this is the first study that validated both nutrient and food intake in Argentinian adult population. Furthermore, our analyses showed an excellent agreement for the most representatives' food groups of Argentinian Guidelines (cereals, meat, fruits and vegetable, dairy products). The Food Guidelines for Argentinian Population6 are represented by the oval healthy eating, compounded of six food groups sorted in ascending order, reflecting the extent to which each group should be eaten (cereal and legumes, fruits and vegetables, dairy products, meat and eggs, fats and oils, sugar, sweets and cakes). Moreover, Schejtman assessed the Latin America diet patterns, and described the Southern Cone pattern as wheat-meat shaft; in which the main foods of the diets are wheat, meat and dairy products²³.

On the other hand, this study has some limitations. First, FFQ was compared to 24-HRs, which is not a gold standard. However, dietary intake cannot be estimated without error and the nature and magnitude of the error depends on both the dietary data collection methodology and the subjects studied⁷. Food intake assessment methods, such as 24-HRs and dietary records, are used by 75% of the validation studies yet⁵. Moreover, the nutritional composition of each FFQ item was calculated as an average of the different foods in a sharing group (for example: almonds, peanuts, hazelnut, and nut for nuts); however the assumption that each explicitly mentioned food item contributes equally to the intake of the food group may be wrong. Additionally, the participants of the validation study were self-selected volunteers and not a random sample from the general population, most validation studies are based on volunteers yet²⁴. Volunteer subjects are suitable for validation studies if they do not differ from the source population in age or gender²⁵, and in this study they were similar to the general population. As a method to assess food intake, the FFQ presents a heavy reliance on long-term memory^1,4,26,27, but this was not a limitation of this study in particular, as it happens on every study that uses the FFQ.

The FFQ showed acceptable reproducibility and relative validity regarding energy, macronutrients and the most important food groups in the Argentinian diet. However, it is important to be careful on applying this FFQ for the evaluation of specific nutrients, especially micronutrients, which tend to be overestimated.

 

Acknowledgments

The authors thank to Maria del Mar Bibiloni for statistical advice and excellent technical support. We are grateful to the participants of the study.

 

Competing interests

The authors confirm that there are no conflicts of interest known associated with this publication and that all the sources of funding had no role in the study design, collection, analysis or data interpretation, writing the manuscript, or the decision to submit the paper for publication.

 

Bibliography

1. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutr Res Rev. 2004; 17(1): 5-22.         [ Links ]

2. Willett WC, Stampfer MJ, Colditz GA, Rosner BA, Hennekens CH, Speizer FE. Dietary fat and the risk of breast cancer. N Engl J Med. 1987; 316(1): 22-8.         [ Links ]

3. Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985; 122(1): 51-65.         [ Links ]

4. Willett W. Nutritional Epidemiology. Second ed. Oxford: Oxford University Press; 1998.         [ Links ]

5. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr. 2002; 5(4): 567-87.         [ Links ]

6. Lema S, Longo EN, Lopresti A. Guías alimentarias para la población argentina. Asociación Argentina de Dietistas y Nutricionistas Dietista, editor. Buenos Aires: Asociación Argentina de Dietistas y Nutricionistas Dietistas; 2003.         [ Links ]

7. Beaton GH. Approaches to analysis of dietary data: relationship between planned analyses and choice of methodology. Am J Clin Nutr. 1994; 59(1 Suppl): 253S-61S.         [ Links ]

8. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1(8476): 307-10.         [ Links ]

9. Navarro A, Osella AR, Guerra V, Muñoz SE, Lantieri MJ, Eynard AR. Reproducibility and validity of a food-frequency questionnaire in assessing dietary intakes and food habits in epidemiological cancer studies in Argentina. J Exp Clin Cancer Res. 2001; 20(3): 365-70.         [ Links ]

10. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative Food Frequency Questionnaire for Argentinean adults. PLoS One. 2012; 7(5): e37958.         [ Links ]

11. Elorriaga N, Irazola VE, Defagó MD, Britz M, Martínez-Oakley SP, Witriw AM, et al. Validation of a self-administered FFQ in adults in Argentina, Chile and Uruguay. Public Health Nutr. 2015; 18(1): 59-67.         [ Links ]

12. Jackson M, Walker S, Cade J, Forrester T, Cruickshank JK, Wilks R. Reproducibility and validity of a quantitative food-frequency questionnaire among Jamaicans of African origin. Public Health Nutr. 2001; 4(5): 971-80.         [ Links ]

13. Shu XO, Yang G, Jin F, Liu D, Kushi L, Wen W, et al. Validity and reproducibility of the food frequency questionnaire used in the Shanghai Women's Health Study. Eur J Clin Nutr. 2004; 58(1): 17-23.         [ Links ]

14. Zhang CX, Ho SC. Validity and reproducibility of a food frequency Questionnaire among Chinese women in Guangdong province. Asia Pac J Clin Nutr. 2009; 18(2): 240-50.         [ Links ]

15. Zhuang M, Yuan Z, Lin L, Hu B, Wang X, Yang Y, et al. Reproducibility and relative validity of a food frequency questionnaire developed for adults in Taizhou, China. PLoS One. 2012; 7(11): e48341.         [ Links ]

16. Trinidad-Rodríguez I, Fernández-Ballart J, Cucó-Pastor G, Biarnés-Jordà E, Arija-Val V. Validación de un cuestionario de frecuencia de consumo alimentario corto: reproducibilidad y validez. Nutr Hosp. 2008; 23(3): 242-52.         [ Links ]

17. Fernández-Ballart JD, Piñol JL, Zazpe I, Corella D, Carrasco P, Toledo E, et al. Relative validity of a semi-quantitative food-frequency questionnaire in an elderly Mediterranean population of Spain. Br J Nutr. 2010; 103(12): 1808-16.         [ Links ]

18. Marques-Vidal P, Ross A, Wynn E, Rezzi S, Paccaud F, Decarli B. Reproducibility and relative validity of a food-frequency questionnaire for French-speaking Swiss adults. Food Nutr Res. 2011; 55.         [ Links ]

19. da Silva NF, Sichieri R, Pereira RA, da Silva RM, Ferreira MG. Reproducibility, relative validity and calibration of a food frequency questionnaire for adults. Cad Saude Publica. 2013; 29(9): 1783-94.         [ Links ]

20. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr. 2003; 6(3): 313-21.         [ Links ]

21. Patterson RE. Methods and tools for dietary intake assessment in individuals vs. groups. En: Berdanier CD, editor. Handbook of nutrition and food. Boca Raton, FL: CRC Press; 2002. p. 523-38.         [ Links ]

22. Fowke JH, Schlundt D, Gong Y, Jin F, Shu XO, Wen W, et al. Impact of season of food frequency questionnaire administration on dietary reporting. Ann Epidemiol. 2004; 14(10): 778-85.         [ Links ]

23. Schejtman A. Consuption patterns and production structures. FAO Online Catalogues (npub); 1997. Disponible en: http://www4.fao.org/cgi-bin/faobib.exe?rec_id=266689&database=faobib&search_type=link&table=mona&back_path=/faobib/mona&lang=sp&format_name=sFMON.         [ Links ]

24. Fornés NS, Stringhini ML, Elias BM. Reproducibility and validity of a food-frequency questionnaire for use among low-income Brazilian workers. Public Health Nutr. 2003; 6(8): 821-7.         [ Links ]

25. Thompson RL, Margetts BM. Comparison of a food frequency questionnaire with a 10-day weighed record in cigarette smokers. Int J Epidemiol. 1993; 22(5): 824-33.         [ Links ]

26. Bingham SA. The dietary assessment of individuals; methods, accuracy, new techniques and recommendations. Nutr Abstracts Rev. 1987; 57: 705-42.         [ Links ]

27. Gibson RS. Principles of Nutritional Assessment. 2.a ed. New York: Oxford University Press; 2005.         [ Links ]

 

 

Correspondence:
*mariaelisazapata@gmail.com

Received: 11/08/2015.
Accepted: 08/10/2015.