Goal
Estimate ‘what people eat’ with machine learning based on images or videos of plates.
Methodology
We can start small by e.g. aiming to estimate portion sizes, in- or exclusion of vegetables at dinnertime or in-or exclusion of fruits in breakfasts. We could ask students to collect both experimental data as photo’s or video’s and more standardized data, e.g. a simple registry of portion size, to train the model.
In addition to this development: application ecological momentary assessment for dietary intake
Challenges
Main challenges in estimating portion size of food using photos from your smartphone?
- Image Quality: The quality of the photos can affect the accuracy of portion size estimates, and factors such as lighting, camera angle, and focus can influence the results.
- Perspective Distortion: Photos taken from different angles can cause perspective distortion, which can make it difficult to accurately estimate the size of objects in the image.
- Variation in Serving Size: Serving sizes can vary widely, even for the same food item, which makes it difficult to develop accurate portion size estimates based on a single image.
- Non-Standardized Food Presentation: Foods can be presented in different ways, such as arranged on a plate or served in a bowl, which can make it challenging to accurately estimate portion size.
- Cultural Differences: Different cultures have different serving size norms, which can make it difficult to develop models that work well globally.
- Complex Food Items: Complex food items, such as soups, stews, and casseroles, can be difficult to estimate portion size for, as they contain multiple ingredients that are combined in different ways.
- Reference Data: Accurate reference data is required to train machine learning models, which can be difficult to obtain for all food items
