Food Image Recognition

Goal

Estimate ‘what people eat’ with machine learning based on images or videos of plates.

Methodology

We can start small by e.g. aiming to estimate portion sizes, in- or exclusion of vegetables at dinnertime or in-or exclusion of fruits in breakfasts. We could ask students to collect both experimental data as photo’s or video’s and more standardized data, e.g. a simple registry of portion size, to train the model.
In addition to this development: application ecological momentary assessment for dietary intake

Challenges

Main challenges in estimating portion size of food using photos from your smartphone?

  • Image Quality: The quality of the photos can affect the accuracy of portion size estimates, and factors such as lighting, camera angle, and focus can influence the results.
  • Perspective Distortion: Photos taken from different angles can cause perspective distortion, which can make it difficult to accurately estimate the size of objects in the image.
  • Variation in Serving Size: Serving sizes can vary widely, even for the same food item, which makes it difficult to develop accurate portion size estimates based on a single image.
  • Non-Standardized Food Presentation: Foods can be presented in different ways, such as arranged on a plate or served in a bowl, which can make it challenging to accurately estimate portion size.
  • Cultural Differences: Different cultures have different serving size norms, which can make it difficult to develop models that work well globally.
  • Complex Food Items: Complex food items, such as soups, stews, and casseroles, can be difficult to estimate portion size for, as they contain multiple ingredients that are combined in different ways.
  • Reference Data: Accurate reference data is required to train machine learning models, which can be difficult to obtain for all food items
Dr. Hani Al-Ers
Senior Researcher
Hani Al-Ers is a researcher in the field of human-machine interactions. He completed his PhD at the Delft University of Technology at the Interactive Intelligence group of the Faculty Computer Science (EEMCS). Philips Research in Eindhoven sponsored his project which was aimed at improving the user experience of Philips tv sets. He completed 2 post-docs at the Delft University of Technology, during which he managed international consortia on topics such as an improved quality of life for the elderly. Currently, he is conducting research in the field of health and education and he leads the Research Education activities at the Dutch Innovation Factory.