Brain scans show that skipping breakfast makes fatty, high calorie foods appear far more attractive later in the day, according to researchers.
Scans of 21 people showed the brain was more attracted to food if breakfast was missed and people had more food at lunch.
Scientists said it made losing weight challenging as missing meals made calorific food even more appealing.
Nutrition experts say breakfast is known to take the edge off appetite.
However, researchers were curious about what happened inside the brain to alter the food people choose to eat.
Twenty one people, who were all normal weight, were shown pictures of calorie packed foods while they were positioned in a magnetic resonance imaging (MRI) machine at Imperial College London.
On one day they were given no breakfast before the scans and on a different day they were fed a large, 730 calorie, breakfast an hour and a half before.
The researchers said skipping breakfast created a "bias" in the brain in favour of high calorie foods.
The results, presented at the Neuroscience 2012 conference, showed the brain changed how it responded to pictures of high calorie foods, but not low calorie foods, when breakfast was skipped.
They showed part of the brain thought to be involved in "food appeal", the orbitofrontal cortex, became more active on an empty stomach.
When the researchers offered the participants lunch at the end of the study, people ate a fifth more calories if breakfast was missed.
Dr Tony Goldstone, from Imperial College London, said: "Through both the participants' MRI results and observations of how much they ate at lunch, we found ample evidence that fasting made people hungrier, and increased the appeal of high calorie foods and the amount people ate.
"One reason it is so difficult to lose weight is because the appeal of high calorie food goes up."
Dr Catherine Hankey, a senior lecturer in nutrition at the University of Glasgow, said research had shown that breakfast "takes the edge off appetite" and that the latest study was an "interesting" insight.
She said breakfast was linked to stable blood sugar levels, which "keeps you on the straight and narrow".
Future studies will investigate how obesity affects the same system in the brain.