Being overweight often has nothing to do with calories or exercise. For a huge number of us, the problem is misfiring hormones. Research is still catching up with this paradigm shift, which has yet to be comprehensively studied. But seeing how this revelation has helped my patients and I slim down and feel better gives me confidence that it’s true for most women who are trying to lose weight and can’t. You already know about some weight-affecting hormone issues, like thyroid and insulin imbalances. But more subtle ones could also be keeping you from the body you want. Here are some other ways your hormones might be causing weight gain.

You’re consuming too much sugar.

I think of leptin as the hormone that says, “Darling, put down the fork.” Under normal circumstances, it’s released from your fat cells and travels in the blood to your brain, where it signals that you’re full. But leptin’s noble cause has been impeded by our consumption of a type of sugar called fructose, which is found in fruit and processed foods alike.

When you eat small amounts of fructose, you’re OK. But if you eat more than the recommended five daily servings of fruit (which in recent decades has been bred to contain more fructose than it used to)—plus processed foods with added sugar—your liver can’t deal with the fructose fast enough to use it as fuel. Instead, your body starts converting it into fats, sending them off into the bloodstream as triglycerides and depositing them in the liver and elsewhere in your belly.

As more fructose is converted to fat, your levels of leptin increase (because fat produces leptin). And when you have too much of any hormone circulating in your system, your body becomes resistant to its message. With leptin, that means your brain starts to miss the signal that you’re full. You continue to eat, and you keep gaining weight.

You’re super stressed.

The so-called stress hormone cortisol can create all kinds of trouble for women who want to shed weight. When cortisol rises, it encourages the conversion of blood sugar into fat for long-term storage. Hoarding body fat in this way was a useful survival adaptation for our ancestors when they faced stressful famines. But not so much today. Obviously, reducing stress in your life will help rein in this fat-storing hormone, but there’s another very common source of the problem: daily coffee, which elevates cortisol levels dramatically, causing your body to hoard fat when you least need to.

Your high estrogen levels are expanding your fat cells.

Although estrogen is responsible for making women uniquely women, it’s also the hormone that can be the most troublesome in the fat department. At normal levels, estrogen actually helps keep you lean by goosing the production of insulin, a hormone that manages blood sugar. When estrogen gets thrown off, though, it turns you into a weight-gain machine.

Here’s how: When you eat, your blood sugar rises. Like a bodyguard, insulin lowers it by escorting glucose into three different places in your body. When insulin is in good working form—not too high and not too low—it sends a small amount of glucose to your liver, a large amount to your muscles to use as fuel, and little to none for fat storage.

If you’re healthy and in good shape, your pancreas produces exactly the right amount of insulin to have your blood sugar softly rise and fall within a narrow range (fasting levels of 70 to 85 mg/dl). But when your estrogen levels climb, the cells that produce insulin become strained, and you can become insulin resistant. That’s when insulin starts to usher less glucose to the liver and muscles, raising the levels of sugar in your bloodstream and ultimately storing the glucose as fat. Your fat tissue can expand by as much as four times to accommodate the storage of glucose.

How do estrogen levels climb, exactly? Meat is one of the primary reasons. You take in a lot less fiber when you eat meat. Research suggests that vegetarians get more than twice as much fiber as omnivores. Because fiber helps us stay regular, and we process excess estrogen through our waste, eating less fiber drives up our estrogen.

Meat also contains a type of fat with its own estrogen problem. Conventionally raised farm animals are overloaded with steroids, antibiotics, and toxins from their feed and the way they’ve been raised. When you eat them, those substances are released into your system. They can behave like estrogen in the body, adding to your overload.

Your low testosterone levels are slowing down your metabolism.

You are confronted with an astounding number of toxins each day, including pesticides, herbicides, genetically modified foods, and about six different synthetic hormones in meat. Toxins are lurking in face creams, prescription drugs, processed foods, your lipstick, the linings of tuna fish cans, the fire-retardant materials in couches, and even the air you breathe. The list goes on.

Many types of these toxins, such as pesticides, plastics, and industrial chemicals, behave like estrogen when absorbed in the body. Experts believe that our increasing exposure to toxins helps explain why so many girls are entering puberty earlier and why many boys exhibit feminine characteristics such as developing breasts. Xeno-estrogens, as these particular toxins are called, have been associated with an elevated risk of estrogen-driven diseases like breast and ovarian cancers and endometriosis.

All this fake estrogen overwhelms your body’s testosterone—which is vital for hormone balance—and contributes to estrogen overload. Testosterone contributes to muscle growth, which in turn supports metabolism. And, as we already know, estrogen overload raises insulin insensitivity. The combination adds pounds to your frame: A study from Sweden published in the journal Chemosphere showed that exposure to a particular type of pesticide called organochloride was linked to a weight gain of 9½ pounds over 50 years.

And that’s just one type of toxin. Your risk of weight gain and disease from exposure to toxins may be greater than you realize. A survey by the CDC demonstrated that 93 percent of the population has measurable levels of bisphenol A (BPA), a chemical found in store receipts and canned foods that disrupts estrogen, thyroid, and androgen hormones. Endocrine disruptors have been shown to interfere with the production, transportation, and metabolism of most hormones.

The bottom line: You have to to address your hormone imbalances.

Now you know the “whys” of your broken metabolism, these are reasons regular diets don’t address the root cause of your weight gain. Hormones dictate what your body does with food. Talk to a doctor about fixing your hormones, and your body will slim down without any extra effort from you.