Dangerous levels of indoor air pollutants from badly ventilated cooking fires are a common hazard, while lack of electricity deprives many of the benefits of refrigeration.
Paul Wilkinson of the London School of Hygiene and Tropical Medicine said the world's richest populations use up to 20 times more energy per head than those from poor countries, posing a challenge to improve energy supply without pollution.
Writing in the Lancet medical journal, Wilkinson and colleagues estimated 2.4 billion people worldwide were exposed to pollution from inefficient burning of solid fuels like wood, coal and dried cow dung.
This causes around 1.6 million premature deaths each year - roughly double the level of deaths from air pollution in cities - and many more non-fatal cases of respiratory diseases. At the same time, around 1.6 billion people worldwide have no electricity.
"Paradoxically, the poor are using much less energy but they are getting all the adverse effects," Wilkinson said in an interview.
"We in the more developed countries have access to clean energy and are using much more of it and are contributing to the global problem of climate change, where the main adverse effects are likely to fall, once again, on lower-income countries."
Global warming could trigger a range of health problems including more extreme heatwaves, increases in water-borne and insect-borne diseases, and threats to food supplies.
Lancet editor Richard Horton said the research showed that the current debate on climate change and new energy sources was unbalanced and too narrow.
"It neglects a far larger set of issues focused on energy and health," he said.