When carbon dioxide comes in contact with the rock, peridotite, the gas is converted into solid minerals such as calcite.
Geologist Peter Kelemen and geochemist Juerg Matter said the naturally occurring process can be supercharged 1 million times to grow underground minerals that can permanently store 2 billion or more of the 30 billion tons of carbon dioxide emitted by human activity every year.
Their study will appear in the November 11 edition of the Proceedings of the Natural Academy of Sciences.
Peridotite is the most common rock found in the Earth's mantle, or the layer directly below the crust. It also appears on the surface, particularly in Oman, which is conveniently close to a region that produces substantial amounts of carbon dioxide in the production of fossil fuels.
"To be near all that oil and gas infrastructure is not a bad thing," Matter said in an interview.
They also calculated the costs of mining the rock and bringing it directly to greenhouse gas emitting power plants, but determined it was too expensive.
The scientists, who are both at Columbia University's Lamont-Doherty Earth Observatory in New York, say they have kick-started peridotite's carbon storage process by boring down and injecting it with heated water containing pressurized carbon dioxide. They have a preliminary patent filing for the technique.
They say 4 billion to 5 billion tons a year of the gas could be stored near Oman by using peridotite in parallel with another emerging technique developed by Columbia's Klaus Lackner that uses synthetic "trees" which suck carbon dioxide out of the air.
More research needs to be done before either technology could be used on a commercial scale.
Peridotite also occurs in the Pacific islands of Papua New Guinea and Caledonia, and along the coast of the Adriatic Sea and in smaller amounts in California.
Big greenhouse gas emitters like the United States, China and India, where abundant surface supplies of the rock are not found, would have to come up with other ways of storing or cutting emissions.
Rock storage would be safer and cheaper than other schemes, Matter said.
Many companies are hoping to cut their greenhouse gas emissions by siphoning off large amounts of carbon dioxide from coal-fired power plants and storing it underground.
That method could require thousands of miles of pipelines and nobody is sure whether the potentially dangerous gas would leak back out into the atmosphere in the future.