The error led the department to say mistakenly that the project, known as FutureGen, had nearly doubled in cost an increase the Bush administration deemed too expensive.
At the time, FutureGen was the leading effort to capture and sequester carbon dioxide, the main heat-trapping gas linked to global warming. If the project were resumed and proved successful, it could provide a model for curbing the carbon dioxide that coal adds to the atmosphere.
The new energy secretary, Steven Chu, has said that he will consider renewing support for FutureGen but that changes will be needed.
He did not specify what those might be.
The report was prepared by the Government Accountability Office for Representative Bart Gordon, chairman of the House Science Committee. The panels subcommittee on energy and environment is holding a hearing on the issue.
Mr. Gordon, Democrat of Tennessee, said, I am astonished to learn that the top leadership of the Department of Energy in the last administration made critical decisions about our nations energy future and capacity to combat global warming based on fundamental budget math errors.
He added, This is math illiteracy on a grand scale and with global consequences.
According to the report, in calculating the costs of the project, the Energy Department mistakenly compared two numbers that should not have been used together. One cost estimate was made in so-called constant dollars, reflecting the purchasing power of a dollar in 2005, and the other in dollars as they would have been spent over the following few years, worth less each year because of inflation.
The Bush administration said the projected cost had nearly doubled, to $1.8 billion from $950 million; the auditors said it had gone to $1.3 billion, up 39 percent.
In January 2008, after investing $174 million in FutureGen, the Bush administration withdrew support. The staff of the House committee found internal communications indicating that the leaders of the Energy Department were looking for reasons to kill the project.
Under the deal, the private sector was supposed to pay 26 percent of FutureGens costs. The other 74 percent would have been paid mostly by the federal government, although 8 percent would have come from foreign government partners, including China and India, two countries that Washington was hoping to persuade to shift to coal plants that can capture their carbon.
Some executives of utilities not involved with FutureGen have said the arrangement was too generous to the private partners. But the cancellation was costly, too. The internal documents found by the staff members of the House committee predicted that killing the Illinois project would set back the technology of carbon capture by 10 years.
What the Energy Department proposed instead was to support carbon capture technologies that would be added to coal plants already built or in the planning stages. But that effort has bogged down, partly because a number of the planned plants were canceled because of high costs.
The stimulus bill passed by Congress may provide money for the original FutureGen project.