Speaking on the BBC's Question Time, the education secretary rejected Labour criticism of the Conservative-Liberal Democrat coalition's bank levy, but also took issue with Barclays chief executive Bob Diamond's recent comments.
"The sense of contrition for what went wrong in the banking crisis seems to be invisible. The banks have to show they get it," he said, adding that some leading figures in the industry "appear ... to be living in a parallel universe".
Although shadow health minister Diane Abbott claimed the government has failed to respond adequately to public anger over the issue, former Lib Dem leader Charles Kennedy insisted a "vigorous debate" continues between the coalition partners.
Last week, ex-Barclays Capital boss Mr Diamond told a committee of MPs that the banking sector's "period of remorse and apology" should be brought to an end, allowing all concerned to move on.
By Claire Archer