In broadcast remarks, Olmert said he was announcing his decision "to resign from my post as prime minister of Israel".
Olmert, who faces criminal indictment in corruption investigations, said at the weekly cabinet session that he believed he was acting "properly and in accordance with good governance" in stepping down.
It was not immediately clear when Olmert would formally submit his resignation to President Shimon Peres. After he does so, he will become caretaker prime minister until Israel has a new government through a coalition deal or an early election.
Olmert was replaced by Foreign Minister Tzipi Livni as leader of the ruling Kadima party in an internal election on Wednesday.
He had promised to resign once a new Kadima chief was chosen. At the cabinet session, Olmert wished Livni well, shook her hand and called the country to support her.
If Livni, Israel's chief negotiator in peace talks with the Palestinians, gets the nod from Peres to try to form a new government, she will have up to 42 days to put together a coalition.
Failure to build a coalition would lead to an early parliamentary election, plunging Israel into deeper political uncertainty and dimming even further prospects of meeting a U.S. target for an Israeli-Palestinian peace deal this year.
