The parliament passed a law on Tuesday, allowing the removal of government officials from their posts.
All those who worked under ousted President Viktor Yanukovych and also former senior Communist and KGB members will be affected.
Ukraine has had months of unrest since Mr Yanukovych was ousted in February.
Government troops had been fighting pro-Russia separatists in the eastern Donetsk and Luhansk regions for months, until a truce was signed on 5 September.
Ukraine and the West accuse Russia of sending its troops and heavy weaponry to help the rebels - a claim denied by the Kremlin.
"About one million civil servants of different kinds will come under this law, including the whole cabinet of ministers, the interior ministry, the intelligence services, the prosecutor's office," Mr Yatseniuk said in a televised cabinet meeting.
Correspondents say the issues of vetting and corruption are emotional subjects for many in Ukraine, who want to cleanse the government of Yanukovych's influence.
The law on "lustration" - the cleansing of the ranks of power - was approved under huge pressure from activists, who took part in mass protests against Yanukovych.
The bill was finally passed after several failed attempts when speaker Okexandr Turchynov warned MPs he would not allow them to leave parliament without a successful result.
Outside the building, MP Vitaly Zhuravsky, who belongs to a party described as pro-Russian, was thrown by angry crowds into a rubbish bin.
The bill was approved on the same day as a new law granting self-rule to parts of the Donetsk and Luhansk regions.
It was part of the truce agreed between separatists and the Ukrainian government, although rebel leaders say they will continue to demand independence, and some Ukrainian MPs have described the move as "capitulation".
At least 3,000 people have been killed in the conflict and more than 310,000 internally displaced in Ukraine, the UN says. line
Ukraine's mainstream press sees the EU association agreements that were ratified on Tuesday as putting the country "At the EU threshold", in the words of the headline in popular tabloid Segodnya.
But commentators are not impressed by parliament's approval for a bill granting self-rule to the rebel-controlled regions in eastern Ukraine.
"This is a great victory for Putin and the Kremlin, because now they have the legitimisation of their terrorist groups in Ukraine", says a commentary in the daily Den.
"President Poroshenko will be held to account for Donbas' 'special status' at the next presidential election, if he decides to run for a second term," says Ukrayina Moloda daily.
Some media voices suggest the self-rule deal was the right move, considering the circumstances.
The adoption of the law "paves the way for the solution to the crucial gas problem on fairly good conditions for Ukraine," according to pro-Russian tabloid Vesti.
Ukrainian social media users are more forthright, expressing outrage at parliament's approval for the bill on special status.
Anger and scepticism from activists and pundits is directed not only at MPs who voted behind closed doors on 16 September, but also at President Poroshenko.
"Putin won, and the war didn't achieve anything. It's a betrayal of Ukraine and all those who died in the fight for Donbas," says one prominent activist on Facebook, in a comment echoed by others on Ukrainian social media.
Also on Tuesday, the Ukrainian and European parliaments voted to ratify a major EU-Ukraine association agreement that aims to bring the ex-Soviet republic closer to the EU.
The agreement lies at the root of Ukraine's crisis.
It was Yanukovych's refusal to sign the deal last November that triggered mass protests and his eventual fall from power.
Following Yanukovych's toppling, Russia annexed Crimea in the south, and pro-Russia rebels seized areas in eastern Ukraine, demanding independence.
The EU agreement would make Ukraine compliant with EU standards in the areas of human rights, security and arms control, and would remove trade barriers.
But negotiations with Russia last week led to the free-trade part of the agreement being postponed until 2016.
Source: BBC News Europe