It was a jab at ousted president Viktor Yanukovych, who along with his government cronies had a notorious penchant for gaudy luxury.
Yatsenyuk's interim government is seeking to carry out sweeping reforms to break from a culture of self-interest, cynicism and corruption that left the country of the verge of bankruptcy.
Images of chandeliers, gilded pillars and ornate marble flooring that emerged from Yanukovych's mansion after he fled caused revulsion across Ukraine.
By contrast, Yatsenyuk flies economy, lets whistleblowers into his home and readily admits to mistakes.
Those qualities have helped him grow in stature and win support for his administration — even as Ukraine struggles to avoid a possible breakup.
Yatsenyuk leads a motley team of young pro-Western professionals, idealists, nationalists and heroes of the Maidan protests — named after the square that was the magnet of dissent — as well as veterans of rough-and-tumble Ukrainian politics.
As the eclectic group assumed power in February, it knew it faced a daunting challenge:
State coffers were empty, the country was deeply polarized and the protest movement was not willing to give the new government any easy breaks.
Then things got dramatically worse:
Ukraine lost Crimea to Russia and the government found itself battling pro-Kremlin separatists, while trying to avoid war with its giant neighbor to the east.
"Nobody fully realized the bonus, so to speak, we would be getting in the annexation of Crimea and separatists movements," Ostap Semerak, who holds the title of Minister of the Cabinet of Ministers and is a close ally of Yatsenyuk, told The Associated Press.
The road has been paved with successes and setbacks.
But during the rocky journey, Yatsenyuk, who exudes the air of a somewhat nerdy intellectual, has gained respect by proving to be steadfast in the face of quixotic tasks, ready to sacrifice personal interests for the country's good.
Yatsenyuk often refers to his new job as a suicide mission — and, when congratulated recently on his post, he quipped that condolences were more in order.
While eastern Ukraine is deeply suspicious of the new government, the rest of Ukraine appears to trust its new leaders:
A nationwide April poll conducted by the International Republican Institute shows full or partial support for the Yatsenyuk government at 52 percent, up from 46 in March.
The poll had 1,200 respondents and a margin of error of 2.8 percentage points.
The positive assessment is not shared by Russia, which casts the new government as a group of nationalist radicals that seized power during an armed coup.
As the country prepares for May 25 presidential elections to choose a new leader, all eyes are on the interim government and its ability to ensure a successful vote, even as eastern and southern regions are riven by unrest.
One undeniable success of Yatsenyuk's Cabinet was getting a $17 billion loan commitment from the International Monetary Fund to avoid a meltdown of the debt-ridden economy.
Getting the loan required implementing highly unpopular reforms such as allowing devaluation of the Ukrainian currency, the hryvna, and hiking household utilities bills.
"I don't promise an improvement, neither today nor tomorrow. Our main task is to stabilize the situation in the country," Yatsenyuk told parliament in February as he was being appointed.
"We have no other way but to make extremely unpopular decisions."
Anticipating widespread discontent, Yanukovych's administration had resisted such reforms for years and failed to secure the aid.
Despite being a wealthy man, Yatsenyuk, flies coach when on trips to Western capitals — a sharp contrast to his predecessors who preferred government jets or first-class luxury.
Yatsenyuk also announced austerity measures for state employees, putting government cars, country resorts and other property up for sale.
"We are introducing the toughest system of control and reduction of budget expenditures in the financial history of Ukraine," Yatsenyuk told lawmakers.
A lawyer by training, Yatsenyuk, 39, made his fortune in banking before moving into government and serving as the country's economy minister, foreign minister and parliament speaker.
He has two daughters with his wife Tereza, 43, who is involved in public activism and charity work.
Tall, skinny and bespectacled, Yatsenyuk was a prominent figure in the Maidan protests movement that ousted Yanukovych, but lacked the charisma to become the ultimate leader of the protests.
He was at one point ridiculed for vowing to take a "bullet in the forehead" from police forces — then shying away from any confrontations.
But as prime minister, candid talk and a willingness to make tough decisions have boosted Yatsenyuk's support.
"Overall, I think Yatsenyuk has come out as a pragmatic, sensible and principled leader, who is trying as far as possible to uphold peace, integrity and the basic principles of Maidan," said Tim Ash, an emerging markets analyst with Standard Bank in London, citing "respect for basic human rights, the rule of law, a market economy and democracy."
But in his push to secure a Western future for Ukraine, Yatsenyuk hasn't been immune to mistakes, observers say.
It was only after the loss of Crimea, three weeks after he assumed office, that Yatsenyuk issued a Russian-language appeal to Ukrainians in the restive Russian-leaning east, in which he guaranteed them the right to speak Russian and promised broader regional powers.
One of the first bills passed by parliament in the afterglow of the Maidan victory was the repeal of a law which allowed the use of Russian language in formal settings in Russian-speaking regions.
In the face of a huge backlash, acting President Oleksandr Turchynov vetoed that bill, but the damage had been done:
Hostility toward the government surged in the Russian-dominated east and the Kremlin threatened to invade Ukraine to protect Russian speakers.
Yatsenyuk voted for the repeal of the law together with his party, but later admitted that it was a mistake.
He has since then proposed amending the constitution to allow regions to make language choices for themselves.
Despite angry words, Kiev put up virtually no resistance to Russia's annexation of Crimea and dragged its feet on evacuating military personnel from the peninsula, leaving them besieged and harassed by Russian forces and pro-Russia militiamen.
The government has suffered several embarrassing setbacks while trying to quash the insurgency in the east, such as the loss of several armored personnel carriers to pro-Russian insurgents and the capture of security officers.
And a lot remains to be done.
The government has yet to deliver on Maidan promises of preventing former officials incriminated in corruption from ever holding office and completing its investigation into the deaths of over 100 activists killed in clashes with police.
The Cabinet must also show that its tough anti-corruption rhetoric translates into real improvements in the long-run.
But many experts say that the government deserves credit given the sheer scale and complexity of the challenge it faces.
"One can criticize the current government a lot," said political analyst Vadim Karasyov.
"But one should take into account the kind of extremely difficult historical and dramatic conditions they must work in today. This government is acting in the conditions of a revolution, a war, presidential elections and an economic crisis simultaneously."