Opposition leaders joined the protest, said to be the largest since the Orange Revolution in 2004.
There are reports of police firing tear gas as protesters tried to break through a cordon around government buildings.
A pro-government rally a few miles away attracted about 10,000 people.
Ukraine made the decision on the EU deal last week, saying it could not afford to break ties with Moscow.
Russia is trying to bring Kiev into its own customs union.
Russian President Vladimir Putin accused the EU of blackmailing Ukraine to sign the deal during a summit in Vilnius next week.
People arrived at the rally, on European Square, with families and children, many holding banners with slogans like "I want to live in Europe" or "Ukraine is part of Europe".
"We want to be together with Europe," Volodymyr Mnikh, a 62-year-old retired chemist, told the Associated Press news agency.
"We want our children to have a future and not to be pressured by Russia."
Several rallies in Kiev and other cities have been held over the last few days, but Sunday's has been the largest so far.
World heavyweight boxing champion Vitali Klitschko, who leads the Udar movement and attended Friday's rally, was not present.
Ukrainian news agency Unian said he had been flying back from the US after celebrating his daughter's birthday but his plane was not allowed to land in Kiev because of weather conditions.
On Friday, Ukrainian Prime Minister Mykola Azarov said the decision not to sign the deal was motivated solely by economics and was "tactical".
He said it did not alter Ukraine's overall development strategy.
The Ukrainian government says it is now looking into setting up a joint commission to promote ties between Ukraine, Russia and the EU.
Ukraine depends on imports of Russian gas, but recently the supplier, Gazprom, complained that Ukraine had fallen behind in payments.
Pipelines transiting Ukraine pump Russian gas to many EU member states.
In 2009 Russia cut off gas supplies to Ukraine in a dispute about prices, causing fuel shortages across Europe in the middle of winter.
Russia has a customs union with two other former Soviet republics - Belarus and Kazakhstan - and has been urging Ukraine to join it.
A free trade agreement with the EU would mean Ukraine adopting different trade rules.
Kiev's decision on Thursday prompted EU Enlargement Commissioner Stefan Fuele to cancel a trip to Ukraine.
It had been planned for the run-up to the key "Eastern Partnership" summit between the EU and several ex-Soviet states, which will be held in Vilnius, Lithuania, on 28 and 29 November.
Source: BBC News