Chancellor Alistair Darling said he was "confident but cautious" that Britain was on the path to a sustained economic recovery.
Mr Darling, in Davos for the annual meeting of the World Economic Forum, said he believed the action taken by governments around the world to shore up their economies was beginning to restore confidence in the private sector.
But with the latest official figures showing that the UK is only just out of recession, he warned that if Government support was withdrawn too quickly there was a danger it would "tip back".
While he emphasised that the £178 billion deficit in the public finances had to be tackled, it was essential that growth was firmly established first.
"Once you get through to that, once you get recovery, then of course you have got to get your borrowing down," he told the BBC Radio 4 Today programme.
"Pull that support prematurely, the Tory prescription if you like, and you run the risk that you will actually tip things back, which is why I attach so much importance to getting our economy back to recovery."
Mr Darling said that he believed that he had struck the right balance and he accused the Conservatives of "running away" from earlier pledges to tackle the deficit more quickly as they realised the dangers.
"I am confident about this, but also I shall remain cautious, but I think the path we have set ourselves, which is similar to most other advanced countries in the world, will see ourselves through to recovery and you will start to see the private sector growth coming in," he said.
