Chancellor Rishi Sunak has denied "ignoring" fraudulently claimed Covid support funds worth billions of pounds.
Mr Sunak said the UK government would "do everything we can" to recover loans stolen by fraudsters.
On Monday, a Conservative minister resigned over the government's "woeful" efforts to stop abuse of pandemic support schemes.
The government last week disputed reports it had written off £4.3bn of loans.
Announcing his resignation in the House of Lords, Treasury minister Lord Agnew accused the Treasury of having "little interest in the consequences of fraud to our society" and of making "schoolboy errors" over the awarding of loans.
In its annual report, HM Revenue and Customs (HMRC) said funds had been lost to fraud, including 2.5% of the £19.7bn delivered through self-employed support.
Last year, the National Audit Office criticised the government for failing to implement measures to prevent people exploiting Covid schemes.
Labour have demanded an investigation.
At Prime Minister's Questions, Labour MP Kate Osamor asked if the prime minister had agreed to the chancellor "writing off £4.3 billion of fraud".
"That's £154 for every household in the country that went directly into the pockets of fraudsters," she added.
In response, Prime Minister Boris Johnson said: "No, of course not. We do not support fraudsters or those who steal from the public purse."
The prime minister's official spokesman later clarified that Mr Johnson meant he did not condone fraud.
Later, in a Twitter thread, Mr Sunak said people were "absolutely right to be" concerned about fraudulently obtained funds.
"No, I'm not ignoring it, and I'm definitely not 'writing it off'," Mr Sunak wrote.
'Needed support quickly'
He said the government had invested more than £100m in an anti-fraud taskforce with 1,265 staff. He said 3,000 one-to-one enquiries were opened in 2020/21 and HMRC wrote to 75,000 people.
Last year, the government stopped or recovered nearly £2.2bn in potential fraud from Bounce Back Loans and £743m of overclaimed furlough grants, Mr Sunak said.
He said businesses were "on the brink of collapse" and "needed support quickly" when assistance schemes were rolled out.
In recent days, Mr Sunak has come under increasing pressure to claw back Covid support lost to fraud.
Labour's shadow chancellor Rachel Reeves said the scale of Covid funds lost to fraud should be a "source of enduring shame to the Chancellor".
The Liberal Democrats said the government had "allowed fraudsters to steal billions of taxpayers' money" and urged the chancellor to explain how he would get the money back.