Environment Secretary Therese Coffey has admitted she is "pretty fed up" with the UK's water firms after they were forced to apologise over sewage spills.
Ms Coffey said it was not "acceptable" that there were 300,000 sewage spill events in England last year.
Her words come after water and sewage firms in England issued a public apology for "not acting quickly enough" on spills and vowed to spend £10bn to clean up their act.
Industry body Water UK said campaigners were "right to be upset about the current quality of our rivers and beaches" as it announced the package of investment on Thursday - which it claimed would be "the biggest modernisation of sewers since the Victorian era".
Asked on the Sophy Ridge on Sunday programme whether she was "angry" at the water firms, the cabinet minister replied: "I would say I'm pretty fed up with the water companies and we've seen an apology this week which is the right thing for them to have done.
"I think what's important is to make sure that we have a plan and we set out a plan for water to try and tackle these issues."
Untreated sewage was pumped into England's rivers and seas at least 301,091 times last year - an average of 824 a day - according to Environment Agency (EA) data.
That represented a fall of almost a fifth on 2021's 372,533 spills, although the EA said that had been "largely down to dry weather, not water company action".
Despite the announcement of £10bn investment, there has been public anger after it emerged it would be paid for by customers through "modest increases to their bills".
Ruth Kelly, chair of Water UK, told Sky News this week that water firms will provide a "huge multi-billion down-payment" to start "the biggest transformation project since Victorian times".
She added: "The way the system works is that over the lifetime of the assets, customers do pay that money back in modest increases in their bills."
Asked by Ridge whether bill payers would be "picking up the tab", Ms Coffey replied: "Through different ways it's going to be a combination - of course penalties and fines are paid for by the company, not by the bill payer, but in terms of general payments, I think you're right to say that a lot of this investment gets repaid through by bills and a small amount of return."
Ms Coffey went on to claim that over £30bn was paid out in dividends to shareholders when Labour was in power from 1997 to 2010, adding: "Going forward the dividends will be significantly lower than what had happened in the past."
Ridge interjected: "Should there be dividends at all? Should they be making profits and making money at times when bill payers are the ones here who are having to pay for what has happened before?
"Well one of the things that Ofwat (the water regulator) does is to make sure that any investments are deemed to be good value for money.
"One of the things is that penalties can end up with reimbursements to customers, but I think it's critical to say that we are getting to grips with a situation - we unveiled that surge of sewage, and that's why our plans I think will be effective in getting these solutions fixed."
Water UK said the £10bn comes in addition to a previous commitment to invest £3.1bn, and will be spent this decade.
This will pay for measures including enlarging and improving pipes and installing the equivalent of thousands of Olympic-sized swimming pools underground to hold surges in rainwater that would otherwise overload the system.
The package aims to cut sewage overflows by up to 140,000 each year, compared with 2020 levels.