Reports show the hospitals admitted to incinerating nearly 16,000 miscarried and aborted fetal remains in their trash-to-energy plants.
The plants help supply heat and power to the facilities.
The BBC reports ten hospitals say they followed the procedures but they have now discontinued the policy.
Even more shocking, some parents were simply told their babies were cremated.
Two of Britain's biggest hospitals, Addenbrooke's in Cambridge and Ipswich Hospital estimated they had incinerated nearly 2,000 at their energy producing facilities.
Health minister Dr. Dan Poulter called the actions "totally unacceptable.
He went on to say, "while the vast majority of hospitals are acting in the appropriate way, that must be the case for all hospitals and the Human Tissue Authority has now been asked to ensure that it acts on this issue without delay."
One mother told The Daily Mail she knew what was happening. The woman had lost her baby at eight weeks and when she "asked one of the nurses what would happen to my baby, she just said, 'Well, it will be incinerated with the rest of the day's waste."
She continued "that was really difficult to hear because to me it wasn't waste, it was my baby.'