The U.K. was set to release thousands of prisoners on Tuesday as part of a government measure to manage its overcrowded British jails.
Recently released figures showed that the prison population in England and Wales was at its highest-ever level.
But with concern about released prisoners reoffending, the government insisted that no violent offenders or domestic abusers would be eligible for early release.
Business minister Jonathan Reynolds told Sky News that it was "a difficult decision" to release the 1,700 prisoners.
Downing Street has said the policy was required to avoid "unchecked criminality," where recently convicted criminals have been spared jail sentences because there were no places available.
Reynolds placed the blame on the last Conservative government, which was voted out in July, saying the decision "should have been made prior to the election" but that it "had given up on governing."
"Of all the scandals we inherited, I think the prison system, the justice system, is probably the worst of all," he added.
Chief Inspector of Probation Martin Jones told BBC radio that the pressure on prisons was "extreme."
"It's the highest population we've ever seen in this country. Prisons are absolutely close to 100% capacity.
"My understanding, as of yesterday there is a couple of 100 spaces remaining."
Jones warned that it was a "certainty that some will reoffend," highlighting that around a third of people released from prisons, each year go on to commit further offenses within a year.
Reynolds insisted that "no violent offenders, no domestic abusers will be eligible" for early release, adding that "we've put those greater protections in place."
The prison population has swollen in recent months by those convicted of taking part in anti-immigration riots across England.
The government has said those involved in the unrest would not be excluded from the early release plans.
