Hungarian Prime Minister Viktor Orban suffered a blow Sunday in his referendum against the EU's migrant quota plan after he failed to rally enough votes to reach the required 50 percent turnout, according to the first polls from the ruling Fidesz party.

A whopping 95 percent of voters supported Orban's 'No' camp, but only 45 percent of the eight-million-strong electorate cast their ballots, making the vote invalid, according to figures released by the party's deputy chief Gergely Gulyas.

Gulyas nonetheless saw the result as "a sweeping victory for all those who reject the relocation plan, for those who believe that only nation states should remain, and for those who believe in democracy."

Orban had earlier downplayed the significance of the low turnout and said there would be "legal consequences" regardless, without giving further details.

His right-wing government led an expensive media offensive urging voters to spurn the EU's quota plan, which aims to share migrants around the 28 member states without the consent of national parliaments.

The firebrand leader has emerged as the standard-bearer of those opposed to German Chancellor Angela Merkel's "open-door" policy, in the wake of the bloc's worst migration crisis since World War II.

Polling stations opened at 0400 GMT and closed at 1700 GMT. Final results were expected later Sunday evening.

Opposition parties and rights groups had called on Hungarians to boycott the referendum or spoil their ballot.