Japan passed a controversial anti-terror law on Thursday that sparked street protests and warnings from critics that it would stomp on citizens' privacy rights and lead to over-the-top police surveillance.

The upper house of parliament passed the conspiracy bill early Thursday morning after a full night of debate by sleepy parliamentarians and unsuccessful efforts on the part of Japan's weak opposition to block it.

Thousands of demonstrators protested outside the legislature over the bill which criminalises the planning of serious crimes.

The government argues it is necessary to prevent terrorism ahead of the 2020 Olympic Games in Japan's capital.

Recent polls however show that the public is divided on the issue.

"We will uphold the law in an appropriate and effective way to protect people's lives," Prime Minister Shinzō Abe told reporters after the legislation passed.

The bill was revised several times over the years as earlier versions met with fierce resistance and never made it through parliament.

The latest bill reduced the number of targeted crimes to over 270 offences and narrowed the definition of terrorist and criminal organisations.

Earlier versions of the law targeted more than 600 crimes unrelated to terrorism or crime syndicates.

Critics argued, however, that the current law still gives police and investigators too much leeway.

The general public could be targeted on conspiracy charges via monitoring phone and online conversations once they are suspected of being a member of criminal group, they warned.