The Rohingya refugees brought to a Bangladesh island fear they will be exposed to the elements as the monsoon season looms, a report by Human Rights Watch (HRW) said Monday, adding that they struggle with "inadequate" health and education facilities.

About 18,800 refugees have been moved from the Cox's Bazar region – where around 850,000 people live in squalid and cramped conditions after fleeing Myanmar – to the low-lying silt island of Bhashan Char in the Bay of Bengal.

A spokesperson for the prime minister's office told Agence France-Presse (AFP) that another 80,000 Rohingya would be shifted to the island soon.

The Bangladesh government has insisted the relocation is voluntary, with the island safe from cyclones and its facilities far better than the Cox's Bazar camps.

But HRW said, after conducting interviews with 167 refugees, that they were moved "without full, informed consent" and prevented from returning to the mainland.

The refugees also spoke of a shortage of health facilities and education for their children, the 58-page report said.

Others feared the monsoon season starting in June could expose them to high winds and flooding on the island.

HRW acknowledged that Dhaka had been "generous and compassionate" in sheltering the Rohingya – many of whom fled Myanmar after a brutal military crackdown in 2017 – but that their rights should be protected.

"The Bangladesh government is finding it hard to cope with over a million Rohingya refugees but forcing people to a remote island just creates new problems," said Bill Frelick, HRW's director of migrant and refugee rights.

"International donors should assist the Rohingya but also insist that Bangladesh return refugees who want to return to the mainland or if experts say island conditions are too dangerous or unsustainable."

Bangladeshi Foreign Minister A.K. Abdul Momen, hit back at the report, telling AFP that "whatever resources we have, we have tried to provide the best of services to these people."

"Their nations are very developed ... If they have so much sympathy, let them take them back to their country," Momen said of the New York-based HRW.

"I am sorry we cannot have better facilities because we are not a rich country. We cannot afford to provide more things."

On Friday, a boat filled with dozens of Rohingya landed off the Indonesian coast after a months-long journey – the latest in a wave of arrivals from the cramped camps in Bangladesh.

Bangladesh police have also arrested Rohingya trying to flee Bhashan Char, while thousands protested against conditions on the island last week during a visit by the U.N. refugee agency.