The resolution was drafted by the U.K. in September and adopted by the 15-member Council with 14 votes in favor. Venezuela abstained, saying the refugee crisis cannot be tackled by military action.
The resolution allows countries, acting nationally or through regional organizations, to inspect "vessels that they have reasonable grounds to believe have been, are being, or imminently will be used by organized criminal enterprises for migrant smuggling or human trafficking from Libya".
It authorizes member states to seize and dispose of the vessels if confirmed as being used for migrant smuggling or human trafficking.
Welcoming the adoption, Britain's UN Ambassador Matthew Rycroft termed the migrant and refugee crisis as one of the greatest challenges to the world community.
"These groups have no respect for human life. They exploit those taking the perilous journey across the Mediterranean. They overfill poorly maintained boats, ill-suited to such a difficult crossing and abandon their passengers at the first sign of trouble," he said.
Libya initially opposed the resolution, but the country's UN envoy, Ibrahim Dabbashi, wrote to the Security Council on Tuesday that the country no longer objected to it after "many substantive amendments has been made to satisfy all of Libya's concerns".
Europe is facing its biggest refugee crisis since the Second World War, with thousands of asylum seekers from Middle Eastern and African countries trying to flee war and persecution.
The UN refugee agency says nearly 530,000 migrants and refugees have crossed the Mediterranean Sea since January and about 3,000 people have died or gone missing during the perilous journey to find safety in the EU.
The majority of the refugees are Syrians fleeing a four-and-a-half year civil war that has claimed more than 250,000 lives and made the country the world's single-largest source of refugees and displaced people.