The decision, which could take up to 72 hours to implement, puts the 28-member military alliance in charge of operations to target Muammar Gaddafi's military infrastructure and protect civilians, as well as implementing a no-fly zone and an arms embargo.
"NATO allies have decided to take on the whole military operation in Libya under the United Nations Security Council resolution," said Secretary General Anders Fogh Rasmussen.
"Our goal is to protect civilians and civilian-populated areas under threat from the Gaddafi regime. NATO will implement all aspects of the U.N. resolution. Nothing more, nothing less."
The operations will be led by Canadian General Charles Bouchard, NATO said.
The decision had been delayed by disagreements between NATO members France and Turkey over political control of the mission, resulting in days of divisive discussion.
French President Nicholas Sarkozy had wanted to restrict NATO's responsibility to the military "machinery" needed to coordinate the air campaign, while political control remained in the hands of the members of the coalition.
Turkey instead wanted to be able to use its NATO veto to limit allied operations against Libya's infrastructure and avoid casualties among Muslim civilians.
Turkish Foreign Minister Ahmet Davutoglu said on Thursday Turkey's fears had been addressed and the command of military operations would be transferred completely to NATO. But it still took three more days of talks to reach an agreement.
NATO taking charge will encourage participation from more countries in the alliance which had been reluctant to join in the operation, a senior U.S. administration official said. NATO will also seek the participation of neighboring Arab countries.
"You will be able to rely on a great number of allies who up to this point, while wanting to participate in the operation were unwilling to do so until it came under NATO," the official told reporters on condition of anonymity.
U.S. NOW PART OF MUCH LARGER EFFORT
"So what we will see now is more countries participating, and that will allow the United States to be part of a much larger effort, rather than having to take the lead. That's why we wanted to hand it off in a matter of days, and we've now done that," the official said.
U.S. Defense Secretary Robert Gates said on U.S. television the United States would begin reducing the military forces it had committed to the Libya mission "beginning this week or within the next week or so."
NATO has said its no-fly zone in Libya, which it agreed to enforce on Thursday, would be "impartial," banning flights both by Gaddafi's forces and his opponents. The arms embargo would be enforced in a similar way, applying to both sides.
With anti-Gaddafi rebel forces apparently back on the advance, analysts said the Western-led military operation could get complicated if the rebels started to approach Tripoli.
Daniel Keohane of the European Institute for Security Studies said fighting there could result in large numbers of casualties, including an increased risk of civilian casualties from any air strikes.
"If rebel forces were seen to be seeking revenge on Gaddafi supporters, it could cause huge political problems for the alliance," he said, "because the U.N. mandate to protect civilians should apply across the board."
Keohane said the risk of causing civilian casualties in towns like Tripoli could lead to disputes in the NATO council as to what is a legitimate target. "It will be very difficult to determine who is who and when protecting civilians becomes taking sides," he said. "That will be the key question."
Asked whether rebel forces could become NATO targets if their operations threatened civilians, the senior U.S. administration official responded: "Right now all the threatening and the striking of civilians is being done by Gaddafi forces, and that's the focus. But our mission is clear, it's about protecting civilians."
Turkish Prime Minister Tayyip Erdogan and Davutoglu are both expected to attend a conference in London on Tuesday along with U.S. Secretary of State Hillary Clinton at which further aspects of the political leadership of the operation to end Gaddafi's 41-year rule will be discussed.