The surprise move by the respected former armed forces chief comes after a series of disputes over the military's values and role in society between ministers in Netanyahu's government and top generals backed by Yaalon.
"I told the prime minister this morning that due to his conduct in recent developments, and in light of my lack of trust in him, I am resigning from the government and Knesset (parliament) and taking a break from political life," Yaalon said on Twitter.
Yaalon's resignation does not take effect for two days and, hours after it was announced, he used the podium of the defence ministry to deliver an impassioned tirade against the extremism that he said was gripping the Likud party and the country as a whole.
"Unfortunately extremist and dangerous elements have taken over Israel and the Likud and are threatening (society)," he said.
"This isn't the Likud I joined," he added, calling on the "sane majority" of Likud voters as well as the rest of the nation "to realise the severe implications of the extremist takeover of the centre, and fight this phenomenon."
Yaalon said he had worked harmoniously with Netanyahu in the past, but recently "found himself in serious dispute over professional and moral issues with the prime minister, a number of ministers and lawmakers."
Reports over the past few days indicate that Netanyahu intends to appoint former foreign minister Avigdor Lieberman to the post.
Lieberman, 57, is one of the country's most polarizing politicians. Over three decades, he has at times been Netanyahu's closest ally and other times a fierce rival. This week, Netanyahu invited Lieberman's party Yisrael Beitenu to join his narrow coalition.