The ban, which officials hope will prevent problems caused by second-hand smoke, adds to the city's 2003 ban on cigarettes in bars and restaurants.
The new law will not be enforced by police but by some 200 parks personnel who watch over the city's 29,000 acres of park land and beaches. Violators face a $50 fine but officials say the ban is meant to be largely self-enforcing.
"We don't think that people should be exposed to those chemicals when they go to a park to enjoy the fresh air," city Health Commissioner Thomas Farley told Reuters.
Saleswoman Polonia Jourdain, sitting on a park bench with her 8-month-old nephew, said she was happy with the ban.
"I don't want to smell smoke wherever I go," said Jourdain, 17. "The smell of cigarettes makes me nauseous and gives me headaches."
New York's City Council voted in February to broaden the city's smoking prohibitions to cover its 1,700 parks, beaches, boardwalks and pedestrian plazas, such as Times Square.
New York follows Chicago and Los Angeles, which have already enacted similar bans.
New Yorkers still can smoke on sidewalks, parking lots, streets and in their homes, although many landlords of rental properties do not allow it.
Not everyone was so pleased by the new law. The ban represents "an extreme revocation of civil liberties rather than telling people to just walk away," said Audrey Silk, a member of New York Citizens Lobbying Against Smoker Harassment.
"If you held up the evidence up to sunlight, it would disintegrate," she said.
Some New Yorkers said they were confused by the limits.
"Can you smoke on the sidewalk next to a park?" asked attorney Alex Roberts, 36, smoking a cigarette in violation of the ban.
New York health officials say cigarettes kill more than 7,000 residents a year. In 2002, 21 percent of New York adults smoked and the percentage is now down to 15.8 percent, Farley said.
Bloomberg has promoted health measures including a ban on trans fats in restaurant food and a requirement that chain restaurants display calorie counts on menus.
He has campaigned nationally for food companies to cut salt levels in their products and for the federal government to ban the purchase of sugary drinks with food stamps -- federal vouchers used by 42 million low-income Americans to buy food.