The measure was not subject to Senate rules that require 60 of its 100 members to agree on most legislation, meaning it could pass with just a 51-vote simple majority. Two Senate Democrats - Joe Manchin of West Virginia and Jon Tester of Montana - joined 50 Republicans in voting for the bill.
The initiative underscores the intense political opposition facing administration efforts to combat COVID-19 at a time when health officials are trying to contain the highly contagious Omicron variant.
"I have always supported a vaccine mandate for federal employees and the military because maintaining essential services and military readiness the federal government provides is a matter of national importance," Manchin said in a statement following the vote.
"However, I do not support any government vaccine mandate for private businesses. It is not the place of the federal government to tell private business owners how to protect their employees from COVID-19 and operate their businesses," he said.
Manchin added he was fully vaccinated and had received a booster shot, and encouraged all Americans to do the same.
Republicans have said they were inundated with calls from businesses with up to 500 workers that are concerned about having to fire employees who oppose COVID-19 vaccines and testing.
"It's got Main Street America scared," Republican Senator Mike Braun, who was spearheading the legislative drive, said before the Senate vote.
The White House said in a statement on Tuesday that employers would face no burden from the mandate because the vast majority of American adults are fully vaccinated and noted that the rule exempts small businesses.
Biden's private business mandate has already been put on hold by a federal appeals court. A U.S. judge in Georgia also blocked a Biden vaccine mandate aimed at federal contractors on Tuesday.
"We are in the middle of a public health crisis. Everyone sees the damage it causes," Senate Majority Leader Chuck Schumer, the top Democrat, said in a speech. "The way to solve this is to be driven by science."
The United States has the world's highest daily average of new reported coronavirus infections. The virus has infected 49.5 million Americans and killed more than 794,000, the highest death toll of any country, according to the Reuters COVID-19 Tracker.
Republicans touted the measure's bipartisan support as a clear message of public concern about the mandate's potential impact on workers, businesses and the economy.
The chamber's top Republican, Mitch McConnell, branded the Biden mandate as an "illegal" and "absurd" government effort to micromanage the lives of private citizens.
(Reporting by David Morgan; Additional reporting by Richard Cowan, Trevor Hunnicutt, Eric Beech and Chris Gallagher; Editing by Sonya Hepinstall and Peter Cooney) ((firstname.lastname@example.org; +1 (202) 898 8326; twitter.com/dmorganreuters; Reuters Messaging: email@example.com))