A BNPB official said in a news conference late on Sunday that 14 people had been killed, nine of whom have been identified, and 56 people suffered injuries, most of them burns.
Around 1,300 people have been evacuated, while nine people remain unaccounted for, BNPB said.
Taufiq Ismail Marzuqi, a resident in the district of Lumajang who had volunteered to help, told Reuters that rescue efforts were "very dire" because of the severed bridge and volunteers lacking experience.
In a video he recorded, police and military officials tried to excavate bodies with their bare hands.
Rescuers in the village of Curah Kobokan, also in Lumajang, found the body of a mother still holding her dead baby, the state news agency Antara reported.
A Reuters witness in the Sumberwuluh area said homes and vehicles were almost completely submerged by thick, grey ash, fallen trees blocked roads and a cow which villagers had been unable to rescue lay by the roadside, the witness said.
Hosniya, a 31-year-old local resident who was evacuated with her family, told Reuters that the eruption was very sudden.
"At first, I thought it was a bomb explosive...suddenly it was all dark, like it was going to destroy the earth," she said.
Hosniya and her family fled, unable to take anything with them other than their official papers.
Heavy rain is expected for the next three days, which could complicate evacuation efforts, a meteorological agency official said late on Sunday. Rock debris and hot volcanic sediment were already limiting movement, local rescuers said.
BNPB will rebuild the wrecked homes, and heavy equipment, including excavators and bulldozers, is being deployed, its chief said.
The agency also said that 10 people trapped in sand mines by the eruption had been evacuated to safety.
Semeru, which according to volcano experts has been in an eruptive phase since 2014, had started emitting hot clouds and lava flows recently, prompting the authorities to issue warnings for people not to go near it from Wednesday.
Indonesia's transportation ministry said on Sunday the eruption had not caused any disruption to flights, though pilots have been alerted to watch out for the ashfall.
Semeru, more than 3,600 metres (12,000 feet) high, is one of Indonesia's nearly 130 active volcanoes.
Indonesia straddles the "Pacific Ring of Fire", a highly seismically active zone, where different plates on the earth’s crust meet and create a large number of earthquakes and volcanoes.
While many Indonesian volcanoes show high levels of continued activity, eruptions can be years apart. In 2010, an eruption of the Merapi volcano on Java island killed over 350 people and displaced 400,000.
(Additional reporting by Stanley Widianto and Nilufar Rizki in Jakarta and Nur-Azna Sanusi in Singapore; Editing by Ed Davies, Kim Coghill, Gerry Doyle and Raissa Kasolowsky) ((Widianto.Stanley@thomsonreuters.com;))