DENVER -- Residents of an apartment complex that caught fire and was deemed uninhabitable will be allowed back to their homes for 48 hours to remove their belongings, officials said.

Only the residents of the first three floors will be allowed inside the complex, said Chris Dann of the Colorado Health Department.

Tenants will be able to retrieve any of their belongings they can salvage. Most of the complex was damaged in a fire that appears to have started accidentally.

Residents who lived on the fourth floor will not be allowed back inside because that floor tested positive for asbestos.

Dann said residents will only have a brief period of time to remove their items because it is only a matter of time before the asbestos on the fourth floor begins to move down and contaminate the other floors.

Residents removing their items were told to be careful while removing items from the walls because it could cause the walls and ceilings to crumble.

Dann said the water damage to the building is so severe, he did not think any residents would be allowed to move back in.

"The only things I have are the clothes on my back," said Kevin Henry told 7NEWS after the fire. Henry lived on the third floor.

"Asbestos gets into the fibers of things and can be pretty timely and expensive to clean," said Henry.

Investigators are not releasing an official cause of the fire, but they say it appears the blaze was started accidentally. They have ruled out the possibility of fireworks or any kind of barbecue grill, firefighters said.

Witnesses said the fire started on a balcony, but quickly spread.

"From the front, it was smoking," said Peter Jentsch who shot video of the fire. "But from the back, you could see the flames from multiple areas of the house."