The Warden Vault Quest - Wisp area not accessible

One of the locks to the warden vault requires you to get a wisp.  Taash blew up the explosive container, but it did not lower the ladder to allow access to the area with the wisp. 

I've reloaded and blew up the barrel multiple times, but I still don't have access to the ladder.