Blocked quest

I was on the Destroy the evil tomes quest and got to the abandoned thaig or recently opened passage and i cleared the area but had merrill in my party so i left and came back without her so i didnt gain rivalry with her and when i tried to go back to the room with the golem the door is locked and it cant be unlocked by rogues in group.