Scattered Glyphs

I must be missing something, or the quest is bugged. Am I supposed to be able to take the torch with me to find the other glyphs? 
If that is the case, then that won't work. I have gone back to the shrine multiple times, and ANY time I leave the shrine I do not carry the torch with me. How am I supposed to find the other glyphs? I go to where they are marked, hit 'v' do all the things, and nothing. ANy help would be appreciated!