I have a different problem. I got the quest after talking to him repeatedly (and the ! disappeared from the map)
And then I went over the bridge and fought Hivernal. At about half his LP, my party was dead and me, being the tank that I am, managed to take all his remaining life away. The problem is, he jumped into the middle of the tower and is just standing there doing nothing. The big red bar in the upper right corner is empty, and I can't even select him anymore. When I hit him, it doesn't show damage points only the words "poisoned" or "immune"