You have to comlete the quest for the Celestial order. talk to the guy in the main building and ask him about celestia or whatever her name is and he will go and open her room for you. I did this after I beat and saved the priest witht he candles. Do not have the game in fron of me but you need to do all this first.

After you get the horn of caldalar or wahteever his name is, you will be sent to the mage he will then give you a letter to take to the mayor guy. You do this and he will ask you to find out if there is a spy in the castle. After that you will be able to tell him what he needs to know and he will then give you a letter to present to the guards to get out of the town.