It would be great fun to have temples hidden away in the caves or in some instances to have them standing at the end of a road etc. There could be quests to find things for the Giants and donate to special shrines inside the temples. Another idea would be that the temples are inside the minds of the giants. Just a thought