There isn't a way to see that. I would recommend picking a giant whose name seems unfamiliar to you and checking your favor with them. Then donate until you get 23 favor points. Your quest may or may not complete at that point. If it doesn't, well, at least you know who the other giant was! Then you can move on and donate to another one. I'd say you're doing pretty well knowing 3 of the giants!
I am having the same issue. I got the quest and started donating all over the place, without keeping track of the giants that I gained 23 favor from. :-(
Now I have to go to all the shrines and figure it out. I'm writing it down this time! :-)
I wish there were a list that would keep track of this for you, but then again, it does add to the game. I like having to work for the rewards.
Try the greasemonkey script (search this forum for greasemonkey) that tells you the favor you get from each giant. It seems to take some time to update it, but then you can see the favor you've generated on the leaderboards page.
I made this mistake before reset and ended up donating more than I needed. After reset I made a list. Now I have a Win7 Sticky Note up with all the things I need to remember (e.g. streets I've visited until I get the Completist badge), how many of each resource I need for a particular cooking quest, what Giant am I trying to get an emblem from).