Do you remember doing anything with emblems from any other shrines or anything that seemed off that might've triggered this? What's the last thing you remember doing before you noticed your favor had dropped to 0?
I may have donated to earn more favor, enough to earn a new emblem, before first picking up an already earned emblem that was waiting to be collected. But I don't think it was at the shrine of Pot.