Especially when you're on a street with 20 butterflies, it can be really annoying to follow up on harvesting - I've massaged a lot of butterflies without being able to milk them, mostly because I couldn't find the butterfly I'd massaged again. It might also be that they're jealous: you can't massage several and go back to milk them.
It would be really nice if animals that you'd petted or massaged were at least highlighted somehow - even if only in the "which butterfly do you want to interact with" popup...