I wouldnt say its a rule but if you watched the episode where Peter meets that invisible guy and he makes a breakthrough after being thrown off the building, Peter says that he doesnt have to forget about the people he cares but instead remember them.
And so he probably can only think of one...