he probably thinks your playing around! unfortunatley there is no other way too get him to stop but too say a stern "dont do that again please"; unfortunatley he will get the wrong idea and probably not talk but it seems like the idea
you could say youre still a great guy, but i dont like you...