Hate it when that happens.
I've done that before. You can unrival them on your profile.
Click on profile on the same drop down for yourself and then look under rivals.
Sometimes if you click down the list to fast you will miss the friend thing and add rival instead.