Should i keep hooking up with my guy friend?

I met this guy a few years ago in uni and we got along so well that we became best friends,, last year i ended up sleeping at his house and we felt this TENSION. A few weeks later we went out clubbing, had a bit too much to drink and ended up hooking up. So now we are seeing each other in the down low just as "friends with benefits" and none of our friends know. The thing is i met some of his friends over the weekend and they said "OH you are X girlfriend" and im left confused because WHAT HAS HE BEEN SAYING ABOUT ME? i thought it was supposed to be a secret and that he didnt want anything else with me.. I do not wanna be his girlfriend so what should i do?