Owners on gmod servers that threaten you if you try to leave

It happened to me on multiple occasions when I have gone on small servers, I use a VPN whenever I play the game anyway. Most of the time they are empty threats but I have had a couple of times when they hit off the VPN.