I have a problem with backdoor =/

I think people are just leaving your server my guy.
Not well versed on Gmod but I would think a backdoor will just have people join another server.