A friend was randomly removed from my friends list but i think im still on his

This isn't exactly a game problem, more of a system thing; if the following doesn't help you, you should try contacting Xbox Support or something.

The way an Xbox One friend request works is, if you send someone a friend request, they will have the option to add you as a friend or ignore it. Upon them ignoring it, this will make them your friend, but you will only be a follower to them. If you're already on his friends list, and you send a friend request, there is nothing for them to accept because they're already on yours.

Unfortunately, this doesn't explain what you're experiencing as you are able to be in voice chat with people who are not your friends, unless you have some sort of privacy/parental control settings activated on your account.

In order to check these Privacy and Online settings:

- Go to Settings

- Select 'Privacy & Online safety'

- Select 'View details & customize'

The setting you're looking for is 'Others can communicate with voice and text'. If this is still set to everybody, I'm not sure what the problem could be.

http://support.xbox.com/en-US/contact-us