While using the PS4, press and maintain pressure on the Playstation button located on the controller. Adjust Sound and Devices may be found towards the bottom of the menu. You'll need to modify the option to Chat Audio once you've selected Output to Headphones.