Go to your Party Chat screen by pressing the PS button. A variety of buttons may be seen on the left side. Toggle between "Current Chat Audio: Party" and "Current Chat Audio: Game" with a button cluster. Though you change this to game mode, it will appear as if you are not in a party.