A stylus is changed usually by pulling straight out and in the opposite direction of the turntable's arm. With the other hand, using a pair of tweezers, grasp the stylus from both sides and pull. Don't force it.
eBay