Previous handle was a pimple on a 2 way screw (no idea what these are called or how it was still holding on) so replaced it with one on 4 bolts on washers for a better grip.
After taking it off it appears that the screw was “sealed” in some sort of wood filler.
Yeah the nuts are cutting into the wood. OP would have to do something different to broach.