Chris
No the outside handle is not removeable
The set screw in the bushing just stops the handle from pulling through when you close and latch the door.
Mike.