And Some GOP Members Want Him Out