> it's his group of followers and it somewhat taints his reputation unfortunately.
No, it's not "his" group of followers.
Jeremy Corbyn is opposed to what the Israeli government is doing to Palestinians.
That attracts support of other people with the same view - and Jeremy welcomes these people.
It may also attract the support of people with an "enemy of my enemy" perspective, which includes people who don't care what the Israeli government is doing, but opposes Israel purely because it's a Jewish nation - and Jeremy condemns those people.
I've challenged you to support your assertions with evidence: find statistics that Labour attracts more anti-Semites than other parties. The Tory-leaning YouGov found they've got the lowest rates amongst voters (despite the above), and there's no reason to expect this to be any different among the party membership.
By believing and repeating that it is a Labour/Corbyn problem without any such proof, you're spreading the propaganda and contributing to the damaged reputation, and in turn reducing the chance of replacing the Conservatives.