Starter problems

[tacks Y]

I have a JD hoe w/4276T engine. Same engine as my 440C. Starter will keep spinning some times after engine starts. If I tap solenoid on starter lightly it will drop out. So solenoid bad right? Well starter shop says bad key switch? He put new solenoid on starter before w/ bushings.  I have changed relay that powers the solenoid. So I put a test probe w/light on coil side of relay, power drops off w/ key as it should. I put it on the contact side of relay and when power stays on starter keeps spinning. This is maybe after 10 starts or so. So I try tapping relay no change, tap solenoid and it stops. The only time starter keeps spinning  I have power on contact side of relay. There is no way I would get power to back feed to that point right? All testing says key switch is fine, relay I changed but maybe bad? But why not dropping out when I tap it? But it does when I tap the starter.     Thanks for any and all ideas.

[Logger RK]

I have had that happen when my batteries are low. Solenoid gets spot wielded on. I have master switches on my Eq. Usually shutting battery off & on will break the contact. But somtimes I have to tap on it. Then I'll charge batteries up & it'll be good 

[tacks Y]

But why do I see power on the starter side of the relay?