Kicked out after one minute every time

I get this too... I try and hit my brown houses and collect before being kicked out, that way at least it's not much to lose if I don't make it, and they'll be ready to collect again by the time I'm actually done. I guess I've just learned how to deal with it. It has been happening less lately though.