I had the roll pin issue last race. The rear lock nut was worn and loosened up, the rim came off the pin just enough to strip out the slot on the back of the rim.
What your describing as an issue, is how my buggy was acting.
That may not be your problem, but it sounded familar.