So after some testing , walking next to the car while going straight I noticed the car goes a lot like a boat almost all the time while sometimes it kept following a turning course while none of the wheels were moving left or right, mainly the back side of it...
So I think the 80% of my problem is because of this:

*let me wear my glasses*

The back tires have the inward toe so each wheel pushes to the center, so the back side in order to stay centered, drifts all the time like a boat. Now, when a wheel grabs better for some reason, forces the car to take that direction leaving the other back tire drifting because on such a light weight car is easy to drift. See the picture below. Kg = weight = better grip.



Now what confuses me is that almost all RC cars I've seen have the inward toe on the back wheels, so why isn't everybody having this problem? Is it because of the flat tires I am on? Should I 3D print the back suspension arms so the wheels don't have inward toe....?