To me its just a hobby. I only race at the club level for now. For those who travel to national events and race becuase their job depends on it, then you could call it a sport.
As for those who say there is no physical endurance needed, have you ever had to chase down a run away in a large lot ? That can be pretty physical.