If the linkage is hard to move without the servo, that could be the issue.

The steering should move freely.

Find where it is binding and then re-attach the servo and test again.