My brother is an electrical engineer; he's working on unmanned aircraft autopilots, among other things. He recommended using a PID controller. “PID” is an acronym for proportional, integral, derivative, the three terms that make up the control equation. The controller measures the error between current and desired position. It also maintains an integral of the error, and estimates the current derivative of the error. Then it linearly combines these three values to drive something intended to correct for the error.
I tried out a PD controller (leaving out the integral term). The error is the difference between the target heading and the current heading. The derivative of the error is then the current angular velocity. These are combined to drive the nozzle angle, which is clamped to stay within its movement range. It's an extremely simple controller but the results are much smoother than what I had before:
In the video I'm aiming the green arrow and manually firing the rocket. The rocket rotates freely in between firings. The steering algorithm is applied to the nozzle even when I'm not firing so you'll see it rotating in preparation to align the rocket with the arrow.
In terms of control I'm pretty happy with this. I may add some additional rotation jets to the rocket to allow rotation without adding large linear velocities; they would probably be controlled with the same algorithm.
Oh yeah: this PD controller is pretty much the same thing as a damped spring. The nozzle angle doesn't translate linearly into rotational acceleration but otherwise it is the same thing.