You can upgrade the belt elevators by selecting another type of elevator and upgrade it like the belts.
So if you're upgrading from Mk. III to Mk. IV, you have to click each segment of the belts, the same goes for the elevator.
It would save you 4 clicks/key strokes at most, if both items are not on the hotbar. :)