The free form ladders are pretty nice, once you get used to running and jumping from one to another around a factory. They are on the sides of the stackable conveyor stands if you haven't noticed. I can successfully navigate down a lookout tower ladder 65% of the time. another 10% I can grab the ladder while falling and make it down safely without damage.
TIPS: If I'm just starting a new factory location, I put a jelly landing pad at the base so I can quickly jump down and avoid that last 25% of the times I would normally take fall damage. Until you have that luxury, pay attention to your hand animations while attempting to go down the ladder, if I recall correctly, they change when you "use" it.
DEVS: As a few people have mentioned, I think an actionable hit box at the top would be appropriate, but instead of sending you down a pole, maybe just get you started going down the ladder. Drop the player down the ladder a foot or two so they know it started working and they can climb the rest of the way down or jump off anywhere along the way as it currently works, also convenient for fast navigation.