Sadly this happens.
I passed my driving test when I was 23. Insurance for the first year was £1800, 2nd year £900 and this year it was £450. It drops quite quick although the big drop was likely because I hit 25 rather than how long I have been driving.