It doesn't make a lot of difference. If you leave one on all night (as security, perhaps) it will use about 1p worth of electricity.
Some bulbs take a minute or two to reach full brightness. This can be nice if using the bathroom with tired eyes in the middle of the night or it can be a nuisance if you are in a rush to find something. Other energy saving bulbs warm up quickly. In my view, you should leave them on to suit your needs rather than "wasting" a minute of valuable time to save a tiny fraction of a penny (I would guess that the extra carbon dioxide emitted by worrying will be more than would be used by the bulb).
Remember to look out for energy saving bulbs at supermarkets and other promotions. I've bought some for 10p each at Tescos and others at 3 for £1 at Morrisons. They are often sponsored by power companies who get a pound or two of carbon credits for each bulb sold because of the fuel saved each time we buy one and put it in a cupboard.