FFF
You are quite wise to withold payment until the work is finished to a satisfactory level. If the builder is not easy to speak to, you should send them a letter by recorded delivery, setting out what the builder needs to do to satisfy the contract between you. Advising that if they finish the work to a satisfactory quality, you will pay the oustanding amount due.
If the builder is a member of the Federation of Master Builders or other trade body, they could have a concilliation service which may be able to help.
If they are not a member of any trade body, your local Trading Standards office may be able to help. These days you have to contact Consumer Direct as some TS do not take calls direct. See the following link. http://www.consumerdirect.gov.uk/before_you_buy/thinking_about/home-improvements/ .
Huckster