You should not pay cash in these circumstances. If they do fit the windows next week, you should ask for a written guarantee after they'd fitted the windows and I would also ask them to wait until you get the work certificated before you pay the full balance (you may ask to pay half now, half later). If you are certain that the company has gone bust, you may consider paying the workmen for fitting the windows or you may write a cheque payable to the company.
The guarantee isn't worth that much whether the company is bust or not but you should certainly make sure that the job has been done properly before paying everything - even if this means waiting for the next rain - but it is likely that you will need to get the work certificated by your council anyway.