Once the seller has accepted your offer (acceptance), then even though you have only handed over a holding deposit, the seller has nevertheless agreed to supply you with the goods at the price agreed, this now forms a contract to supply on those terms.
This is all well and good in theory; but in practice forcing the retailer to supply on those terms stipulated or trying to sue for a, 'loss of bargain' would be time consuming and probably more hassle than it was worth. If they are offering you a full unconditional cancellation, I would be inclined to just take the refund and accept that a genuine mistake may have occurred, then shop elsewhere for similar goods at the best possible price with a competent retailer.