Without knowing your circumstances it is hard to say but the chances are that yes you probably would save money by moving elsewhere. Lenders do not reward loyalty and so don't offer them any.
If you switch to another provider then the chances are that they can offer something that will be cheaper than you are paying right now. For example, Santander SVR is 4.24% and for a mortgage of £60,000 overt 17 years this will equate to a repayment each month of around £411 per month but if you switched to another lender and were able to secure a better rate of maybe 2.80% fixed but over 14 years instead then your repayments would then become £430 per month. I know this is a rise of £20 per month but the effect of reducing your term by 3 years on this particular loan would mean that you could probably save around £4,100 - £7,300 in interest repayments over the term.
When considering any mortgage product for my clients I look at just a few key rules to narrow the market.
Is the mortgage under £50,000? - If yes then look for fee free products from the lenders. Accept no lender arrangement fees. Go for a fee free product. It will have a higher interest rate but this will be more cost effective than adding a £1,000 (2%) fee to a small mortgage which may take 6 months just to pay this off alone
If the mortgage is over £50,000 then a fee may prove to be more effective in keeping the cost of the mortgage down but don't go crazy just to get a good rate. Check the fees on the product and if it is more than 1% of the mortgage balance then I would be looking for another deal and I personally do not often recommend any mortgage product with more than a £999 fee attached. I have yet to see a mortgage that charged much more than this offer something that would be substantially more beneficial than a £999 fee product unless you have a mortgage of more than £500,000.
Lastly, if remortgaging and switching lender then always look to see if they have a free legal and free valuation package. Most lenders will have this option and it generally does not cost anything more on the interest rate to get it. A typical valuation for mortgage purposes on a £200,000 house would cost around £300 and the legal work on a standard mortgage could be anywhere from £400 - £900. Most lenders will offer this for free and if they don't then you should be looking for a new lender.
Never trawl the high street walking into different banks, they rarely offer advice anymore (costs them too much to do this, Barclays only made £989 million in pre-tax profit last year) and when you are offered products it is on an information only/ non advised basis which means you have no accountability with the lender. Always use a mortgage adviser or financial adviser for exactly the opposite reason as listed above. You have accountability, they know the whole market, they know what lenders will accept, they probably know who and which product will be best suited after only 2-3 minutes of talking to you. FA's and MA's do generally charge a fee now but a report out 2 years ago showed that use of a 3rd party professional saves up to £4,000 on the average mortgage over the term.
Last thing I would look at if using a broker is to ensure that their fees are fair, and not misleading. Make sure you are fully aware of any and ALL fees before you agree to anything with a broker. My rule of thumb here is to never accept a broker that charges an upfront fee for his services. How do you know he will complete his work if he is already paid? Ensure the fee charged is fair. I am a mortgage broker myself and I charge between £195 and £395 per client per mortgage depending on the mortgage and how complicated the case may be. I think this is a fair price for the work involved. I do not see how some brokers can justify charging in excess of £1,000 for their work. Rule of thumb (again) Accept no fees more 0.5% of the loan value or no more than £500 from a broker. Remember that most brokers will also get paid by the lender too.
Hope that this hasn't been too long winded.