Guidance on the export of firearms is published on the GOV.UK website. See: https://www.gov.uk/firearms-and-export-control-forms
This guidance is issued by the Export Control Organisation (ECO) within the Department for Business, Innovation and Skills (BIS).
You will have to apply for a licence (either an Open General Export Licence or a Standard Individual Export Licence) from ECO even if the items are historic military goods.
The most flexible licensing option is to register for the OGEL (Historic Military Goods) which allows - subject to certain conditions - the export of historic military goods from the UK or any other European Union (EU) member state, where the exporter is established in the UK, providing the goods are worth less than £35,000 and were manufactured before 1897.
You need to meet all the terms and conditions of the OGEL before registering to use this licence via SPIRE - https://www.spire.bis.gov.uk.
To check conditions before registering for the licence you can use ECO's OGEL Checker - http://www.ecochecker.bis.gov.uk
If you cannot meet all the conditions then you will have to apply for a SIEL from ECO via SPIRE.
You might also need to apply for a separate licence for cultural goods issued by the Arts Council (links are in the above guide).
ECO's helpline for further information is 020 7215 4594 or firstname.lastname@example.org