I have a online shop and I would like to start exporting to EU countries. I do not know the steps I have to follow for export legally.
There is here in the UK a website https://www.gov.uk/dispatching-your-goods this is an HM Revenue and Customs the heading of which is HM Revenue and Customs Dispatching Goods within the EU. Read the basic guide to trading abroad on the HMRC Website the main exclusions to this are goods subject to export licensing controls: (For example Military Goods or Class A drugs, and goods classed as excise products (alcohol, tobacco and hydrocarbon oils). There are countries where things do not need a license, so please check.
In France I would go to you local Tax Office and ask for their leaflets on the EU Single Market and they should be able to advise you further.