The treatment of VAT within the EU depends on whether your business is VAT registered, and also whether your client is VAT registered in France.
If you are both VAT registered in your respective countries, you should obtain your client's VAT registration number, and quote it on your sales invoice. You then do not have to charge VAT on your sales price. He will declare the transaction in France & pay VAT at the French rate to his authorities. You should then declare the transaction on your VAT return, and on an ESL (EC Sales List).
If your client is not VAT registered, and cannot prove that they are a commercial business, you should charge them UK VAT, and declare it in your VAT return.
If you are not VAT registered, you would not charge VAT anyway, but if you are thinking of doing more exports, you might want to consider registering for VAT.
If you are supplying goods, rather than services you may also need to declare all EU transactions to HMRC, using a procedure called Intrastat. For details of both, go to the HMRC website.