Does a battery lose mass according to the formula e=mc^2 when it is used?

Answer

Strictly speaking, yes.

The energy content of a battery is of the order of magnitude GJ/m³. A standard battery contains something like a few thousand joules. According to E=mc² corresponds to this a mass of 10^-14 kilograms.

This difference in mass is mainly due to a different binding energy between the molecules in the battery.