The maximum potential difference a battery can supply is known as its electromotive force, or emf for short. The symbol for this is ε.
Consider a battery connected to a variable resistor of resistance R. An ideal battery with an emf of ε will supply a current given by I = ε/R. As R goes to zero, the current gets very large.
In the real world it's not quite this simple, because batteries have an internal resistance r that becomes increasingly important as the current supplied by the battery increases. This internal resistance comes from the fact that charge has to flow from one terminal to the other inside the battery to complete the circuit.
A real battery can be treated as an ideal battery with an emf of ε in series with the internal resistance r. The potential difference that appears across the terminals of the battery supplying a current I is:
VT = ε - Ir
If the battery is not connected to anything, I is zero and the voltage across the terminals is the battery emf. When current is being supplied, the voltage lost across the internal resistor is Ir. As I increases the larger that voltage is, and the less there is for the external circuit.