Potential energy is energy associated with position. Usually we view potential energy as energy that is stored temporarily.

One type of potential energy is gravitational potential energy. When g is constant this is defined as:

U_g = mgy

When g is constant you are free to set the zero for potential energy. Define the zero at some convenient level, and measure all vertical distances and potential energies from there.

The critical number is not the value of the potential energy, it's the change in potential energy. ΔU is the same no matter where you define the zero.