Accurate all-atom energy functions are crucial for successful high-resolution protein structure prediction. In this chapter, we review both physics-based force fields and knowledge-based potentials used in protein modeling. Because it is important to calculate the energy as accurately as possible given the limitations imposed by sampling convergence, different components of the energy, and force fields representing them to varying degrees of detail and complexity are discussed. Force fields using Cartesian as well as torsion angle representations of protein geometry are covered. Since solvent is important for protein energetics, different aqueous and membrane solvation models for protein simulations are also described. Finally, we summarize recent progress in protein structure refinement using new force fields.