This paper investigates total-factor energy efficiency and analyses the trends of the efficiency changes in China's agricultural production across 30 provinces and three regions from 2001 to 2011, based on data envelopment analysis (DEA) approach. The potential amount of energy savings and five potential factors for energy efficiency improvement are also empirically studied by Tobit regression model. The findings show that (1) total-factor energy efficiency in China's agricultural sector is increasing over years but performs heterogeneously across regions; (2) agriculture intensive regions and energy abundant provinces tend to be relatively energy inefficient in agricultural production; and (3) economic structure, agricultural production structure, technological progress and income effect are major potentials for improving energy efficiency, whereas energy price is not a significant factor. This phenomenon results from the divergence of economic development, endowment effects as well as the scale of agricultural production. Policy implications drawn from this research are to upgrade industrial structure and promote agricultural transformation to enhance farmers' income as well as to establish a land market with entitling land property rights to farmers. This conclusion can assist to form more scientific rural energy policy decision-making in China and also can be extended to other developing economies for sustainable agriculture.