A major limitation for wireless video communication on portable devices is the limited energy budget. For this reason, efficient usage of the scarce energy becomes a critical design constraint, in addition to meeting the Quality of Service constraints related to the video quality. In this chapter the authors focus on minimizing the energy cost of the two main energy consumers in the handheld wireless video device: the video encoding and wireless communication tasks. For this purpose, they present a cross-layer approach that explores the tradeoff between coding and communication energies. They then exploit the Rate-Distortion-Complexity tradeoffs and flexibility of the Scalable Video Codec. The results show that by adapting the codec configuration at runtime to the specific scenarios up to 50% of the total energy can be saved with marginal video quality loss. Moreover, the approach presented is of low complexity and easily deployable in practical systems.