Ground state solutions for Hamiltonian elliptic systems with super or asymptotically quadratic nonlinearity

This article concerns the Hamiltonian elliptic system: $$ \textstyle\begin{cases} -\Delta \varphi +V(x)\varphi =G_{\psi }(x,\varphi ,\psi ) & \mbox{in } \mathbb {R}^{N}, \\ -\Delta \psi +V(x)\psi =G_{\varphi }(x,\varphi ,\psi ) & \mbox{in } \mathbb {R}^{N}, \\ \varphi , \psi \in H^{1}(\mathbb {R}^{N}). \end{cases} $$ { − Δ φ + V ( x ) φ = G ψ ( x , φ , ψ ) in R N , − Δ ψ + V ( x ) ψ = G φ ( x , φ , ψ ) in R N , φ , ψ ∈ H 1 ( R N ) . Assuming that the potential V is periodic and 0 lies in a spectral gap of $\sigma (-\Delta +V)$ σ ( − Δ + V ) , least energy solution of the system is obtained for the super-quadratic case with a new technical condition, and the existence of ground state solutions of Nehari–Pankov type is established for the asymptotically quadratic case. The results obtained in the paper generalize and improve related ones in the literature.