Cyber security has become increasingly challenging due to the proliferation of the Internet of things (IoT), where a massive number of tiny, smart devices push trillion bytes of data to the Internet. However, these devices possess various security flaws resulting from the lack of defense mechanisms and hardware security support, therefore making them vulnerable to cyber attacks. In addition, IoT gateways provide very limited security features to detect such threats, especially the absence of intrusion detection methods powered by deep learning. Indeed, deep learning models require high computational power that exceeds the capacity of these gateways. In this paper, we introduce Realguard, an DNN-based network intrusion detection system (NIDS) directly operated on local gateways to protect IoT devices within the network. The superiority of our proposal is that it can accurately detect multiple cyber attacks in real time with a small computational footprint. This is achieved by a lightweight feature extraction mechanism and an efficient attack detection model powered by deep neural networks. Our evaluations on practical datasets indicate that Realguard could detect ten types of attacks (e.g., port scan, Botnet, and FTP-Patator) in real time with an average accuracy of 99.57%, whereas the best of our competitors is 98.85%. Furthermore, our proposal effectively operates on resource-constraint gateways (Raspberry PI) at a high packet processing rate reported about 10.600 packets per second.