Online Service Function Chain Deployment for Live-Streaming in Virtualized Content Delivery Networks: A Deep Reinforcement Learning Approach

Video delivery is exploiting 5G networks to enable higher server consolidation and deployment flexibility. Performance optimization is also a key target in such network systems. We present a multi-objective optimization framework for service function chain deployment in the particular context of Live-Streaming in virtualized content delivery networks using deep reinforcement learning. We use an Enhanced Exploration, Dense-reward mechanism over a Dueling Double Deep Q Network (E2-D4QN). Our model assumes to use network function virtualization at the container level. We carefully model processing times as a function of current resource utilization in data ingestion and streaming processes. We assess the performance of our algorithm under bounded network resource conditions to build a safe exploration strategy that enables the market entry of new bounded-budget vCDN players. Trace-driven simulations with real-world data reveal that our approach is the only one to adapt to the complexity of the particular context of Live-Video delivery concerning the state-of-art algorithms designed for general-case service function chain deployment. In particular, our simulation test revealed a substantial QoS/QoE performance improvement in terms of session acceptance ratio against the compared algorithms while keeping operational costs within proper bounds.

Deep Reinforcement Learning with Importance Sampling for QoE Enhancement in Edge-Driven Video Delivery Services

Adaptive bitrate (ABR) algorithms are used to adapt the video bitrate based on the network conditions to improve the overall video quality of experience (QoE). Further, with the rise of multi-access edge computing (MEC), a higher QoE can be guaranteed for video services by performing computations over the edge servers rather than the cloud servers. Recently, reinforcement learning (RL) and asynchronous advantage actor-critic (A3C) methods have been used to improve adaptive bit rate algorithms and they have been shown to enhance the overall QoE as compared to fixed-rule ABR algorithms. However, a common issue in the A3C methods is the lag between behavior policy and target policy. As a result, the behavior and the target policies are no longer synchronized with one another which results in suboptimal updates. In this work, we present the deep reinforcement learning with an importance sampling based approach focused on edge-driven video delivery services to achieve an overall better user experience. We refer to our proposed approach as ALISA: Actor-Learner Architecture with Importance Sampling for efficient learning in ABR algorithms. ALISA incorporates importance sampling weights to give higher weightage to relevant experience to address the lag issues incurred in the existing A3C methods. We present the design and implementation of ALISA, and compare its performance to state-of-the-art video rate adaptation algorithms including vanilla A3C and other fixed-rule schedulers. Our results show that ALISA provides up to 25%-48% higher average QoE than vanilla A3C, whereas the gains are even higher when compared to fixed-rule schedulers.

Tele-Rehabilitation for People with Dementia during the COVID-19 Pandemic: A Case-Study from England

Introduction: The Promoting Activity, Independence and Stability in Early Dementia (PrAISED) is delivering an exercise programme for people with dementia. The Lincolnshire partnership National Health Service (NHS) foundation Trust successfully delivered PrAISED through a video-calling platform during the Coronavirus Disease 2019 (COVID-19) pandemic. Methods: This qualitative case-study aimed to identify participants that video delivery worked for, to highlight its benefits and its challenges. Interviews were conducted between May and August 2020 with five participants with dementia and their caregivers (n = 10), as well as five therapists from the Lincolnshire partnership NHS foundation Trust. The interviews were analysed through thematic analysis. Results: Video delivery worked best when participants had a supporting caregiver and when therapists showed enthusiasm and had an established rapport with the client. Benefits included time efficiency of sessions, enhancing participants’ motivation, caregivers’ dementia awareness, and therapists’ creativity. Limitations included users’ poor IT skills and resources. Discussion: The COVID-19 pandemic required innovative ways of delivering rehabilitation. This study supports that people with dementia can use tele-rehabilitation, but success is reliant on having a caregiver and an enthusiastic and known therapist.