Assessment of QoE for Video and Audio in WebRTC Applications Using Full-Reference Models

WebRTC is a set of standard technologies that allows exchanging video and audio in real time on the Web. As with other media-related applications, the user-perceived audiovisual quality can be estimated using Quality of Experience (QoE) measurements. This paper analyses the behavior of different objective Full-Reference (FR) models for video and audio in WebRTC applications. FR models calculate the video and audio quality by comparing some original media reference with the degraded signal. To compute these models, we have created an open-source benchmark in which different types of reference media inputs are sent browser to browser while simulating different kinds of network conditions in terms of packet loss and jitter. Our benchmark provides recording capabilities of the impairment WebRTC streams. Then, we use different existing FR metrics for video (VMAF, VIFp, SSIM, MS-SSIM, PSNR, PSNR-HVS, and PSNR-HVS-M) and audio (PESQ, ViSQOL, and POLQA) recordings together with their references. Moreover, we use the same recordings to carry out a subjective analysis in which real users rate the video and audio quality using a Mean Opinion Score (MOS). Finally, we calculate the correlations between the objective and subjective results to find the objective models that better correspond with the subjective outcome, which is considered the ground truth QoE. We find that some of the studied objective models, such as VMAF, VIFp, and POLQA, show a strong correlation with the subjective results in packet loss scenarios.

Download Full-text

Evaluation of Video Quality in Wireless Multimedia Sensor Networks

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v6i1.pp223-234 ◽

2016 ◽

Vol 6 (1) ◽

pp. 223

Author(s):

Mustafa Shakir ◽

Obaid Ur Rehman ◽

Zeeshan Abbas ◽

Abdullah Masood ◽

Wajeeha Shahid

Keyword(s):

Sensor Networks ◽

Signal To Noise Ratio ◽

Video Quality ◽

Discrete Event ◽

Wireless Multimedia ◽

Multimedia Sensor Networks ◽

Opinion Score ◽

Different Types ◽

Multimedia Transmissions

<p class="Default">Simulating wireless sensor networks; there implementation and evaluation, require the use of a discrete event simulator. Omnet++ is quite a powerful simulator which supports concise and easy modeling of wired as well as wireless sensors environment. Scenarios involving multimedia transmissions with characteristics of video quality control and evaluation must be computed on the basis of Quality of Experience which relies on user’s perception to maintain the video quality. For the multimedia growth and awareness of future WMSNs, it is quite necessary that the performance should be tested for different types of radio models. So varying the radio parameters may allow for the optimization and improvement of the video quality. In this paper we have provided a test bench for the easy evaluation and optimization of the performance of WMSNs using different radio models. The performance is evaluated based on the QoE metrics; i.e. PSNR(Peak Signal-to-Noise ratio) and MoS(Mean Opinion Score), which depend on user’s perception to maintain the video quality.</p>

Download Full-text

Analisis Perbandingan Performa VPN IPSec dan ZRTP pada VoIP

10.31227/osf.io/5sxd7 ◽

2018 ◽

Author(s):

muhammad iqbal ◽

Galih Yogi Noviantoro ◽

Amarudin

Keyword(s):

Quality Of Service ◽

Packet Loss ◽

Peer To Peer ◽

Mean Opinion Score ◽

Total Delay ◽

Delay Jitter ◽

Diffie Hellman ◽

Opinion Score

Protokol ZRTP berperan melakukan enkripsi data pada sistem komunikasi VoIP, serta adanya suatu teknologi jaringan komputer VPN IPsec dengan sistem melewatkan data dalam suatu ip private virtual atau sebagai tunnel untuk media transmisi data yang aman. Perbandingan performa pada penerapan kedua metode pengamanan data pada sistem komunikasi VoIP bertujuan untuk mengetahui protokol mana yang aman untuk diterapkan. Pada dasarnya ZRTP menggunakan pertukaran kunci Diffie-Hellman sebagai pertukaran kunci komunikasi antar client, yang mana kunci untuk komunikasi antar client tersebut menggunakan hash dari Diffie-Hellman dan dilakukan secara peer-to-peer melalui paket RTP VoIP. Sedangkan pada VPN IPsec terdapat protokol ESP (Encapsulating Security Payload) yang berfungsi sebagai enkripsi dan membuat tunnel transport RTP pada sistem komunikasi VoIP. Acuan standarisasi pengukuran kualitas VoIP yaitu rekomendasi dari ITU (International Telecomunication Union) yang merupakan standarisasi internasional untuk mengukur kualitas suara dalam komunikasi telephone. Pengujian dilakukan menggunakan variasi bandwidth 32, 64, 128, 256, 512, dan 1024 kbps. Parameter QoS (Quality of Service) yaitu delay, jitter, packet loss, dan throughput. Hasil pengujian QoS (Quality of Service) menunjukkan bahwa protokol VPN IPsec menghasilkan total delay rata-rata yang cukup rendah sebesar 69,78 % dibandingkan dengan VoIP ZRTP, throughputrata-rata yang lebih tinggi sebesar 15,53 % tetapi nilai jitter cukup tinggi dibadingkan dengan ZRTP. Sedangkan packet loss pada sistem komunikasi VoIP ZRTP menghasilkan nilai packet loss yang stabil yaitu 0 %, sedangkan untuk VoIP VPN IPsec menghasilkan packet loss mencapai 4,62 %.. MOS (Mean Opinion Score) menunjukan bahwa hasil perhitungan MOS Score yang menyatakan bahwa pemakaian limit bandwidth 128 kbps dan 256 kbps performa VoIP ZRTP lebih unggul sebesar 11,21 % dan 9,63 % dari performa VoIP VPN IPsec. Akan tetapi dalam pemakaian limit bandwidth 512 dan 1024 kbps performa VoIP VPN IPsec mulai mengalami peningkatan dan lebih unggul 1,18 % dan 3,23 % dari performa VoIP ZRTP.

Download Full-text

Blending the Subjective and Objective Realms of Sacred Architecture at the Pantheon: Creating a Comparative Framework for Evaluating Transformative Experiences in Ritual Contexts

Religions ◽

10.3390/rel13010075 ◽

2022 ◽

Vol 13 (1) ◽

pp. 75

Author(s):

Brandon Richard Ro

Keyword(s):

Built Environments ◽

Subjective Experiences ◽

Subjective Analysis ◽

Sacred Architecture ◽

Transformative Experiences ◽

Comparative Framework ◽

Different Types ◽

Large Survey ◽

Architecture Analysis

This paper seeks to create a comparative framework for evaluating transformative experiences for different types of ritual contexts found in sacred architecture by bridging the gap between the phenomenology of human experience and architecture’s built conditions. The methodology creates a framework for statistical analysis, whereby evidence of people’s actual (i.e., real, lived) “subjective” experiences can be evaluated against the “objective” architectural conditions. The comparative framework is put to the test by comparing the experiential and environmental conditions found at the Pantheon in Rome. Experiential data for the Pantheon is extracted from Julio Bermudez’s large survey database (N = 2872) of “extraordinary architectural experiences” for this study. This data is compared against “objective” graphical architecture analysis using Lindsay Jones’ “morphology of ritual-architectural priorities” with a specific focus on ritual contexts. The quantitative and qualitative data reveals that the Pantheon produces transformative experiences for visitors that are related to the expected outcomes of specific design features. The percentages from the “objective” and “subjective” analysis both rank the priorities of theatre, contemplation, and sanctuary in the same order. This study concludes that built environments possessing a higher presence and quality of “ritual-architectural priorities” are more likely to be perceived as sacred and produce transformative experiences.

Download Full-text

Evaluation of Video Quality in Wireless Multimedia Sensor Networks

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v6i1.7573 ◽

2016 ◽

Vol 6 (1) ◽

pp. 223

Author(s):

Mustafa Shakir ◽

Obaid Ur Rehman ◽

Zeeshan Abbas ◽

Abdullah Masood ◽

Wajeeha Shahid

Keyword(s):

Sensor Networks ◽

Signal To Noise Ratio ◽

Video Quality ◽

Discrete Event ◽

Wireless Multimedia ◽

Multimedia Sensor Networks ◽

Opinion Score ◽

Different Types ◽

Multimedia Transmissions

Download Full-text

Subjective annotation for a frame interpolation benchmark using artefact amplification

Quality and User Experience ◽

10.1007/s41233-020-00037-y ◽

2020 ◽

Vol 5 (1) ◽

Author(s):

Hui Men ◽

Vlad Hosu ◽

Hanhe Lin ◽

Andrés Bruhn ◽

Dietmar Saupe

Keyword(s):

Image Quality ◽

Optical Flow ◽

Quality Assessment ◽

Image Quality Assessment ◽

Visual Quality ◽

Ground Truth ◽

Paired Comparisons ◽

Frame Interpolation ◽

Full Reference

Abstract Current benchmarks for optical flow algorithms evaluate the estimation either directly by comparing the predicted flow fields with the ground truth or indirectly by using the predicted flow fields for frame interpolation and then comparing the interpolated frames with the actual frames. In the latter case, objective quality measures such as the mean squared error are typically employed. However, it is well known that for image quality assessment, the actual quality experienced by the user cannot be fully deduced from such simple measures. Hence, we conducted a subjective quality assessment crowdscouring study for the interpolated frames provided by one of the optical flow benchmarks, the Middlebury benchmark. It contains interpolated frames from 155 methods applied to each of 8 contents. For this purpose, we collected forced-choice paired comparisons between interpolated images and corresponding ground truth. To increase the sensitivity of observers when judging minute difference in paired comparisons we introduced a new method to the field of full-reference quality assessment, called artefact amplification. From the crowdsourcing data (3720 comparisons of 20 votes each) we reconstructed absolute quality scale values according to Thurstone’s model. As a result, we obtained a re-ranking of the 155 participating algorithms w.r.t. the visual quality of the interpolated frames. This re-ranking not only shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks, the results also provide the ground truth for designing novel image quality assessment (IQA) methods dedicated to perceptual quality of interpolated images. As a first step, we proposed such a new full-reference method, called WAE-IQA, which weights the local differences between an interpolated image and its ground truth.

Download Full-text

Traffic Analysis of Quality of Service (QoS) for Video Conferencing between Main Campus and Sub Campus in Laboratory Scale

EMITTER International Journal of Engineering Technology ◽

10.24003/emitter.v3i2.41 ◽

2016 ◽

Vol 3 (2) ◽

Author(s):

Amang Sudarsono ◽

Anang Siswanto ◽

Heru Iswanto ◽

Qoirul Setiawan

Keyword(s):

Quality Of Service ◽

Distance Learning ◽

Packet Loss ◽

Video Conferencing ◽

Learning System ◽

Main Campus ◽

Distance Learning System ◽

Audio Transmission ◽

Video And Audio

Recently, in the distance learning system, video conferencing becomes one of expected course material delivery systems for creating a virtual class such that lecturer and student which are separated at long distance can engage a learning activity as well as face to face learning system. For this reason, the service availability and quality should be able to guaranteed and fulfilled. In this research, we analyze QoS of video conferencing between main campus and sub campus as the implementation of distance learning system in laboratory scale. Our experimental results show that the channel capacity or bandwidth of WAN connection between main campus and sub campus at 128 kbps is able to generate the throughput of video transmission and reception at 281 kbps and 24 kbps, respectively. Meanwhile, throughput of audio transmission and reception is 64 kbps and 26 kbps with the number of total packet loss for video and audio transmission is 84.3% and 29.2%, respectively. In this setting, the total jitter for video and audio transmission is 125 ms and 21 ms, respectively. In this case, there is no packet loss for traffic transmitting and receiving with jitter is not more than 5 ms. We also implemented QoS using Trust CoS model dan Trust DSCP for improving the quality of service in term of jitter up to 12.3% and 22.41%, respectively.Keywords: quality of service, throughput, delay, jitter, packet loss, Trust CoS, Trust DSCP

Download Full-text

Analisis Perbandingan Mean Opinion Score Aplikasi VoIP Facebook Messenger dan Google Hangouts menggunakan Metode E-Model pada Jaringan LTE

ELKOMIKA Jurnal Teknik Energi Elektrik Teknik Telekomunikasi & Teknik Elektronika ◽

10.26760/elkomika.v6i3.379 ◽

2018 ◽

Vol 6 (3) ◽

pp. 379

Author(s):

RAMADHINA FITRIYANTI ◽

LINDAWATI LINDAWATI ◽

ARYANTI ARYANTI

Keyword(s):

Network Topology ◽

Packet Loss ◽

User Satisfaction ◽

Mean Opinion Score ◽

Test Results ◽

Lte Networks ◽

Opinion Score ◽

Call Process ◽

The Mean

ABSTRAKPenelitian ini membahas mengenai perbandingan kualitas layanan dua aplikasi VoIP yaitu Facebook Messenger dan Google Hangouts pada jaringan LTE. Kualitas layanan dua aplikasi tersebut ditinjau menggunakan Mean Opinion Score dengan metode objektif yaitu, E-Model. Nilai MOS kedua aplikasi didapatkan dengan menjalankan simulasi menggunakan GNS3 yang berfungsi untuk mensimulasikan topologi jaringan dengan menggunakan router MikroTik. Setelah itu, dilakukan pengukuran parameter QoS yaitu, delay dan packet loss yang terjadi pada saat melakukan panggilan yang berlangsung selama satu jam. Dari hasil pengujian, diperoleh bahwa nilai MOS pada Google Hangouts lebih besar dibandingkan dengan Facebook Messenger, yaitu 4.0. Sedangkan pada Facebook Messenger hanya sebesar 3.6. Jika dikorelasikan ke tingkat kepuasan pengguna untuk mendapatkan nilai MOS, Google Hangouts dikategorikan "baik" dan Facebook Messenger dikategorikan "cukup baik".Kata kunci: MOS, LTE, VoIP, E-Model, GNS3 ABSTRACTThis study discusses the comparison of the quality of service of two VoIP applications, Facebook Messenger and Google Hangouts on LTE networks. The service quality of the two applications is reviewed using the Mean Opinion Score with an objective method namely, E-Model. MOS values from both applications are obtained by running simulations using GNS3 which can simulate network topology using a MikroTik router. After that, QoS parameters are measured, which are delay and packet loss during the call process which lasts one hour. From the test results, it was found that the MOS value on Google Hangouts was greater than Facebook Messenger, that is 4.0 while on Facebook Messenger only 3.6. If correlated with the level of user satisfaction to get MOS value, Google Hangouts is categorized as "good" and Facebook Messenger is categorized as "good enough".Keywords: MOS, LTE, VoIP, E-Model, GNS3

Download Full-text

Performance evaluation of VoIP technology in an extended service set, in concordance with IEEE 802.11g

Sistemas y Telemática ◽

10.18046/syt.v15i42.2541 ◽

2017 ◽

Vol 15 (42) ◽

pp. 85-100

Author(s):

Marcelo David Núñez Cuadrado ◽

Carlos Andres Jativa Huilcapi ◽

Román Alcides Lara Cueva

Keyword(s):

Packet Loss ◽

The Other ◽

Injection Technique ◽

Conversational Analysis ◽

Delay Jitter ◽

Optimal Delay ◽

Opinion Score ◽

Service Degradation ◽

Extended Service

In this paper, we evaluate the performance in function of the metrics associated to Quality of Service [QoS] and Quality of user Experience [QoE] in an experimental way in the VoIP service for G.711 and G.729 códecs. This was performed over an extended service set based on Wi-Fi technology in concordance with IEEE 802.11g standard using embedded systems. QoS related metrics are obtained by using the intrusive traffic injection technique. In addition, we assessed the QoE using the MOSc [Mean Opinion Score conversational] analysis. The best results were obtained for G.729, reaching up to 25 simultaneous injections with optimal delay, jitter and packet loss values according to the ITU-T recommendation for VoIP. However, the G.711 codec presented a better throughput. On the other hand, QoE evaluation indicates a slight superiority of G.729 in the MOSc appreciation. Finally, we conclude that packet loss and delay are the most influential metrics in VoIP service degradation.

Download Full-text

Full-Reference Quality Metric Based on Neural Network to Assess the Visual Quality of Remote Sensing Images

Remote Sensing ◽

10.3390/rs12152349 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2349 ◽

Cited By ~ 3

Author(s):

Oleg Ieremeiev ◽

Vladimir Lukin ◽

Krzysztof Okarma ◽

Karen Egiazarian

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Visual Quality ◽

Quality Metrics ◽

Remote Sensing Images ◽

Opinion Score ◽

Different Types ◽

Reference Quality ◽

Improved Performance

Remote sensing images are subject to different types of degradations. The visual quality of such images is important because their visual inspection and analysis are still widely used in practice. To characterize the visual quality of remote sensing images, the use of specialized visual quality metrics is desired. Although the attempts to create such metrics are limited, there is a great number of visual quality metrics designed for other applications. Our idea is that some of these metrics can be employed in remote sensing under the condition that those metrics have been designed for the same distortion types. Thus, image databases that contain images with types of distortions that are of interest should be looked for. It has been checked what known visual quality metrics perform well for images with such degradations and an opportunity to design neural network-based combined metrics with improved performance has been studied. It is shown that for such combined metrics, their Spearman correlation coefficient with mean opinion score exceeds 0.97 for subsets of images in the Tampere Image Database (TID2013). Since different types of elementary metric pre-processing and neural network design have been considered, it has been demonstrated that it is enough to have two hidden layers and about twenty inputs. Examples of using known and designed visual quality metrics in remote sensing are presented.

Download Full-text

Towards Layer Adaptation for Audio Transmission

International Journal of Interdisciplinary Telecommunications and Networking ◽

10.4018/ijitn.2014100104 ◽

2014 ◽

Vol 6 (4) ◽

pp. 35-41

Author(s):

Jan Holub ◽

Oldřich Slavata ◽

Pavel Souček ◽

Odysseas Zisimopoulos ◽

Dimitris Toumpakaris ◽

...

Keyword(s):

Signal To Noise Ratio ◽

Rate Adaptation ◽

Physical Layer ◽

Wireless Channel ◽

Audio Quality ◽

Opinion Score ◽

Constant Rate ◽

Channel Conditions ◽

Channel Errors

When audio is transmitted over the wireless channel, the quality of the audio depends on the signal-to-noise ratio (SNR). The purpose of this paper is to investigate if rate adaptation can be avoided, and a system can rely instead on the audio encoder and decoder to alleviate the effect of channel errors. To this end, the paper reports on a set of experiments on various combinations of channel conditions, constellation sizes and audio encoding used and on the final audio quality achieved. The Mean Opinion Score (MOS) is used for performance evaluation. The MOS values are generated using the ITU-T P.862 (PESQ) and P.863 (POLQA) algorithms, and also using tests by experts. The results support the common practice of adapting the physical layer parameters to changing channel conditions. However, in some cases, it is possible to maintain a constant rate without impacting significantly the quality of the audio. This means that the complexity associated with physical layer and audio rate adaptation can be avoided leading to simpler and more robust designs.

Download Full-text