IP Telephony: Comparison of Subjective Assessment Methods for Voice Quality Evaluation

Therdpong DAENGSI; Chai WUTIWIWATCHAI; Apiruck PREECHAYASOMBOON; Saowanit SUKPARUNGSEE

Authors

Therdpong DAENGSI Department of Information Technology, Faculty of Information Technology, King Mongkut’s University of Technology North Bangkok, Bangkok 10800
Chai WUTIWIWATCHAI Speech and Audio Laboratory, National Electronics and Computer Technology Center Pathumthani, Pathumthani 12120
Apiruck PREECHAYASOMBOON Metropolitan Sale and Services, TOT Public Company Limited, Bangkok 10150
Saowanit SUKPARUNGSEE Department of Applied Statistics, Faculty of Applied Science, King Mongkut’s University of Technology North Bangkok, Bangkok 10800

Keywords:

IP Telephony, VoIP, voice quality, subjective tests, MOS

Abstract

One problem that often occurs after installing/implementing an IP telephony system is voice quality. Although there are objective measurement tools for voice quality evaluation, the prices of these are very expensive. Therefore, back to basics, this paper focuses on subjective tests. This study used the data from three tests, consisting of listening-opinion tests, conversational tests and interview tests. All were conducted using the same IP telephony system with G.711 codec. All tests, following the ITU-T P.800, were in the best condition. The subjects who participated in the tests were 163 students and 1 worker in King Mongkut’s University of Technology North Bangkok (KMUTNB). This study compared and analyzed the data from 3 kinds of subjective tests using ANOVA, to see if the data from the interview tests were consistent significantly with the data from the listening and conversational-opinion tests. The results, called the Mean Opinion Score (MOS) values, from the interview, conversational, and listening-opinion tests were 4.14, 4.16 and 4.23 respectively. Also, the analyzed result shows a p-value of 0.511. This means the MOS values from these three methods are not significantly different. Therefore interview tests can be used to evaluate voice quality, and is as good as other subjective methods, without high cost of expensive tools, making it very applicable in developing countries.

doi:10.14456/WJST.2014.31

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

F De Rango, M Tropea, P Fazio and S Marano. Overview on VoIP: Subjective and objective measurement methods. Int. J. Comput. Sci. Netw. Secur. 2006; 6, 140-53.

S Karapantazis and FN Pavlidou. VoIP: A comprehensive survey on a promising technology. Comput. Netw. 2009; 53, 2050-90.

A Sulkin. PBX Systems for IP Telephony. McGraw-Hill, New York, 2002, p. 150-3.

Avaya IP Voice Quality Network Requirements. Available at: http://downloads.avaya.com/css/P8/documents/100018203, accessed November 2012.

ITU-T Recommendation P.800 (Methods for subjective determination of transmission quality). Available at: http://www.itu.int/rec/T-REC-P.800-199608-I/en, accessed November 2012.

T Triyason and P Kanthamanon. Perceptual evaluation of speech quality measurement on Speex codec VoIP with Tonal language Thai. In: Proceeding of the 5th International Conference on Advances in Information Technology, Bangkok, Thailand, 2012, p. 181-90.

ITU-T Recommendation P.805 (Subjective evaluation of conversational quality). Available at: http://www.itu.int/rec/T-REC-P.805-200704-I/en, accessed November 2012.

ITU-T Recommendation P.82 (Methods for evaluation of service from the standpoint of speech transmission quality). Available at: http://www.itu.int/rec/T-REC-P.82-198811-W/en, accessed November 2012.

ITU-T Recommendation G.711 (Pulse code modulation (PCM) of voice frequencies). Available at: http://www.itu.int/rec/T-REC-G.711-198811-I/en, accessed November 2012.

Avaya IP Telephony Implementation Guide. Available at: https://downloads.avaya.com/elmodocs2/

comm_mgr/r3_1/avaya-iptel-imp-guide3.1.pdf, accessed November 2012.

T Daengsi, C Wutiwiwatchai, A Preechayasomboon and S Sukparungsee. VoIP quality measurement: Insignificant voice quality of G.711 and G.729 codecs in listening-opinion tests by Thai users. Inform. Tech. J. 2012; 8, 77-82.

T Daengsi, C Wutiwiwatchai, A Preechayasomboon and S Sukparungsee. A study of VoIP quality evaluation: User perception of voice quality from G.729, G.711 and G.722. In: Proceeding of the 9th Annual IEEE Consumer Communications and Networking Conference, Las Vegas, Nevada, USA, 2012, p. 342-5.

T Daengsi, C Wutiwiwatchai, A Preechayasomboon and S Sukparungsee. Speech quality assessment of VoIP: G.711 VS G.722 based on interview tests with Thai users. Int. J. Inform. Tech. Comput. Sci. 2012; 4, 19-25.

T Daengsi, A Preechayasomboon, S Sukparungsee, P Chootrakool and C Wutiwiwatchai. The Development of a Thai Speech Set for Telephonometry. Available at: http://desceco.org/O-COCOSDA2010/proceedings/paper_53.pdf, accessed November 2012.

ITU-T Recommendation P.810 (Modulated Noise Reference Unit (MNRU)). Available at: http://www.itu.int/rec/T-REC-P.810-199602-I/en, accessed November 2012.

T Daengsi and K Tontiwattanakul. A case of improvement of building acoustics using available equipments and limited resources. In: Proceeding of the 6th Naresuan Research Conference, Phitsanulok, Thailand, 2010, p. 2-13.

AW Rix. Comparison between subjective listening quality and P.862 PESQ score. Available at: http://82.165.4.28/com/docs/downloads/pesq/pdf/wp_sub_v_pesq.pdf, accessed November 2012.