Publication 2019 DECEMBER VOL 1 Issue 1

Real- Time Hand Gesture Recognition Using Deep Learning

Malavika Suresh, Avigyan Sinha, Aneesh R P  


Abstract - With the impetuous advancement of informatics, human knowledge is unable to bridge the boundaries and human computer interaction is paving the way for new eras. Here, a real-time human gesture recognition using an automated technology called Computer Vision is demonstrated. This is a type of noncognitive computer user interface, having the endowment to perceive gestures and execute commands based on that. The design is implemented on a Linux system but can be implemented by installing modules for python on a windows system also. OpenCV and KERAS are the platforms used for the identification. Gesture displayed in the screen is recognized by the vision-based algorithms. Using background removal technique, an assortment of skin color masks was trained by Lenet architecture in KERAS for the recognition. The users have tested and produced over 5000 masks with KERAS to generate 96% more accurate results..



Gesture Recognition, Computer Vision, Open CV, Lenet, KERAS.




      [1] A. D. Bagdanov, A. Del Bimbo, L. Seidenari, and L. Usai, “Real-time hand status recognition from RGB-D imagery,” in Proceedings of the 21stInternational Conference on Pattern Recognition (ICPR '12), pp. 2456–2459, November 2012.

      [2].M. Elmezain, A. Al-Hamadi, and B. Michaelis, “A robust method for hand gesture segmentation and recognition using forward spotting scheme conditional random fields,” in Proceedings of the 20th International Conference on Pattern Recognition (ICPR '10), pp. 3850–3853, August 2010.

      [3]. C.-S. Lee, S. Y. Chun, and S. W. Park, “Articulated hand configuration and rotation estimation using extended torus manifold embedding,” in Proceedings of the 21st International Conference on Pattern Recognition (ICPR '12), pp. 441–444, November 2012.

      [4]. M. R. Malgireddy, J. J. Corso, S. Setlur, V. Govindaraju, and D. Mandalapu, “A framework for hand gesture recognition and spotting using sub-gesture modeling,” in Proceedings of the 20th International Conference on Pattern Recognition (ICPR '10), pp. 3780–3783, August 2010.

      [5] P. Suryanarayan, A. Subramanian, and D. Mandalapu, “Dynamic hand pose recognition using depth data,” in Proceedings of the 20th International Conference on Pattern Recognition (ICPR '10), pp. 3105–3108, August 2010.

      [6]. S. Park, S. Yu, J. Kim, S. Kim, and S. Lee, “3D hand tracking using Kalman filter in depth space,” Eurasip Journal on Advances in Signal Processing, vol. 2012, no. 1, article 36, 2012.

      [7]. J. L. Raheja, A. Chaudhary, and K. Singal, “Tracking of fingertips and centers of palm using KINECT,” in Proceedings of the 2nd International Conference on Computational Intelligence, Modelling and Simulation (CIMSim '11), pp. 248–252, September 2011.

      [8] Y. Wang, C. Yang, X. Wu, S. Xu, and H. Li, “Kinect based dynamic hand gesture recognition algorithm research,” in Proceedings of the 4th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC '12), pp. 274–279, August 2012.

      [9]. M. Panwar, “Hand gesture recognition based on shape parameters,” in Proceedings of the International Conference on Computing, Communication and Applications (ICCCA '12), pp. 1–6, February 2012.

      [10]. Z. Y. Meng, J.-S. Pan, K.-K. Tseng, and W. Zheng, “Dominant points based hand finger counting for recognition under skin color extraction in hand gesture control system,” in Proceedings of the 6th International Conference on Genetic and Evolutionary Computing (ICGEC '12), pp. 364–367, August 2012.

      [11]. R. Harshitha, I. A. Syed, and S. Srivasthava, “Hci using hand gesture recognition for digital sand model,” in Proceedings of the 2nd IEEE International Conference on Image Information Processing (ICIIP '13), pp. 453–457, 2013.

      [12] Swapnil Athavale, Mona Deshmukh, International Journal of Engineering Research and General Science Volume 2, Issue 2, Feb-Mar 2014

      [13] Murthy, G.R.S. & Jadon, A review of vision based hand gesture recognition. International Journal of Information Technology and Knowledge Management

      [14] Ruchi Manish Gurav P.G.” Real time Finger Tracking and Contour Detection for Gesture Recognition using OpenCV” 2015 International Conference on Industrial Instrumentation and Control (ICIC) College of Engineering Pune, India. May 28-30, 2015

      [15] Anupam Agrawal, Rohit Raj and Shubha Porwal "Vision-based Multimodal Human-Computer Interaction using Hand and Head Gestures" IEEE Conference on Information and Communication Technologies ICT 2013

      [16] Pratibha Pandey, Vinay Jain, "Hand Gesture Recognition for Sign Language: A review,", IJSETR, Vol.4, Issue 3, March 2015

      [17] Hsiang-Yueh Lai, Han-Jheng. Lai,“ Real- Time Dynamic Hand Gesture Recognition” published on 2014 at International Symposium on Computer, Consumer and Control.

      [18] Mr. Deepak K. Ray, MayankSoni, PrabhavJohri, Abhishek Gupta, “Hand Gesture Recognition using Python” presented in “International Journal on Future Revolution in Computer Science & Communication Engineering Volume: 4 Issue: 6”

      [19] Kim, Jonghwa & Mastnik, Stephan & Andre, Elisabeth. (2008). EMG-based hand gesture recognition for realtime biosignal interfacing. 30-39. 10.1145/1378773.1378778

      [20].R. Yang, S. Sarkar, and B. Loeding, “Handling movement epenthesis and hand segmentation ambiguities in continuous sign language recognition using nested dynamic programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 3, pp. 462–477, 2010.

      [21]. Z. Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti, “American sign language recognition with the kinect,” in Proceedings of the 13th ACM International Conference on Multimodal Interfaces (ICMI '11), pp. 279–286, November 2011.

      [22]. A. Shimada, T. Yamashita, and R.-I. Taniguchi, “Hand gesture based TV control system—towards both user—& machine-friendly gesture applications,” in Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV '13), pp. 121–126, February 2013.

      [23] C. Keskin, F. Kiraç, Y. E. Kara, and L. Akarun, “Real time hand pose estimation using depth sensors,” in Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV '11), pp. 1228–1234, November 2011.

      [24]. J. Zeng, Y. Sun, and F. Wang, “A natural hand gesture system for intelligent human-computer interaction and medical assistance,” in Proceedings of the 3rd Global Congress on Intelligent Systems (GCIS '12), pp. 382–385, November 2012.

      [25] P.Suganya, R.Sathya, K.Vijayalakshmi, International Journal of Pure and Applied Mathematics, 2018

      [26] Noreen, Uzma & Jamil, Mutiullah & Ahmad, Nazir. (2016). Hand Detection Using HSV Model. Advances in Computer Science and Engineering.

      [27] M. Shin, D. Goldgof, and K. Bowyer, An Objective Comparison Methodology of Edge Detection Algorithms using a Structure from Motion Task, in Empirical Evaluation Techniques in Computer Vision, K. Bowyer and P. Philips, IEEE Computer Society Press, Los Alamitos, CA

      [28] Srisha, Ravi & Khan, Am. (2013). Morphological Operations for Image Processing : Understanding and its Applications.

      [29] Zainab Shukur Mahmood Mahmood, Sait Ali Uymaz, Anatolian Journal of Computer Science,2019

      [30] Ahmed El-Sawy, Mohamed Loey: CNN for Handwritten Arabic Digits Recognition Based on LeNet-5. In book: Proceedings of the International Conference on Advanced Intelligent Systems and Informatics (2016)

      [31] Geo_rey E. Hinton, Nitish Srivastava, Krizhevsky A., Sutskever, I., Ruslan R.Salakhutdinov: Improving neural networks by preventing co-adaptation of feature detectors. Department of Computer Science, University of Toronto, (2012)

      [32] Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, Xindong Wu, “Object Detection with Deep Learning: A Review”,IEEE Publications,2019

      [33] Md Zahangir Alom , Tarek M. Taha ,Chris Yakopcic , Stefan Westberg, Paheding Sidike,Mst Shamima Nasrin , Mahmudul Hasan, Brian C. Van Essen , Abdul A. S. Awwaland Vijayan K. Asari,” A State-of-the-Art Survey on Deep Learning Theory and Architectures” Published: 5 March 2019


Authors :

Malavika Suresh

Regional center IHRD, Thiruvananthapuram


Avigyan Sinha

Regional center IHRD, Thiruvananthapuram


Regional center IHRD, Thiruvananthapuram


Copy Right 2017 -These are open access articles distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Click the link journals for the details of published journals