• JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
  • JoomlaWorks Simple Image Rotator
 
  Bookmark and Share
 
 
Master's Dissertation
DOI
https://doi.org/10.11606/D.59.2022.tde-18082022-081459
Document
Author
Full name
Pedro Henrique D'Almeida Giberti Rissato
E-mail
Institute/School/College
Knowledge Area
Date of Defense
Published
Ribeirão Preto, 2022
Supervisor
Committee
Macedo, Alessandra Alaniz (President)
Murta Junior, Luiz Otavio
Pimentel, Maria da Graça Campos
Watanabe, Carolina Yukari Veludo
Title in Portuguese
Reconhecimento de praxia não verbal em imagens da face humana utilizando aprendizado de máquina e rede neural
Keywords in Portuguese
Face humana
Landmarks
Reconhecimento de padrões
Visão computacional
Abstract in Portuguese
A capacidade de comunicar-se por meio da fala é essencial para qualquer ser humano. Contudo, pessoas com Transtorno de Fala (TF) decorridas de apraxia de fala na infância, desordem fonológica ou fonética necessitam de terapia fonoaudiológica. O profissional fonoaudiológico propõe uma série de exercícios para fortalecer os músculos orofaciais. Nesse contexto, os movimentos e sons não articulatórios como, por exemplo, sopro, estalo de língua ou beijo, exercitam e fortalecem boca, lábios, língua e bochechas que apoiam e sustentam a fala. Nesse sentido, o objetivo deste estudo consistiu em propor um método para o reconhecimento de beijo, estalo de língua e sopro na face humana utilizando pontos de marcação, denominados de landmarks. O método consiste em reconhecer o rosto humano, extrair a distância Euclidiana entre a análise combinatória de 20 landmarks da boca humana, para construir um vetor de distâncias. Esse vetor de distâncias foi utilizado para induzir modelos com os algoritmos de Árvore de Decisão, k-vizinhos mais próximos, Random Forest, Support Vector Machine e treinar uma rede neural do tipo Multilayer Perceptron. Por meio do método desenvolvido, o modelo induzido com Random Forest apresentou os melhores resultados e foi capaz de classificar entre as classes: (i) beijo e estalo; (ii) estalo e sopro e (iii) beijo e sopro, com uma acurácia de 93%, 93% e 65%, respectivamente. A separação entre os movimentos foi satisfatória e o modelo generalizado pode ser utilizado como apoio ao tratamento fonoaudiológico de pacientes com Transtornos de Fala.
Title in English
Recognition of non-verbal praxis in images of the human face using machine learning and neural network
Keywords in English
Computer vision
Human face
Landmarks
Pattern recognition
Abstract in English
The ability to communicate through speech is essential for any human being. However, people with Speech Disorder (SD) due to childhood speech apraxia, phonological disorder or phonetics need speech therapy. The speech therapist proposes a series of exercises to strengthen the orofacial muscles. In this context, non-articulatory movements and sounds (such as blow, tongue snap, or kiss) strengthen the mouth, lips, tongue, and cheeks to support and sustain speech. In this sense, our goal was to propose a method to recognize kisses, tongue snaps and blows on the human face using landmarks. This method consists of the following steps: recognize the human face, extract the Euclidean distance between the combinatorial analysis of twenty landmarks from the human mouth, and create a vector of distances. This distance vector induces models with the Decision Tree, k-nearest neighbours, Random Forest, Support Vector Machine algorithms. It also trains a Multilayer Perceptron neural network. By using the proposed method, the model induced with Random Forest presented the best results and was able to classify between the classes: (i) kiss and snap; (ii) snap and blow and (iii) kiss and blow, with an accuracy of 93%, 93% and 65%, respectively. The distinction between the movements was satisfactory, and the generalized model can be used to support the speech therapy treatment of patients with Speech Disorders.
 
WARNING - Viewing this document is conditioned on your acceptance of the following terms of use:
This document is only for private use for research and teaching activities. Reproduction for commercial use is forbidden. This rights cover the whole data about this document as well as its contents. Any uses or copies of this document in whole or in part must include the author's name.
Publishing Date
2022-08-26
 
WARNING: Learn what derived works are clicking here.
All rights of the thesis/dissertation are from the authors
CeTI-SC/STI
Digital Library of Theses and Dissertations of USP. Copyright © 2001-2024. All rights reserved.