Human intention recognition and classification based on operator 4.0 and artificial intelligence

Pannon Egyetem

Vegyészmérnöki és Anyagtudományok Doktori Iskola

Ruppert Tamás

Abonyi János

bio-, környezet- és vegyészmérnöki tudományok

This research aims to use multimodal techniques with operator 4.0 and artificial intelligence to monitor and classify human intention. The proposed idea will use the following parameters as inputs to the deep learning neural network. Speech signals and gesture dynamic: this is user-dependent and based on specific body gesture that reflects the emotional status. Face expression and gaze tracking, and finally apply Galvanic Skin Response GSR (Skin Conductance)
The human intention will be classified for each input separately for the first time. Then all the inputs will be fed together (speech signals, face expression, GSR, etc.) into the system to classify the intentions precisely. Classification of intentions will be achieved using the data from the participants, and also available free data.

The tasks of the applicant while using process mining would be:
• Develop a framework for monitoring human intention parameters.
• Gather data from physiological data, and vital sign of human operators
• Develop an AI system for processing the collected data individually and together
• Study the efficiency of the designed system for each input separately (Speech signals, face expression, GSR, etc.)

1 fő

PE

2021-01-31

Human intention recognition and classification based on operator 4.0 and artificial intelligence