Human Behavior and Activities Modeling

Modeling and recognition of human behavior is a key challenge in a number of application areas such as E-Learning, Industry 4.0, entertainment etc.

The first challenge is to develop non-intrusive human observation systems, which can accurately obtain data of multiple persons during normal activities such as learning or working. We have developed Human activities observation system based on Kinect sensor, which can obtain features such as body skeleton, face points, face gaze, hand gestures etc. We use video and depth sensors to obtain RGB + 3D representation of the scene, from which motion and structural features are estimated.

The second challenge involves development of accurate models of human behavior and activities. We use machine learning and deep learning to train models of human activities, which allow accurate prediction of activity and attention classes from sensor data.

Automatic Analysis of Human Behaviour might include the following tasks:

  • estimate basic behaviour patterns such as: writing notes, observing, listening, .. from observation data
  • estimate engagement and attention of a person during tasks (such as lectures, learning, working on an assignment etc.)

Tools and Methods:

  • image/video processing to define and detect low-level features,
  • machine learning and deep learning to derive models of human activities and obtain estimates (predictions) of activity/attention classes from the input data.


The applications might include estimation of efficiency of learning, improvements of industrial or other training, surveillance of work processes in industry etc.

Use case: Estimation of Attention level of students