Thesis supervisor: Bálint Gyires-Tóth
Location of studies (in Hungarian): Department of Telecommunications and Media Informatics Abbreviation of location of studies: TMIT
Description of the research topic:
Various data are based on sequential structure, such as speech, natural language, state-action pairs in reinforcement learning models, temporal data and time series. Among the major challenges associated with sequential data modeling are (1) expanding the receptive field, (2) multi-dimensional data modeling, and (3) multiscale data modeling. A larger receiving field and multiple dimensions provide more information, while multiscale modeling could provide insight into the hierarchy of the underlying process. A low signal-to-noise ratio, however, may result in a significant degradation of performance. If the data are limited or the process can be estimated by a certain number of parameters, it is rational to utilize parametric models. Modeling performance of such methods is limited when dealing with complex processes and large amounts of data. In recent years, deep neural networks have demonstrated outstanding results in a variety of research areas, including sequence modeling. By addressing implicitly or explicitly, to some extent, these three challenges, such models exhibit outstanding representation learning capabilities. As a result, the learned representations could be utilized to realize a variety of downstream tasks, including regression, classification, and anomaly detection. Due to the possible rapid changes in the distribution of sequential data, the task of establishing a high quality, robust representation can be extremely challenging. In addition, both maximizing data efficiency and reducing energy consumption are critical to training and inference in large deep learning models.
The goal of this research is to develop novel deep learning methods for modeling and learning representations of sequential data. Aside from performance improvements (measured by accuracy, training convergence speed, and/or robustness), explainability and sustainability should also be taken into account in the research. A minimum of one application scenario must be used to demonstrate the effectiveness of the proposed methods. The application scenario may be natural language processing or financial modeling, for instance.
The possible research tasks of the PhD candidate are the following:
• Overview of the related scientific papers, including fundamentals of deep neural networks and novel results of sequence modeling.
• Design and realize baseline and existing state of the art sequential and temporal data modeling approaches for benchmark.
• Elaborate novel deep learning based methods for modeling and representation learning of sequential and temporal data.
• Investigate the performance of the proposed models in terms of accuracy, speed, robustness, explainability and sustainability.
• Demonstrate the effectiveness of the results at least in one application scenario.
• Objective and (if possible) subjective or fundamental evaluation of the proposed methods.
The research can be performed in English and Hungarian.
Required language skills: angol Number of students who can be accepted: 1