WP2: Attention mechanisms

  1. 1

    Development of Deep Learning algorithms for detection of different sleep events, overcoming limitations of current approaches.

  2. 2

    Integration of Transformers within the PSG analysis framework due to their ability to model complex time series dynamics without sequential analysis.

  3. 3

    Use the Perceiver architecture to reduce space complexity by adding a cross-attention layer between the input sequence and multi-headed attention.

  4. 4

    Further refinement: use the Perceiver(IO) for complex outputs

In this WP we are going to develop deep learning attention mechanisms algorithms to allow concurrent and individual identification of different types of events in the context of each PSG interval, intending to overcome limitations of current approaches and being more robust, generalizable and computationally efficient.

Here we plan the integration of Transformers within the PSG analysis framework because they do not analyze inputs sequentially, having therefore the potential of modeling complex time series dynamics. This can be done in several ways that must be studied.

A further step down this line can be taken by considering a prediction problem in which each token represents the value of a single variable per time step. Transformers are then free to attend to the values of any variable at any time in order to produce more accurate predictions. Since the complexity of Transformers might become a limiting factor in our PSG context it is planned to use the Perceiver architecture to reduce space complexity by adding a cross attention layer between the input sequence and multi-headed attention. A supercharged version, namely Perceiver(IO), has also been recently proposed which adds extra cross-attention in the last layer of the decoder, this, in the proposed processing pipeline can allow complex output configuration, including event positioning, which might be a convenient approach for integration of self-attention in the event identification framework.

Publications

2025

Multi-Task Deep-Learning for Sleep Event Detection and Stage Classification

[Conference] IEEE Symposium Series on Computational Intelligence

Anido-Alonso, Adriana; Alvarez-Estevez, Diego

A multi-task deep-learning approach, inspired by computer vision's object detection, is proposed to simultaneously detect sleep events and construct hypnograms, streamlining the complex process of polysomnographic sleep analysis.

2024

Performance Analysis of Parameterized Quantum Kernel Methods on a Selection of Machine Learning Datasets

[Conference] Proceedings XoveTIC 2024

Alvarez-Estevez, Diego

This work explores Quantum Kernel Estimation (QKE) and Quantum Kernel Training (QKT) in the context of classification tasks. Two quantum feature maps are analyzed for this purpose, in comparison to classical Support Vector Machine counterparts. Classification performance is analyzed on a selection of both ad-hoc and classical datasets, with QKT applied to optimize kernel parameters in QKE. Experimental data shows that quantum methods outperform classical ones in ad-hoc data. However, when confronting classical datasets, they frequently encounter difficulties in generalization, despite achieving high accuracy on the training set. We conclude that the choice of the feature mapping and the optimization of kernel parameters are critical for maximizing the effectiveness of the quantum methods.

Simultaneous Sleep Events Detection and Stages Classification Using a Multi-Task Deep-Learning Approach

[Conference] Proceedings XoveTIC 2024

Anido-Alonso, Adriana; Alvarez-Estevez, Diego

Accurate detection of sleep stages and sleep events is essential for the effective diagnosis and treatment of sleep disorders. However, current state-of-the-art methods often fall short in integrating these multiple tasks simultaneously. This study introduces a novel multi-task deeplearning approach for the joint detection of sleep events and hypnogram construction in a single pass. The proposed method adapts state-of-the-art single-shot object detection techniques to multi-channel time-series data, enabling simultaneous classification and detection of sleeping events. Experiments were conducted using diverse input channel combinations, including electroencephalogram (EEG), electrooculogram (EOG), electromyogram (EMG), oxygen saturation, airflow, and thoracic-abdominal movements, as well as multiple output configurations regarding sleep stages, EEG arousals, and respiratory events, both individually and combined. Results demonstrate improved accuracy in detecting respiratory events and sleep stages with additional channels. The multi-task approach enhanced overall performance, benefiting from shared knowledge across tasks.