Implementation of Cyclist Stress Research Paper

Code Files

preprocessing.py - Data Preprocessing, Denoising, Downsampling
neural_networks.py - Neural networks involved in the project
contrastive_loss.py - Implementation of Cross Domain Contrastive Loss (CDCL) in PyTorch
uda_contrastive_loss_pipeline.py - Training the encoders to achieve domain invariance using Contrastive loss
warped_gaussian_process.py - Implementation of Multi-task Warped Gaussian Process using GPyTorch
wgp_pipeline.py - Training and Testing MTWGP
classification_pipeline.py - Training and Testing the final classifier
CDCL.md - CDCL formula

Setup

Main Libraries used: PyTorch | GPyTorch | SciPy | noisereduce

pip install -r requirements.txt

Latest versions are recommended

Modules

1. Unsupervised Domain Adaptation with Cross Domain Contrastive Loss

This module uses a multi-modal neural network with separate encoders for Audio and IMU data. The goal is to make the encoder representations invariant across labeled source domain (simulated data) and unlabeled target domain (real-world data). This is achieved by using Cross-domain Contrastive Loss. The encoder representations are first concatenated and flattened, then passed into a projection network to ensure a common dimension space. The loss function encourages the encoders to produce similar representations for semantically related instances from different domains. Once, the training is complete, the projection network is discarded.

For Audio data, amplitude is used, while for IMU, acceleration and gyroscope data along three axes is used. The novelty lies in the parallel branching of each IMU component, enabling the model to learn and capture nuanced patterns independently for enhanced feature extraction.

To run this, first you should have the following data hierarchies:

    /Source_Data
    ├── Participant 1
    |   ├── Session 1
    │   │   ├── bike_data.csv - IMU data
    |   |   ├── bike_audio_timestamp.csv - Labels
    |   |   └── bike_audio_timestamp.pcm - Audio data
    |   ├── Session 2 .....
    |   └── Session 3 ..... so on   
    |   
    ├── Participant 2 ..... so on
    |

    /Target_Data
    ├── Participant 1
    |   ├── Session 1
    │   │   ├── bike_data.csv - IMU data
    |   |   └── bike_audio_timestamp.pcm - Audio data
    |   ├── Session 2 .....
    |   └── Session 3 ..... so on   
    |   
    ├── Participant 2 ..... so on
    |

Then, run the pipeline using:

python uda_contrastive_loss_pipeline.py "path/to/source/data" "path/to/target/data"

This will run the preprocessing pipeline, create pseudo labels, run the domain adaptation training pipeline, and generate data for MTWGP pipeline.

2. Multi Task Warped Gaussian Process (MTWGP)

The representations obtained from domain invariant encoders are used to learn the relationship between IMU data (x) and Audio data (y) using MTWGP. The goal is to minimize energy consumption. Once, the MTWGP is trained, the IMU data can be used to generate the audio representations directly without having to collect audio data through recording. Since, audio data might not be Gaussian, a Warping function is introduced.

python wgp_pipeline.py

This will run the MTWGP pipeline and generate data for classification pipeline

3. Classification

Now, we can use IMU representation obtained from domain invariant encoder, generate audio reprentations using MTWGP, concatenate and flatten them again to train and test the classifier and get the pressure levels (1 - low, 2 - medium, 3 - high).

python classification_pipeline.py

This will run the final classification and output the pressure levels.

4. After training workflow

Record IMU data -> Pass to IMU encoder -> Pass the reps to MTWGP to get Audio reps -> concatenate + flatten -> Pass to classifier -> Pressure level

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!