Skip to content

GSoC 2021 project report

Scott Veirs edited this page Sep 23, 2021 · 5 revisions

This repository was created as part of the 2021 Google Summer of Code project Github Actions Workflows for Scheduled Algorithm Deployment.

Code contributions

Finished/in progress/planned work can be tracked on the corresponding board.

Infrastructure

  • .gitignore [1, 2]
  • License PR
  • Code Quality
    • pre-commit hooks with isort, Black, pyupgrade PR
    • flake8 PR
    • pep8-naming PR
    • generic hooks PR
    • linting through GitHub Actions on PRs and pushes using pre-commit hooks PR (this was later changed to only pushes as we switched to pre-commit.ci PR)
    • rewritten linting workflow to utilize matrix jobs PR

Features

  • Orcasound workflow PR

    Functionally it's similar to the OOI workflow, however with a hardcoded data source folder for tests. Spectrogram creation is different from OOI as well. prints were replaced with proper logging (for OOI too). Orcasound Python script also has some inputs (input_dir, output and NFFT (spectrogram parameter). Some NFFT investigation [1, 2]. .ts to .wav conversion was done using ffmpeg-python.

    • bug fixes: fix handling of very short files (corresponding issue upstream), fix redefinition of builtin function
  • OOIPy in OOI workflow PR

    OOI workflow was rewritten to use OOIPy package. Instead of processing each raw file it now processes 5-minute time chunks. Spectrogram creation is now reused between OOI and Orcasound, works for mono and stereo audio.

  • OOI inputs PR

    Added input arguments to OOI Python script (using argparse, similar to Orcasound) and to the workflow itself. Now it is possible to choose hydrophone node, start and end time, segment length (acquired audio is split into 5-minute length chunks by default), output directory and NFFT. This PR also added the first tests, for utility functions (spectrogram creation) and integration tests for OOI with downloading data.

    • bug fixes: fix failing test for spectrograms
  • GitHub Action for OOI

    Repository. Docker Action acquiring data from OOI, basically a replica of existing workflow, input arguments: node, start, end time, output directory. Example run.

Blog posts

Miscellaneous

  • Investigated using Sox for spectrogram creation
  • Discovered some extra files in AWS buckets
  • Contributed to the OOIPy package
  • Met lots of great people and had fun this summer 🎉
Clone this wiki locally