Development Guide

Here are details of contributing to CharGer.

Workflow overview

A typical workflow of updating CharGer involves these steps:

  1. Add new code

  2. Add new tests for the new code

  3. Pass all the style checks (isort, black, and flake8)

  4. Pass all the type checks (mypy)

  5. Pass all the functional tests (pytest)

  6. Update documentation (sphinx)

  7. Commit the change

Below are the details of each step.

Set up environment

CharGer packages are managed by poetry. Install poetry following its official documentation. On macOS with Homebrew, install poetry by:

brew install poetry

Make sure the conda is available and has set up bioconda channels. Create a new conda environment (for example, charger_py38):

conda create -n charger_py38 python=3.8 pip cyvcf2=0.30 pysam=0.16
conda activate charger_py38
git clone https://github.com/ding-lab/CharGer charger
cd charger
poetry install      # install charger and all its dependencies
pre-commit install  # enforce style check at every commit

Check styles

Style checks are enforced to pass before any git commit (by running pre-commit install once). To run all the style checks at any time, run the following command:

pre-commit run -a

Otherwise, run the style checks manually by:

isort               # Sort the import order
black src tests     # Format the code
flake8              # Check coding style

Check types

Type checks use mypy to infer the data types of the Python variables and report any potential mismatches (say, passing string where an integer is required). Run type checks by:

mypy --pretty src tests

Run functional tests

Functional tests check if the program works as expected. All the tests are under tests folder. Some tests check one particular functions, and some other tests check if the output is expected given a certain CharGer config.

Run functional tests by:

pytest -v

Functional tests and type checks are not run automatically for every commit, but it’s recommended to run and pass all the tests and type checks.

The repo should always pass all the tests (style checks, type checks, and functional tests) described above.

Test multiple Python versions

To make sure CharGer works on all Python versions, we use tox to run tests on each supported Python version:

conda install tox tox-conda
tox -p auto -c tox_conda.ini

Build documentation

CharGer’s documentation is powered by sphinx under docs. Build or update the documentation by:

cd docs
make html

And the documentation will be available under docs/_build/html.

To publish the newly built documentation:

ghp-import -r origin -b gh-pages --push \
    -m 'Update documentation' \
    --no-jekyll \
    _build/html/

Update Dependencies

To list all the upgradable dependencies:

poetry show --outdated

Change the pypoetry.toml if the newer version is applicable. Then Update the fixed package versions by poetry.lock by:

poetry update

Develop using Visual Studio Code

Here are some additional setup that utilize Visual Studio Code’s IDE:

  • Run style checks (black, mypy and flake8) at every file save

  • The default build task will call sphinx to build the documentation

  • A debug shortcut to go into CharGer internals using a startup script scripts/debug_example.py

Add the following workspace settings .vscode/settings.json:

{
    "python.autoComplete.extraPaths": ["src"],
    "python.formatting.provider": "black",
    "editor.formatOnSave": true,
    "editor.wordWrapColumn": 120,
    "python.linting.enabled": true,
    "python.linting.flake8Enabled": true,
    "python.linting.mypyEnabled": true,
    "python.linting.mypyArgs": ["--follow-imports=normal", "--show-column-numbers"],
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": ["-o", "junit_family=xunit1"],
}

Set up the documentation build as a task in .vscode/tasks.json:

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Build document",
            "type": "process",
            "options": {
                "cwd": "${workspaceFolder}/docs"
            },
            "command": "${config:python.pythonPath}",
            "args": ["-m", "sphinx", "-b", "html", ".", "_build/html"],
            "group": {
                "kind": "build",
                "isDefault": true
            },
            "presentation": {
                "echo": true,
                "reveal": "silent",
                "focus": false,
                "panel": "dedicated",
                "showReuseMessage": true,
                "clear": true
            }
        }
    ]
}

Set up the the debug shortcut in .vscode/launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Run CharGer",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/scripts/debug_example.py"
        }
    ]
}