Skip to content

HOW TO CONTRIBUTE TO TQDM#

TL;DR: Skip to [QUICK DEV SUMMARY]

This file describes how to

  • contribute changes to the project, and
  • upload released to the PyPI repository.

Most of the management commands have been directly placed inside the Makefile:

make [<alias>]  # on UNIX-like environments
python -m pymake [<alias>]  # if make is unavailable

The latter depends on py-make>=0.1.0.

Use the alias help (or leave blank) to list all available aliases.

HOW TO COMMIT CONTRIBUTIONS#

Contributions to the project are made using the "Fork & Pull" model. The typical steps would be:

  1. create an account on github
  2. fork tqdm
  3. make a local clone: git clone https://github.com/your_account/tqdm.git
  4. make changes on the local copy
  5. test (see below) and commit changes git commit -a -m "my message"
  6. push to your GitHub account: git push origin
  7. create a Pull Request (PR) from your GitHub fork (go to your fork's webpage and click on "Pull Request." You can then add a message to describe your proposal.)

WHAT CODE LAYOUT SHOULD I FOLLOW?#

Don't worry too much - maintainers can help reorganise contributions. However it would be helpful to bear in mind:

  • The standard core of tqdm, i.e. tqdm.std.tqdm
    • must have no dependencies apart from pure python built-in standard libraries
    • must have negligible impact on performance
    • should have 100% coverage by unit tests
    • should be appropriately commented
    • should have well-formatted docstrings for functions
      • under 76 chars (incl. initial spaces) to avoid linebreaks in terminal pagers
      • use two spaces between variable name and colon, specify a type, and most likely state that it's optional: VAR<space><space>:<space>TYPE[, optional]
      • use [default: ...] for default values of keyword arguments
    • will not break backward compatibility unless there is a very good reason
      • e.g. breaking py26 compatibility purely in favour of minor readability changes (such as converting dict(a=1) to {'a': 1}) is not a good enough reason
    • API changes should be discussed carefully
    • remember, with millions of downloads per month, tqdm must be extremely fast and reliable
  • Any other kind of change may be included in a (possibly new) submodule
    • submodules are likely single python files under the main tqdm/ directory
    • submodules extending tqdm.std.tqdm or any other module (e.g. tqdm.notebook.tqdm, tqdm.gui.tqdm)
    • CLI wrapper tqdm.cli
      • if a newly added tqdm.std.tqdm option is not supported by the CLI, append to tqdm.cli.UNSUPPORTED_OPTS
    • can implement anything from experimental new features to support for third-party libraries such as pandas, numpy, etc.
    • submodule maturity
      • alpha: experimental; missing unit tests, comments, and/or feedback; raises tqdm.TqdmExperimentalWarning
      • beta: well-used; commented, perhaps still missing tests
      • stable: >10 users; commented, 80% coverage
  • .meta/
    • A "hidden" folder containing helper utilities not strictly part of the tqdm distribution itself

TESTING#

Once again, don't worry too much - tests are automated online, and maintainers can also help.

To test functionality (such as before submitting a Pull Request), there are a number of unit tests.

Standard unit tests#

The standard way to run the tests:

  • install tox
  • cd to the root of the tqdm directory (in the same folder as this file)
  • run the following command:
[python -m py]make test
# or:
tox --skip-missing-interpreters

This will build the module and run the tests in a virtual environment. Errors and coverage rates will be output to the console/log. (Ignore missing interpreters errors - these are due to the local machine missing certain versions of Python.)

Note: to install all versions of the Python interpreter that are specified in tox.ini, you can use MiniConda to install a minimal setup. You must also ensure that each distribution has an alias to call the Python interpreter (e.g. python312 for Python 3.12's interpreter).

Alternative unit tests with pytest#

Alternatively, use pytest to run the tests just for the current Python version:

  • install test requirements: [python -m py]make install_test
  • run the following command:
[python -m py]make alltests

MANAGE A NEW RELEASE#

This section is intended for the project's maintainers and describes how to build and upload a new release. Once again, [python -m py]make [<alias>] will help. Also consider pip installing development utilities: [python -m py]make install_build at a minimum, or a more thorough conda env create.

Pre-commit Hook#

It's probably a good idea to use the pre-commit (pip install pre-commit) helper.

Run pre-commit install for convenient local sanity-checking.

Semantic Versioning#

The tqdm repository managers should:

Checking pyproject.toml#

To check that the pyproject.toml file is compliant with PyPI requirements (e.g. version number; reStructuredText in README.rst) use:

[python -m py]make testsetup

To upload just metadata (including overwriting mistakenly uploaded metadata) to PyPI, use:

[python -m py]make pypimeta

Merging Pull Requests#

This section describes how to cleanly merge PRs.

1 Rebase#

From your project repository, merge and test (replace pr-branch-name as appropriate):

git fetch origin
git checkout -b pr-branch-name origin/pr-branch-name
git rebase master

If there are conflicts:

git mergetool
git rebase --continue

2 Push#

Update branch with the rebased history:

git push origin pr-branch-name --force

Non maintainers can stop here.

Note: NEVER just git push --force (this will push all local branches, overwriting remotes).

3 Merge#

git checkout master
git merge --no-ff pr-branch-name

4 Test#

[python -m py]make alltests

5 Push to master#

git push origin master

Building a Release and Uploading to PyPI#

Formally publishing requires additional steps: testing and tagging.

Test#

Ensure that all online CI tests have passed.

Tag#

  • ensure the version has been tagged. The tag format is v{major}.{minor}.{patch}, for example: v4.4.1. The current commit's tag is used in the version checking process. If the current commit is not tagged appropriately, the version will display as v{major}.{minor}.{patch}.dev{N}+g{commit_hash}.

Upload#

GitHub Actions (GHA) CI should automatically do this after pushing tags. Manual instructions are given below in case of failure.

Build tqdm into a distributable python package:

[python -m py]make build

This will generate several builds in the dist/ folder. On non-windows machines the windows exe installer may fail to build. This is normal.

Finally, upload everything to PyPI. This can be done easily using the twine module:

[python -m py]make pypi

Also, the new release can (should) be added to GitHub by creating a new release from the web interface; uploading packages from the dist/ folder created by [python -m py]make build. The wiki can be automatically updated with GitHub release notes by running make within the wiki repository.

Docker images may be uploaded to https://hub.docker.com/r/tqdm/tqdm. Assuming docker is installed:

make -B docker
docker login
docker push tqdm/tqdm:latest
docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)

Snaps may be uploaded to https://snapcraft.io/tqdm. Assuming snapcraft is installed (snap install snapcraft --classic --beta):

make snap
snapcraft login
snapcraft push tqdm*.snap --release stable

Notes#

  • you can also test on the PyPI test servers test.pypi.org before the real deployment
  • in case of a mistake, you can delete an uploaded release on PyPI, but you cannot re-upload another with the same version number
  • in case of a mistake in the metadata on PyPI (e.g. bad README), updating just the metadata is possible: [python -m py]make pypimeta

Updating Websites#

The most important file is .readme.rst, which should always be kept up-to-date and in sync with the in-line source documentation. This will affect all of the following:

  • README.rst (generated by mkdocs.py during make build)
  • The main repository site which automatically serves the latest README.rst as well as links to all of GitHub's features. This is the preferred online referral link for tqdm.
  • The PyPI mirror which automatically serves the latest release built from README.rst as well as links to past releases.
  • Many external web crawlers.

Additionally (less maintained), there exists:

Helper Bots#

There are some helpers in .github/workflows to assist with maintenance.

  • Comment Bot
    • allows maintainers to write /tag vM.m.p commit_hash in an issue/PR to create a tag
  • Post Release
  • Benchmark

QUICK DEV SUMMARY#

For experienced devs, once happy with local master, follow the steps below. Much is automated so really it's steps 1-5, then 11(a).

  1. test ([python -m py]make alltests or rely on pre-commit)
  2. git commit [--amend] # -m "bump version"
  3. git push
  4. wait for tests to pass a) in case of failure, fix and go back to (1)
  5. git tag vM.m.p && git push --tags or comment /tag vM.m.p commit_hash
  6. [AUTO:GHA] [python -m py]make distclean
  7. [AUTO:GHA] [python -m py]make build
  8. [AUTO:GHA] upload to PyPI. either: a) [python -m py]make pypi, or b) twine upload -s -i $(git config user.signingkey) dist/tqdm-*
  9. [AUTO:GHA] upload to docker hub: a) make -B docker b) docker push tqdm/tqdm:latest c) docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)
  10. [AUTO:GHA] upload to snapcraft: a) make snap, and b) snapcraft push tqdm*.snap --release stable
  11. Wait for GHA to draft a new release on https://github.com/tqdm/tqdm/releases a) replace the commit history with helpful release notes, and click publish b) [AUTO:GHA] attach dist/tqdm-* binaries (usually only *.whl*)
  12. [SUB][AUTO:GHA-rel] run make in the wiki submodule to update release notes
  13. [SUB][AUTO:GHA-rel] run make deploy in the docs submodule to update website
  14. [SUB][AUTO:GHA-rel] accept the automated PR in the feedstock submodule to update conda
  15. [AUTO:GHA-rel] update the gh-pages project benchmarks a) [python -m py]make testasvfull b) asv gh-pages

Key:

  • [AUTO:GHA]: GitHub Actions CI should automatically do this after git push --tags (5)
  • [AUTO:GHA-rel]: GitHub Actions CI should automatically do this after release (11a)
  • [SUB]: Requires one-time make submodules to clone docs, wiki, and feedstock