Forget PIP, Conda, and requirements.txt! Use Poetry Instead And Thank Me Later

Pain-free dependency management is finally here

Image by me with Midjourney

Library A requires Python 3.6. Library B relies on Library A but needs Python 3.9, and Library C depends on Library B but requires the specific version of Library A that is compatible with Python 3.6.

Welcome to dependency hell!

Since native Python is rubbish without external packages for data science, data scientists can often find themselves trapped in catch-22 dependency situations like the one above.

Tools like PIP, Conda, or the laughable requirements.txt files can’t solve this problem. Actually, dependency nightmares exist largely because of them. So, to end their suffering, the Python open-source community developed the charming tool known as Poetry.

Poetry is an all-in-one project and dependency management framework with over 25k stars on GitHub. This article will introduce Poetry and list the problems it solves for data scientists.

Let’s get started.

Installation

While Poetry can be installed as a library with PIP, it is recommended to install it system-wide so you can call poetry on the CLI anywhere you like. Here is the command that runs the installation script for Unix-like systems, including Windows WSL2:

curl -sSL https://install.python-poetry.org | python3 -

If, for some weird reason, you use Windows Powershell, here is the suitable command:

(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -

To check if Poetry is installed correctly, you can run:

$ poetry -v

Poetry (version 1.5.1)

Poetry also supports tab completion for a variety of shells like Bash, Fish, Zsh, etc. Learn more about it here.

1. Consistent structure for all projects

Since Poetry is an all-in-one tool, you can use it from the start to the very end of your project.

When starting a fresh project, you can run poetry new project_name. It will create a default…