How to Set up a Python Development Environment in 2023

As a programming language, Python has a variety of technical merits that make it a versatile tool for software development. This goes for seasoned developers as well as an excellent first language for new programmers. Python's appeal lies in its clear, readable syntax, first-class functions, strong community and ecosystem, and extensive AI and ML capabilities. 

Python is a great language to know, and with the rise of generative AI and Large-Language-Models (LLMs) over the past few months, it’s an incredibly valuable tool to have available. However, knowing Python is just the first step. To take full advantage of its capabilities, developers need a functional, productive development environment, something Python has not always been adept at providing. Has the developer experience story gotten better in 2023? It has, and this article will explore the tools that can help build a great Python development environment.

Basic Tools for Python Development 

Python Installation

In software development, dependency issues are a common occurrence, irrespective of the programming language in use. These issues typically surface when a project is reliant on external packages or libraries, which in turn may depend on other packages. Python, a widely used high-level programming language, is no exception to this rule.

For those new to Python, a common pitfall is to simply use the system-installed Python, start installing packages, and begin writing code. While this approach may seem straightforward, it often leads to a messy and overloaded dependency environment. This is because different projects may require different versions of the same package, leading to conflicts and errors. To avoid these issues, it's advisable to use a version management tool like pyenv. This tool allows users to install and manage multiple versions of Python on the same system, enabling the use of specific Python versions for different projects. This can be particularly useful when a project requires a specific version of Python that is different from the system's default version.

Choosing a Text Editor or IDE

If there’s one topic that’s sure to ignite a passionate exchange of opinions, it's the choice and relative merits of various text editors and Integrated Development Environments (IDEs). There's a wide array of text editors and IDEs available, each with its unique features, strengths, and weaknesses. Some of the most popular choices among developers include VSCode, PyCharm, and Vim/NeoVim. 

VSCode, developed by Microsoft, has gained significant popularity in the developer community due to its feature set. This includesIntelli Sense for code completion, debugging tools, and a sizable library of extensions. It's a great choice for new and experienced programmers, supporting a wide range of programming languages and offering a highly customizable interface.

PyCharm, on the other hand, is a Python-specific IDE developed by JetBrains. It offers a comprehensive suite of tools tailored for Python development, including intelligent code assistance, automated code refactoring, and integrated unit testing. PyCharm is an excellent choice for Python developers seeking a powerful, all-in-one development environment.

Vim and its modern iteration, NeoVim, are text editors that prioritize minimalism. Vim is a “modal” editor, which means users interact with it based on “modes”, such as “visual”, “insert”, and “normal”. Vim/NeoVim have a fairly steep learning curve as a result of their barebones default configuration and heavy use of keyboard shortcuts and macros. However, experienced Vim users are able to write and edit code in an incredibly fast and efficient manner, having mastered the key shortcuts and plugin utilization.

There are many other choices for IDEs, but these make great choices, and more importantly all of them support the Language Server Protocol (LSP). LSP is a protocol developed by Microsoft that standardizes communication between a text editor or IDE and a language server that provides programming language-specific features. With LSP support, an IDE can offer helpful functionality such as auto-completion, go-to-definition, and error-checking for a variety of languages.

Ultimately, VSCode is probably the best choice for most use-cases. It has a robust plugin ecosystem, broad adoption, supports most programming languages, and is easy set up.

Package Management

Package management is an important part of software development. Without it, developers would be limited to only the features present in the standard library of a given programming language. A package manager is a tool that automates the process of installing, upgrading, configuring, and removing software packages. It's a way of bundling together code into reusable modules that can be easily distributed and used in other projects.

pip is the standard package manager for Python. It allows users to install and manage additional libraries and dependencies not distributed as part of the standard Python library. 

Conda is an open-source, cross-platform, package manager. It was created for Python programs but can package and distribute software for any language. Conda is the package manager in the Anaconda and Miniconda platforms, which are popular for scientific computing.

Most Python development is probably still using vanilla pip. However, developers can still encounter messy library and dependency issues using standard tools. Poetry handles packaging and dependency management, as well as virtual environments. Poetry has more features than are possible to cover here, but some of the major features that are useful to new developers include:

• Dependency management

• Separation of development dependencies(can define packages for linting/formatting etc without packaging them as a release dependency)

• Virtual environments

Poetry makes a compelling case as the recommended tool for package and dependency management. It makes more complex application development easier; the pyproject.toml and poetry.lock can be committed to version control and shared with other project developers, enabling reproducible builds and homogenous dependency chains.

The next section will cover intermediate tools, including virtual environments.

Intermediate Tools for Python Development 

Working with Virtual Environments 

The other components of good dependency management and a hygienic development environment is using Python virtual environments (virtualenvs). A Python virtual environment is an isolated environment in which the user can install packages without affecting the system's Python setup or other Python projects on the same machine. 

A modern software development organization is often tasked with maintaining several distinct, complex applications. A developer might be asked to make code changes to multiple applications over the course of a single project, or even day. With a virtualenv,  each locally cloned repo can have a completely sanitized, isolated Python dependency environment. While past tools like venv and virtualenv did not always provide the best user experience, Poetry now also offers integrated virtual environments. Users can  type poetry shell to activate a project-specific virtual environment. For most Python development scenarios, using the integrated virtualenv management of Poetry is the best choice, especially if it’s already being used for package management.

Linting, Formatting, and Type Checking

An important principle for productive development work is to cut down on “cognitive overhead”: how much thinking/logical bridging is required before work is done. Ensuring all developers for a given team or project share formatting and general coding standards is incredibly helpful in reducing that cognitive load. The Python ecosystem has several tools; different ones may work better or worse given the development environment and project requirements.

Here’s a basic linting and formatting configuration that can be easily shared and offers a lot of helpful functionality:

1. Git VCS-hosted python program (GitHub/GitLab or something that supports pre-commit hooks). 

2. Use poetry to install the following development dependencies:

   1. black - code formatter that automatically formats Python code to conform to the PEP 8 style guide.

   2. isort - sorts imports at the top of modules to a consistent format.

   3. mypy - Python supports type hints but does not enforce them. If a project utilizes type hints, mypy can provide a common set of checks

   4. ruff - this is a Python linter tool.

   5. tox - our tox image is also available on Github and Docker too.

   Note that the tools we use for linting are packaged in a Docker image and available on GithubTesting Libraries.

3. Install the pre-commit framework along with pre-commit hooks for each of these tools.

   1. Commit  the pyproject.toml and .pre-commit-config.yaml files to version control along with a README.md or CONTRIBUTING.md containing instructions for usage and initial environment setup.

Pre-commit hooks will enforce these checks before a commit can be applied. Implementing basic development standards can help significantly reduce the cognitive overhead associated with coding, allowing developers to focus more on solving problems and less on adhering to formatting standards.

Test-Driven-Development (TDD) is a somewhat polarizing concept, but tests can contribute a lot to the overall stability and anti-fragility of a software application, particularly as it grows more complex. It’s much easier to catch and fix bugs early in development versus when the code runs in production.

Inevitably, Python developers will encounter scenarios where tests will add a non-trivial amount of safety and improvement to debugging efficiency. Some of the more popular testing tools and frameworks include:

• unittest: Also known as PyUnit, this is the standard testing framework for Python, built into the Python standard library. It's heavily inspired by JUnit in Java, and it supports concepts like test cases, test suites, and test runners.

• pytest: Another popular Python testing module that allows developers to write simple, scalable tests. It has become popular for its simplicity and ease of use. Pytest supports fixtures, parameterized testing, and many plugins for things like parallel testing and advanced reporting.

• nose/nose2: Nose is a flexible test discovery and running module that extends unittest. It's no longer maintained, but it was influential and popular, and many of its features have been integrated into other tools like pytest. Nose2 is the successor to nose.

• mock: This is a library for simulating APIs, system components, and external dependencies. Mock is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.

Like the linting and formatting tools, these should be installed as development dependencies in Poetry. The best choice for most development environments will be pytest. It’s compatible with unittest (which is already in the standard library), and doesn’t require a lot of setup or boilerplate to achieve functional testing.

Advanced Tools for Python Development

Docker

Developers looking for additional tools to help build a productive development environment should look no further than a containerization platform such as Docker. Developers can use Docker or other container engines to offer an additional layer of isolation for dependencies, as well as to package a complete build artifact for production deployment.

After making local changes, developers can use Docker to bundle and launch a local copy of the app to validate functionality, getting immediate feedback on the changes without waiting for long test suites or CI tooling.

Other Tools and Programs to Help Productivity

These modules provide specific functionality that can be incredibly useful in a given development context:

• Pydantic - Data validation library. Developers can define strict (or lax) schemas around objects and data, and how they should be handled. Pydantic makes it much clearer and easier to manage complex data models across multiple applications and developers.

• Jupyter notebooks - Interactive development environment for Python that sees heavy use in machine learning, AI, data science etc… Can be a good environment for getting productive quickly with integrated documentation.

• FastAPI - Modern framework for building APIs in Python 3.7+ that takes advantage of the newer type-hinting system.

  If you want to know more about FastAPI, Divio's Site Reliability Engineer Lucy Linder (@derlin) wrote a complete tutorial on how to get started with FastAPI, Celery and Poetry. Check it out here.

Conclusion

Python is a great language across a variety of tasks, including complex applications, websites, APIs, machine learning, data science and artificial intelligence, and more. All the more impressive is that such a versatile language is also great for first-time programmers. However, getting a functional local environment set up can seem daunting, and the past user experience hasn’t always been a positive one. Fortunately, modern tools have helped improve it immensely. Having the solid foundation of a great development environment helps new programmers learn faster and experienced development teams be more productive.

Interested to learn more about how Python can be used with the Divio PaaS? Reach out now!

Stay informed! Join our LinkedIn and X/Twitter community to access exclusive insights and be the first to know about our latest blog posts.