The problem of interpreter and dependency management is quite common, but also challenging.
brew in Ruby, ansible in Python.In order to run they need always need two things:
This boils down to the following problems. How to download and keep interpreters of different versions? How to download, keep and manage dependencies? How to include or not to include the interpreter and dependencies into a distribution?
Combined problems represent a “reproducible build” problem, which is, simply speaking, “works on my machine” statement when something went wrong.
Here is the idea of using a PATH variable. When you enter a command to the shell, it checks every directory in PATH variable, this means you can manipulate it, and add the directory containing the interpreter to the PATH dynamically. Want ruby version 3.3.3? No problem, download, build, add to the PATH and use it. It’s common practice to automate PATH management, for example rbenv checks for the file .ruby-version in the folder, and automatically adjusts PATH to include the proper directory with the corresponding interpreter.
It makes a bit more complicated if you use IDEs, they typically offer some built-in support for PATH management that may differ. Just keep in mind, PATH in shell and PATH in your editor depends on how you run the application.
Historically, dependency management was a separate problem from interpreter management. Though, some tools often try to target both cases.
For the interpreted language it means also solving a sub-problem of transient dependencies.
Why is the dependency management a problem at all? Just download the source of a library, copy it to somewhere so your app should find it when runs. Done.
The trade-off is typically is either to reuse the same dependencies, or to keep a copy. And the second big issue is how to update those. Keeping in mind, different libraries can depend on the same sources but different (maybe incompatible) versions.
Example of duplication, you may have heard about size of node_modules folder? pnpm tries to address this problem. Flip, Maven with Java ecosystem puts dependencies to .m2 folder and reuses them as much as possible.
Size is used to be the major trade-off here. But with the time and hardware evolution the direction has changed.
The hardware grows in capacity, storage and also networks became fast and capable. It’s often that the distribution includes entire Chrome browser and all dependencies packed together. For all platforms.
Before docker gained it’s popularity, using virtual machines for isolation also during the development was a common thing. Among the downsides can be named heavy resource requirements, slowness and difficulties with managing changes.
Interesting enough, but docker itself solves a problem of isolation. This means, you don’t need to support different versions of interpreters and libraries in the same system.
Docker image should contain only one version of interpreter and only one set of dependencies.
Though, with the raise of the idea of using docker container for developement, I often see tools for interpreter management also being a part of the docker image. Such as in devcontainers.
Another attempt to solve the problem of a “reproducible build” for all languages and dependencies in one shot is Nix or NixOS.
Nix represents a tools with own configuration language and packages for majority of languages and packages. Often associated with certain complexity, but highly worthy for investing efforts.