Create Jupyter Notebook that runs everywhere

How to embed “requirements.txt” in your Jupyter Notebooks and make it run everywhere

There is a popular proverb in software development — “It works on my machine…” It is no different in Data Science since it is just one aspect of software development. I sometimes think you do more coding in data science due to its interactive nature than software development, but I digress.

Let's say you created a Jupyter notebook and you want to share your notebook with your colleague in the same organization or even different organizations, you will eventually run into the problem of “It works on my machine….”. The same applies when you use services like Google Colab, Google Cloud Vertex AI, Amazon SageMaker, Deepnote, Azure Machine Learning, Databricks Notebooks, etc.; you will run into the same issue. You can resolve the issue by providing “requirements.txt” with all dependencies, but it is just another file and some people do not know what to do with it.

Before I start, let’s get logistics out of the way. Since I need an image for this article, I have embedded this picture from Unsplash.

Photo by Kelly Sikkema on Unsplash

The solution is simple — just embed “requirements.txt” in Jupyter Notebooks by using its magic commands.

On top of your Jupyter Notebook before import statement, create two code cells with the following where the first code cell writes “requirements.txt” file in your development environment with the following output in your notebook => “Writing requirements.txt”.

%%writefile requirements.txt

pandas==1.3.5
matplotlib==3.2.2
jax==0.3.8
numpyro==0.9.2
statsmodels==0.10.2

The second code cell executes the command to install python package dependencies included in “requirements.txt” file.

!pip install -r requirements.txt

This should either install python package dependencies with its associated version or state “Requirement already satisfied”.

Now every time you run your notebook, it should install all dependencies, enabling your notebook to run everywhere, assuming you have listed all your dependencies.

I hope this article is helpful. If you have any questions, please leave comments and feedback here.