Wednesday Exercise 2.4: Using Docker¶

In this exercise, you will run the same Python script as the previous exercises, but using a Docker container.

Background¶

Containers are another way to manage software installations. We don't have the time to go fully into the details of building and using containers, but can use a publicly available Python container to run our Python job.

One caveat for using containers: not all systems will support them. HTCondor has built-in features for using Docker and many Open Science Grid resources have Singularity installed, but they are still not in widespread use.

Submit File Changes¶

Make a copy of your submit file from the previous exercise.
Add the following three lines to the submit file or modify existing lines to match the lines below:
```
universe = docker
docker_image = python:3.7.0-stretch
requirements = (OpSysMajorVer == 7)
```
Here we are requesting HTCondor's Docker universe and using a pre-built python image that, by default, will be pulled from a public website of Docker images called DockerHub. The requirements line will ensure that we run on computers whose operating system can support Docker.
Adjust the executable and arguments lines. The executable can now be the Python script itself, with the appropriate arguments:
```
executable = fib.py
arguments = 90
```
Finally, we no longer need to transfer a Python tarball (whether source code or pre-built) or our Python script. You can remove both from the transfer_input_files line of the submit file.

Python Script¶

Open the Python script and add the following line at the top:
```
#!/usr/bin/env python3
```
This will ensure that the script uses the version of Python that comes in the Docker container.

Once these steps are done, submit the job.