OSG School Materials

School Overview and Intro

Welcome Slides

View the slides: pdf

Introduction to High Throughput Computing

View the slides: pdf

Intro to HTCondor Job Execution

HTCondor: Introductory Slides

View the slides: pdf

Exercises: Intro to HTCondor

• Exercise 1.1: Log in to the local submit machine and look around
• Exercise 1.2: Experiment with HTCondor commands
• Exercise 1.3: Run jobs!
• Exercise 1.4: Read and interpret log files
• Exercise 1.5: Determining Resource Needs
• Exercise 1.6: Remove jobs from the queue
• Bonus Exercise 1.7: Explore condor_q
• Bonus Exercise 1.8: Explore condor_status

HTCondor Multiple Job Slides

View the Slides: pdf

Exercises: Running Many HTC Jobs

• Exercise 2.1: Work with input and output files
• Exercise 2.2: Use queue N, $(Cluster), and $(Process)
• Exercise 2.3: Use queue from with custom variables
• Bonus Exercise 2.4: Use queue matching with a custom variable

HTC worksheet

• Intro slides: pdf
• Worksheet: pdf or Google Drive

Software

Software Slides

View the Slides: pdf pptx

Software Exercises

• Exercise 1.1 - Run and Explore Apptainer Containers
• Exercise 1.2 - Use Apptainer Containers in OSPool Jobs
• Exercise 1.3 - Use Docker Containers in OSPool Jobs
• Exercise 1.4 - Build and Use an Apptainer Container
• Exercise 2.1 - Choose Software Options
• Exercise 2.2 - Finding Containers
• Exercise 2.3 - Apptainer Examples
• Exercise 2.4 - Apptainer Definition Files
• Exercise 2.5 - Example of Manual Installation
• Exercise 3.1 - Build Your Own Docker Container
• Exercise 4.1 - Build an HTC-Friendly Executable

The Open Science Pool (OSPool)

OSPool Slides

View the slides: pdf

Exercises: Researching the OSPool

• Exercise 1.1: Where Do Jobs Run?
• Exercise 1.2: How Much Can I Get?
• Exercise 1.3: How Does Capacity Change?
• Exercise 1.4: What Is In an Execution Point?
• Bonus Exercise 1.5: Viewing OSPool Information

Data

Data Slides

View the Slides: pptx

Data Exercises

• Exercise 1.1: Understanding a job's data needs
• Exercise 1.2: transfer_input_files, transfer_output_files, and remaps
• Exercise 1.3: Splitting input
• Exercise 2.1: OSDF for inputs
• Exercise 2.2: OSDF for outputs

Troubleshooting

Troubleshooting Slides

View the Slides: pdf ppt

Troubleshooting Exercises

• Exercise 1.1: Troubleshooting Jobs
• Exercise 1.2: Job Retry

Workflows with DAGMan

DAGMAn Slides

View the slides: pptx, pdf

DAGMan Exercises

• Exercise 1.1: Coordinating set of jobs: A simple DAG
• Exercise 1.2: A brief detour through the Mandelbrot set
• Exercise 1.3: A more complex DAG
• Exercise 1.4: Handling jobs that fail with DAGMan
• Exercise 1.5: Workflow Challenges

Biology Resources

Machine Learning

Machine Learning

View the slides: pptx, pdf

Research Computing Facilitation

Research Computing Facilitation

View the slides: pptx, pdf

Final Talks

Slides

• Philosophy: [Slides coming soon]
• Forward (Tim’s final talk): pdf