OSG Exercise 1.1: Log In to the OSPool Access Point¶
The main goal of this exercise is to log in to an Open Science Pool Access Point so that you can start submitting jobs into the OSPool. But before doing that, you will first prepare a file on Monday‘s Access Point to copy to the OSPool Access Point. Then you will learn how to efficiently copy files between the Access Points.
If you have trouble getting
ssh access to the OSPool Access Point, ask the instructors right away!
Gaining access is critical for all remaining exercises.
Part 1: On the PATh Access Point¶
The first few sections below are to be completed on
ap1.facility.path-cc.io, the PATh Access Point.
This is still the same Access Point you have been using since yesterday.
Preparing files for transfer¶
When transferring files between computers, it’s best to limit the number of files as well as their size. Smaller files transfer more quickly and, if your network connection fails, restarting the transfer is less painful than it would be if you were transferring large files.
Archiving tools (WinZip, 7zip, Archive Utility, etc.) can compress the size of your files
and place them into a single, smaller archive file.
tar command is a one-stop shop for creating, extracting, and viewing the contents of
Its usage is as follows:
To create a tarball named
<archive contents>, use the following command:
$ tar -czvf <archive filename> <archive contents>
<archive filename>should end in
<archive contents>can be a list of any number of files and/or folders, separated by spaces.
To extract the files from a tarball into the current directory:
$ tar -xzvf <archive filename>
To list the files within a tarball:
$ tar -tzvf <archive filename>
Comparing compressed sizes¶
You can adjust the level of compression of
tar by prepending your command with
<COMPRESSION> can be either
fast for the least compression, or
best for the most compression (the default
compression is between
While still logged in to
- Create and change into a new folder for this exercise, for example
wgetto download the following files from our web server:
taron each file and use
ls -lto compare the sizes of the original file and the compressed version.
Which files were compressed the least? Why?
Part 2: On the Open Science Pool Access Point¶
For many of the remaining exercises, you will be using an OSPool Access Point,
which submits jobs into the OSPool.
To log in to the OSPool Access Point,
use the same username (and SSH key, if you did that) as on
If you have any issues logging in to
please ask for help right away!
ssh in to the server and take a look around:
- Log in using
ssh [email protected](substitute your own username)
- Try some Linux and HTCondor commands; for example:
- Linux commands:
ls, and so on
- What is the operating system?
unameand (in this case)
- HTCondor commands:
- Linux commands:
In the next exercise, you will submit the same kind of job as in the previous exercise. Wouldn’t it be nice to copy the files instead of starting from scratch? And in general, being able to copy files between servers is helpful, so let’s explore a way to do that.
Using secure copy¶
Secure copy (
scp) is a command based on SSH
that lets you securely copy files between two different servers.
It takes similar arguments to the Unix
cp command but also takes additional information about servers.
Its general form is like this:
scp <source 1> <source 2>...<source N> [username@]<remote server>:<remote path>
<remote path> may be omitted if you want to copy your sources to your remote home directory
[username@] may be omitted if your usernames are the same across both servers.
For example, if you are logged in to
and wanted to copy the file
foo from your current directory
to your home directory on
and if your usernames are the same on both servers,
the command would look like this:
$ scp foo ap1.facility.path-cc.io:
Additionally, you could pull files from
The following command copies
bar from your home directory on
to your current directory on
and in this case, the username for
ap1 is specified:
$ scp [email protected]:bar .
Also, you can copy folders between servers using the
If you kept all your files from the HTCondor exercise 1.3 in a folder named
you could use the following command to copy them to your home directory on
$ scp -r [email protected]:htc-1.3 .
Secure copy to your laptop¶
During your research, you may need to transfer output files
from your submit server to inspect them on your personal computer,
which can also be done with
scp on your laptop, follow the instructions relevant to your computer‘s operating system:
Mac and Linux users¶
scp should be included by default and available via the terminal on both Mac and Linux operating systems.
WinSCP is an
scp client for Windows operating systems. Install WinSCP from https://winscp.net/eng/index.php
Once completed, move onto the next exercise: Running jobs in the OSG