Running Frontier Squid in a Container¶
Frontier Squid is a distribution of the well-known squid HTTP caching proxy software that is optimized for use with applications on the Worldwide LHC Computing Grid (WLCG). It has many advantages over regular squid for common distributed computing applications, especially Frontier and CVMFS. The OSG distribution of frontier-squid is a straight rebuild of the upstream frontier-squid package for the convenience of OSG users.
OSG recommends that all sites run a caching proxy for HTTP to help reduce bandwidth and improve throughput.
This document outlines how to run Frontier Squid in a Docker container.
Before starting the installation process, consider the following points (consulting the Frontier Squid Reference section as needed):
- Docker: For the purpose of this guide, the host must have a running docker service
and you must have the ability to start containers (i.e., belong to the
- Network ports: Frontier squid communicates on ports 3128 (TCP) and 3401 (UDP).
- We encourage sites to allow monitoring on port 3401 via UDP from CERN IP address ranges, 184.108.40.206/16, 220.127.116.11/17, 18.104.22.168/20 and 22.214.171.124/17. See the CERN monitoring documentation for additional details.
- If outgoing connections are filtered, note that CVMFS always uses ports 8000, 80, or 8080.
- Host choice: If you will be supporting the Frontier application at your site, review the upstream documentation to determine how to size your equipment.
Environment variables (optional)¶
In addition to the required configuration above (ports and file systems), you may also configure the behavior of your cache with the following environment variables:
|SQUID_IPRANGE||Limits the incoming connections to the provided whitelist.||By default only standard private network addresses are whitelisted.|
|SQUID_CACHE_DISK||Sets the cache_dir option which determines the disk size squid uses. Must be an integer value, and its unit is MBs. Note: The cache disk area is located at /var/cache/squid.||Defaults to 10000.|
|SQUID_CACHE_MEM||Sets the cache_mem option which regulates the size squid reserves for caching small objects in memory.||Defaults to "128 MB".|
Cache Disk Size
For production deployments, OSG recommends allocating at least 50 to 100 GB (50000 to 100000 MB) to SQUID_CACHE_DISK.
In order to preserve the cache between redeployments, you should map the following areas to persistent storage outside the container:
|Mountpoint||Description||Example docker mount|
|/var/cache/squid||This directory contains the cache for squid. See also SQUID_CACHE_DISK above.||-v /tmp/squid:/var/cache/squid|
|/var/log/squid||This directory contains the squid logs.||-v /tmp/log:/var/log/squid|
For more details, see the Frontier Squid documentation.
Configuration customization (optional)¶
More complicated configuration customization can be done by mounting
.awk files into /etc/squid/customize.d.
For details on the names and content of those files see the comments in the
and see the
on configuration customization.
Running a Frontier Squid Container¶
To run a Frontier Squid container with the defaults:
[email protected] $ docker run --rm --name frontier-squid \ -v <HOST CACHE PARTITION>:/var/cache/squid \ -v <HOST LOG PARTITION>:/var/log/squid \ -p <HOST PORT>:3128 opensciencegrid/frontier-squid:3.6-release
You may pass configuration variables in
KEY=VALUE format with either
-e options or in a file specified with
Running a Frontier Squid container with systemd¶
An example systemd service file for Frontier Squid.
This will require creating the environment file in the directory
This example systemd file assumes
<HOST PORT> is
<HOST CACHE PARTITION> is
<HOST LOG PARTITION> is
Create the systemd service file
/etc/systemd/system/docker.frontier-squid.service as follows:
[Unit] Description=Stash Cache Container After=docker.service Requires=docker.service [Service] TimeoutStartSec=0 Restart=always ExecStartPre=-/usr/bin/docker stop %n ExecStartPre=-/usr/bin/docker rm %n ExecStartPre=/usr/bin/docker pull opensciencegrid/frontier-squid:3.6-release ExecStart=/usr/bin/docker run --rm --name %n --publish 3128:3128 -v /tmp/squid:/var/cache/squid -v /tmp/log:/var/log/squid --env-file /opt/xcache/.env opensciencegrid/frontier-squid:3.6-release [Install] WantedBy=multi-user.target
Enable and start the service with:
[email protected] $ systemctl enable docker.frontier-squid [email protected] $ systemctl start docker.frontier-squid
Validating the Frontier Squid Cache¶
The cache server functions as a normal HTTP server and can interact with typical HTTP clients, such as
<HOST PORT> is the port chosen in the
docker run command,
3128 by default.
[email protected] $ export http_proxy=http://localhost:<HOST PORT> [email protected] $ wget -qdO/dev/null http://frontier.cern.ch 2>&1 | grep X-Cache X-Cache: MISS from 797a56e426cf [email protected] $ wget -qdO/dev/null http://frontier.cern.ch 2>&1 | grep X-Cache X-Cache: HIT from 797a56e426cf
Registering Frontier Squid¶
See the Registering Frontier Squid instructions to register your Frontier Squid host.
To get assistance, please use the this page.