OSG Networking Area
Welcome to OSG Networking ! This is an entry point for those interested in Networking in OSG/WLCG or for those OSG/WLCG users experiencing network problems. It provides an overview of the networking goals, plans and various activities and subtopics underway regarding networking in the Open Science Grid (OSG) and World-wide LHC Computing Grid (WLCG), operated as a joint project. This area started in June 2012 with initial focus on the network monitoring as monitoring is critical to provide needed visibility into existing networks and site connectivity. OSG is working to provide needed networking information and tools for users, sites and experiments/VOs.
This documentation is divided into several sub-sections, each covering a specific area of activities.
Network Monitoring in WLCG and OSG (perfSONAR)
WLCG and OSG jointly operate a network of
perfSONAR agents deployed world-wide, which provides an open platform that can be used to baseline network performance and debug any potential issues. The following subsections provide details on the motivation, deployment and operations of the perfSONARs in WLCG/OSG:
- Motivation - overview, core concepts, motivation
- Deployment Guide - deployment models and options, hardware requirements
- Installation and Administration Guide - installation, configuration and maintanance
- Frequently Asked Questions
Users with network issues should check the troubleshooting link below for initial guidance on how best to get their issue resolved. In addition, you can refer to the ESNet network performance guide for a detailed instructions on how to identify and isolate network performance issues using perfSONAR.
OSG operates an advanced platform to collect, store, publish and analyse the network monitoring data it gathers from perfSONAR and other locations. All measurements are collected and available via streaming or through APIs. The following services are available:
- perfSONAR infrastructure monitoring - monitors state of perfSONAR network and reports on availability of core services
- OSG Distributed Network Datastore - distributed datastore based on ElasticSearch holding all the network measurements and providing an API to expose them via JSON is available at two locations (University of Chicago and University of Nebraska).
- OSG pSConfig Web Admin (PWA) - centralized configuration of the tests performed by the OSG/WLCG perfSONAR infrastructure . In case you'd like to start/manage particular mesh, please contact our support channels to get access.
- OSG Dashboards - set of dashboards showing an overview of the network state as seen by the perfSONAR infrastructure (NOTE: this instance is being deprecated and we plan to introduce dashboards that will replace MaDDash over the coming 2023-2024 year).
- WLCG Dashboards) - set of dashboards showing WLCG and OSG network performance by combining multiple sources of data including perfSONAR, FTS, ESNet/LHCOPN traffic, etc.
University of Chicago has set up an analytics platform using
Kibana4 as well as
Jupyter that can be used to access and analyse all the existing network measurements.
Support and Feedback
If you suspect a network problem and wish to follow up on it, we have a number of tools available. We have a ToolkitInfo page that can help you find resources to identify and explore problems. In general, networks problems are best resolved by opening a ticket with your site's network provider (see https://osg-htc.org/networking/network-troubleshooting/). If you want WLCG/OSG specific support, please open a ticket with the appropriate support unit: For
OSG sites please open a ticket with GOC; For
WLCG sites please open a GGUS ticket to
WLCG Network Throughput support unit. If you'd like to get help in setting up a WLCG/OSG perfSONAR instance please open a ticket with GOC or via GGUS to WLCG perfSONAR support. If you have problems or questions specific to perfSONAR, please email the perfSONAR user mailing list. For any other requests or to provide feedback, please open a ticket at GGUS and mention OSG networking.
- ESNet network performance tuning and debugging https://fasterdata.es.net/
- perfSONAR toolkit is part of the perfSONAR project.
- OSG/WLCG mesh configuration interface is available at https://psconfig.opensciencegrid.org
- OSG dashboard instance https://psmad.opensciencegrid.org/maddash-webui/index.cgi (NOTE: deprecated)
- OSG perfSONAR infrastructure monitoring https://psetf.opensciencegrid.org/etf/check_mk/
- OSG Analytics platform https://atlas-kibana.mwt2.org/s/networking/app/kibana
- WLCG dashboards https://monit-grafana-open.cern.ch/d/000000523/home?orgId=16
- SAND (an previous OSG Satellite project) focusing on analyzing perfSONAR and related network metrics: https://sand-ci.org