In the interest of getting the word out to everyone, a RFC was suggested to help with that.
Starting Oct 1, all external SSH access into LDF services will require going through the new lsst-login nodes. These nodes are available for use now. This change centralizes NCSA DUO two-factor authentication (2FA) and minimizes the number of jump hosts necessary for accessing the LDF resources. These new login nodes replace the need to use NCSA cerberus or lsst-bastion jump hosts to access LDF environments.
The login nodes are lightweight servers intended primarily for access to other systems and to centralize the DUO 2FA. Other work, especially intensive work requiring high CPU/memory usage, long running jobs, storage IO, etc. should be performed on new devl nodes. These new devl nodes essentially replace the current dev nodes – they can be used for software development, new batch job submission, and longer running interactive work. The following new devl nodes will be available as follows:
- lsst-devl01.ncsa.illinois.edu (Dell Intel - available now)
- lsst-devl02.ncsa.illinois.edu (Dell Intel - available by Sep 14)
- lsst-devl03.ncsa.illinois.edu (Dell AMD Rome - available by Sep 14)
- Users are encouraged to submit batch jobs for work that requires significant resources (e.g. some historical data processing that happened on lsst-dev).
- Documentation for the updated batch resources at NCSA is available here: https://developer.lsst.io/services/batch.html
- HTCondor: The HTCondor DAC pool (available now) is available for general use. Users working on "formal data products" can also use the HTCondor Prod pool.
- Slurm: A new Slurm cluster has been built (also available now) and compute nodes will be slowly migrated to it throughout August and September. Jobs can be submitted to the new Slurm cluster from any of the new devl or condor submit nodes.
- lsst-dev nodes will be unavailable after Oct 1 according to the schedule below. Developers should instead use login or devl nodes. (dev03 will be turned off Sept 21, dev02 turned off Sept 28, dev01 turned off Oct 1.)
- lsst-dev-db MySQL/MariaDB server will be turned off on Oct 1.
- lsst-bastion01 server will be turned off on Oct 1. Use the lsst-login nodes instead.
- Legacy Slurm “verification cluster” will be turned off on Oct 1. It is being refreshed as the new general-use Slurm Cluster.