Example Text for Grant Proposal:
The optimization cluster has dell rack-mounted servers hosted in the lower level of the Wisconsin Institutes for Discovery in Madison, WI. The cluster allows Optimization Members to compute, submit and execute Condor jobs, and obtain files and software from the local disks and networked file systems. Each server is centrally managed, runs Linux, is secured behind a firewall, and is scanned frequently for security vulnerabilities.
DiscoverIT services are described on the Discovery KnowledgeBase page on Applying for Grants
Every Optimization Cluster machine:
- is in the .discovery.wisc.edu domain unless noted
- mounts software in /progs
- mounts a building-wide DataVault space under /mnt/dv/wid for shared project files
- has local disk space available in /scratch that is NOT backed up
- runs Hyper-Threaded cores and displays twice the number of physical cores as processors
Visualizing Usage / Available Hosts
The DiscoverIT Ganglia page displays current and historical Optimization Cluster usage at https://ganglia.discovery.wisc.edu/?c=Optimization
| Machine Name | Chassis | Memory | Processors | Purchased |
|---|---|---|---|---|
| opt-a001 | Dell R820 | 256 | 40 Cores: 4 2.20G E5-4640 Xeon | 2014-06 |
| opt-a002 | Dell R740 | 768 | 36 Cores: 2 3.1G Xeon Gold 6254 | 2019-09 |
| opt-a003 | Dell R740 | 768 | 36 Cores: 2 3.1G Xeon Gold 6254 | 2019-09 |
| opt-a004 | Dell R740 | 768 | 36 Cores: 2 3.1G Xeon Gold 6254 | 2019-09 |
| opt-a005 | Dell R820 | 256 | 40 Cores: 4 2.20G E5-4640 Xeon | 2014-06 |
| opt-a006 | Dell R910 | 256 | 40 Cores: 2 2.0G E7-4850 Xeon | 2013-03 |
| opt-a007 | Supermicro SYS-421GE-TNRT | 1055.98 | 128 Cores: 2 4.1G 6448Y Xeon Gold, 1 NVIDIA H100 80G PCIe | 2023-08 |
| submit1.wid.wisc.edu | Virtual Host | 16 |
Cluster Software
Email support@discovery.wisc.edu if you need additional software installed.
Local Python Libraries: Cython, Ipython, numexpr, scipy, pytables, matplotlib
Licensed Software in /progs: ampl, CPLEX, Gams, Gurobi, MATLAB
NOTE: Additional software is available in /mnt/ws/progs
CPLEX Environment Variables:
export ILOG_LICENSE_FILE=/progs/CPLEX_Studio/access.ilm
PATH=${PATH}:/progs/CPLEX_Studio/cplex/bin/x86-64_sles10_4.1
Gurobi Environment Variables:
(Gurobi is currently (12/15/15) at v6.5)
export GUROBI_HOME="/progs/gurobi/linux64"
export PATH="${PATH}:${GUROBI_HOME}/bin"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${GUROBI_HOME}/lib"
export GRB_LICENSE_FILE="/progs/gurobi/gurobi.lic"
Condor
CHTC provides an introductory page on running condor jobs at: http://chtc.cs.wisc.edu/helloworld.shtml
Condor jobs to the Discovery Condor Pool should be started from opt-submit.discovery.wisc.edu. Add the following line to your submit file to get the highest priority for your job:
+group = "WID" +WIDsTheme = "Optimization"
If you need to utilize all the resources on an optimization machine, you are able to turn off condor by running:
/usr/sbin/condor_off -peaceful
. If you need to stop all currently running condor jobs NOW, you can leave off -peaceful but that may kill very long running jobs. Once you are done with your work, please remember to run:
/usr/sbin/condor_on
See also: Discover Compute Cluster, WID Compute Servers