Skip to content

Wigner

Wigner site

The Wigner site extends the Slurm cluster with a dedicated shared storage volume and additional GPU compute capacity, connected to the main cluster over a private network. Access works the same way as the rest of the cluster: the user only interacts with the slurm-master controller node, which reaches the Wigner resources internally. No separate login or VPN is required.

Wigner shared storage

A dedicated SSD-backed NFS volume is mounted at /wigner_storage on the controller (slurm-master) and on the Wigner GPU nodes.

Property Value
Capacity 9 TB total (SSD)
Mount point (from the controller node) /wigner_storage
  • The volume is visible from the Wigner GPU nodes and the controller — files written on one are immediately available on the other.
  • Note: /wigner_storage is not mounted on the rest of the fleet (interactive_cpu_m2.large, batch_gpu_g2.2xlarge_32) — those partitions only see /storage, and /storage in turn is not mounted on the Wigner nodes. A job (or a Singularity --bind) that assumes the wrong path for its partition will fail. See the Overview section's Storage visibility note for the full picture.
  • It is the correct place for datasets, checkpoints, and job output that must outlive a single job.
  • It is not local scratch. High-throughput random I/O from many nodes at once shares a single NFS server and one network path. Please stage hot data to node-local disk instead — on the Wigner GPU nodes this is /tmp. Available space varies per node and isn't centrally published here, so check with df -h /tmp inside your job before copying large datasets.

Every user is provisioned with a personal folder at /wigner_storage/<username> Please, always store your data there rather than at the root of the volume. For copy and upload examples, see the List of useful Slurm commands section.

Storage quota
Storage on /wigner_storage is limited per user, separately from the quota on the main /storage volume described in the multi-user environment section:

  • soft limit: 100 GB
  • hard limit: 110 GB

Usage above the soft limit is only permitted for a grace period, after which writes will be refused until usage is brought back under the soft limit. To check current usage, use the quota wigner command described in the List of useful Slurm commands section.

Wigner GPU compute

Two dedicated NVIDIA A100 (40 GB) GPU nodes are available at the Wigner site through the batch_gpu_wigner_g2.2xlarge_40 partition (max walltime 1 day).

Note: The batch_gpu_wigner_g2.2xlarge_40 partition is currently running in beta, with access limited to a restricted set of users. It will be opened up to all users soon — until then, submissions from outside the beta group will be rejected.

Note: These nodes run an older NVIDIA driver / CUDA stack (CUDA 12.0) than the rest of the fleet. If you compile code against a specific CUDA version, keep this in mind when your job lands on a Wigner GPU node.

For more details on scheduling jobs against this partition, please visit the List of useful Slurm commands section.