Skip to content

Commit

Permalink
Merge pull request #361 from argonne-lcf/SearchEditSBX
Browse files Browse the repository at this point in the history
excluded Cooley, Theta, and ThetaGPU docs from search
  • Loading branch information
alcf-haritha authored Mar 11, 2024
2 parents 3e92c71 + 274629d commit 92e630b
Show file tree
Hide file tree
Showing 90 changed files with 450 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/cooley/compiling-and-linking/compiling-and-linking.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Compiling and Linking on Cooley
## Compilers and MPI
GNU compilers, version 4.4.7, are installed and are available in your default environment. To use version 4.8.1 of the GNU compilers instead, add ```+gcc-4.8.1 to .soft.cooley``` in your home directory.
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/cooley-overview.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Cooley System Overview

The primary purpose of Cooley is to analyze and visualize data produced on ALCF supercomputers. Equipped with state-of-the-art graphics processing units (GPUs), Cooley converts computational data into high-resolution visual representations. The resulting images, videos, and animations help users to better analyze and understand the data generated by ALCF supercomputers. Cooley can also be used for statistical analysis, helping to pinpoint trends in the simulation data. Additionally, the system is capable of preprocessing efforts, such as meshing, to assist users preparing for production simulations.
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/job-submission/job-and-queue-scheduling.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Queueing and Job Submission on Cooley
## Overview
Like our other computing resources, Cooley uses the Cobalt job scheduler.
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/performance-tools/darshan.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Darshan on Cooley
## Overview
Darshan is a lightweight I/O instrumentation library that can be used to investigate the I/O behavior of production applications. It records statistics, such as the number of files opened, time spent performing I/O, and the amount of data accessed by an application.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Remote Visualization on Cooley Using VNC
When running graphics applications on Cooley, it is best to use the client/server mode when available.

Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/programming-models/kokkos.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Kokkos on Cooley
## Overview
Kokkos implements a programming model in C++ for writing performance portable applications targeting all major HPC platforms. It provides abstractions for both parallel execution of code and data management. Kokkos is designed to target complex node architectures with N-level memory hierarchies and multiple types of execution resources. It can use OpenMP, etc as backend programming model. For more information please visit [https://github.com/kokkos/kokkos](https://github.com/kokkos/kokkos)
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/programming-models/opencl.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# OpenCL on Cooley
## Overview
OpenCL (Open Computing Language) is an open standard from the Khronos group that enables heterogeneous device programming (e.g. CPUS, GPUs, and FPGAs). Complete descriptions of the API, memory hierarchy, and usage can be found in OpenCL documentation and technical specifications found on the Khronos website.
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/programming-models/openmp.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# OpenMp on Cooley
## Overview
The OpenMP API is an open standard for parallel programming. The specification document can be found here: [https://www.openmp.org](https://www.openmp.org). The specification describes directives, runtime routines, and environment variables that allow an application developer to express parallelism (e.g. shared memory multiprocessing and device offloading). Many compiler vendors provide implementations of the [OpenMP specification](https://www.openmp.org).
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/programming-models/raja.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# RAJA on Cooley
## Overview
RAJA is a collection of C++ software abstractions, being developed at Lawrence Livermore National Laboratory (LLNL), that enable architecture portability for HPC applications. The overarching goals of RAJA are to:
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/software-and-libraries/jupyter-notebooks.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Jupyter Notebooks on Cooley
## Jupyter Notebooks on Cooley
Frequently, it's very useful for prototyping, analysis, or debugging to have an interactive session on Cooley using jupyter notebooks to debug your python scripts. The single node containers supported by the ALCF datascience group have jupyter and common python packages installed, and so it is possible to use jupyter notebooks with the production ML/DL software in an interactive way.
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/software-and-libraries/machine-learning-tools.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Machine Learning Tools on Cooley
Cooley is designed as a visualization cluster, because of the support for Nvidia GPUs. However, it is also possible to run machine learning workflows on Cooley.

Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/software-and-libraries/paraview-tutorial.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# ParaView Tutorial on Cooley
## Overview
This tutorial is intended to be a hands-on resource for users interested in learning the basic concepts of ParaView. The examples can easily be run on a laptop, using the example data set provided.
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/software-and-libraries/paraview.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Paraview on Cooley
The recommended way of running ParaView on Cooley is in client/server mode. This consists of running the ParaView client on your local resource, and the ParaView server (pvserver) on the Cooley visualization nodes. There are two ways to accomplish this, detailed below. In both cases, the ParaView client needs to first be installed on your local resource, and needs to match the version that you run on Cooley.

Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/software-and-libraries/singularity-cooley.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Singularity on Cooley
## Singularity on ALCF Resources
Singularity is a software container solution to enable fine grained control over application environments on a diverse set of hardware systems. For details on singularity, please see the Singularity [documentation](https://www.sylabs.io/guides/3.0/user-guide/index.html).
Expand Down
5 changes: 5 additions & 0 deletions docs/cooley/software-and-libraries/visit-on-cooley.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Visit on Cooley
## Getting Started

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Gromacs on ThetaGPU

## What is Gromacs?
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Compiling and Linking on ThetaGPU
## Overview
ThetaGPU has AMD processors on the service nodes (thetagpusn1,2) and AMD processors and NVIDIA A100 GPUs on the compute nodes [see overview page]. The service nodes can be used to create containers and launch jobs.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Continuous Integration on ThetaGPU
## Overview
Continuous Integration (CI) in software development is the practice of committing code changes regularly to a version control system and having automated processes perform build, test, package, and deploy activities. The key concepts of CI include high frequency, repeatability, and automation in order to realize increased quality and ease of delivery. The main goal CI aims to achieve is the elimination of build and deployment issues, which in turn improves development cycles, provides a timely feedback loop with developers, and results in higher quality deliverables with reduced development time.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Building Python Packages
To build Python packages for ThetaGPU, there are two options: build on top of a bare-metal build or build on top of (and within) a singularity container. Additionally, you can build a new container from NVIDIA's docker images.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Containers on Theta(GPU)

On Theta(GPU), container creation can be achieved by using Docker on your local machine as mentioned in [Example `Dockerfile`](../../../theta/data-science-workflows/containers/containers.md#example-dockerfile), or using a Singularity recipe file and building on a Theta(GPU) worker node.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Data Science Software Availability
On ThetaGPU, currently we support the major deep learning frameworks through two paths: Singularity containers, based off of NVIDIA's Docker containers, and through bare-metal source builds. The bare-metal builds are for TensorFlow 2.X PyTorch. TensorFlow 1.X is supported only via NVIDIA's containers at this time.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# DeepSpeed

The base `#!bash conda` environment on ThetaGPU comes with Microsoft's
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Distributed Training on ThetaGPU Using Data Parallelism
There are two schemes for distributed learning:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Running PyTorch with Conda
Be aware that these builds use CUDA and will not work on login nodes, which do not have CUDA installed as there are no GPUs.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Running TensorFlow with Conda
Be aware that these builds use CUDA and will not work on login nodes, which do not have CUDA installed as there are no GPUs.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# TensorBoard Instructions
If you are able to install TensorBoard on your local machine, it is often easiest to copy the requisite files from ALCF file systems (via `sftp`, `scp`, Globus, etc.) to your local machine and run a TensorBoard there.

Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/data-science-workflows/gpu-monitoring.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# GPU Monitoring
Each GPU on ThetaGPU hosts 8 A100 GPUs. You can see information about these GPUs via the command ```nvidia-smi```.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# GPU Node Queue and Policy

**Note:** Users will need an allocation on ThetaGPU to utilize the GPU nodes. Request for an allocation by filling out this form: [Allocation Request](https://accounts.alcf.anl.gov/#!/allocationRequest).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Python/C++ Code Interoperability
These are the steps to build code that has Python/C++ code interoperability.

Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/getting-started.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Getting Started on ThetaGPU
## References
In addition to the content below, here is a [getting started video](https://youtu.be/N0Xip1mUZY8) covering the basics of using ThetaGPU and a related video on Lustre File Striping Basics. This should help you get up and running quickly on the GPU nodes.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# ThetaGPU Machine Overview
ThetaGPU is an extension of Theta and is comprised of 24 NVIDIA DGX A100 nodes. Each DGX A100 node comprises eight NVIDIA A100 Tensor Core GPUs and two AMD Rome CPUs that provide 22 with 320 GB of GPU memory and two nodes with 640 GB of GPU memory (8320 GB aggregately) of GPU memory for training artificial intelligence (AI) datasets, while also enabling GPU-specific and -enhanced high-performance computing (HPC) applications for modeling and simulation

Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/performance-tools/darshan.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Darshan on ThetaGPU
## Overview
Darshan instrumentation on ThetaGPU is not automatically included into the binary like Theta. The user must set the ```LD_PRELOAD``` variable as part of running the job.
Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/performance-tools/nvidia-nsight.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# NVIDIA Nsight
## References
- [NVIDIA Nsight Systems Documentation](https://docs.nvidia.com/nsight-systems/)
Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/programming-models/kokkos.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Kokkos on ThetaGPU
## Overview
Kokkos implements a programming model in C++ for writing performance portable applications targeting all major HPC platforms. It provides abstractions for both parallel execution of code and data management. Kokkos is designed to target complex node architectures with N-level memory hierarchies and multiple types of execution resources. It can use OpenMP, etc as a backend programming model. For more information please visit https://github.com/kokkos/kokkos
Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/programming-models/openmp.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# OpenMP on ThetaGPU
## OpenMP threading on CPU
All the compilers available on ThetaGPU supports OpenMP threading.
Expand Down
5 changes: 5 additions & 0 deletions docs/theta-gpu/programming-models/raja.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# RAJA ThetaGPU
## Overview
RAJA is a collection of C++ software abstractions, being developed at Lawrence Livermore National Laboratory (LLNL), that enable architecture portability for HPC applications. The overarching goals of RAJA are to:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Job and Queue Scheduling on ThetaGPU
## Queues and Job Scheduling

Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/elpa.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# ELPA

## What is ELPA?
Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/gromacs.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Gromacs on Theta

## What is Gromacs?
Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/lammps.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# LAMMPS on Theta
## Overview
LAMMPS is a general-purpose molecular dynamics software package for massively parallel computers. It is written in an exceptionally clean style that makes it one of the most popular codes for users to extend and it currently has dozens of user-developed extensions.
Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/namd.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# NAMD

## What Is NAMD?
Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/nwchem.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# NWChem
NWChem is a parallel quantum chemistry mainly written in Fortran77 and that uses MPI and OpenMP for distributed and multicore computing. NWChem was designed to solve large-scale electronic structure calculations with Hartree-Fock, Density Functional Theory, and other wavefunction correlated methods. See full features at [http://www.nwchem-sw.org](http://www.nwchem-sw.org).

Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/qbox.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# Qbox on Theta
## What is Qbox?
Qbox is a C++/MPI scalable parallel implementation of first-principles molecular dynamics based on the plane-wave, pseudopotential formalism. As described on the Qbox website http://qboxcode.org/index.htm, Qbox is designed for operation on large parallel computers.
Expand Down
5 changes: 5 additions & 0 deletions docs/theta/applications-and-libraries/applications/qmcpack.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
search:
exclude: true
---

# QMCPACK on Theta
## Overview
QMCPACK is a modern, high-performance, open-source Quantum Monte Carlo (QMC) simulation code. Its main applications are electronic structure calculations of molecular, quasi-2D, and solid-state systems.
Expand Down
Loading

0 comments on commit 92e630b

Please sign in to comment.