Installing Conda on a HPC Cluster

This guide explains how to install the latest version of Miniconda3 in your project space on an HPC cluster.

Step 1: Start an Interactive Session

Before installation, start an interactive session on a compute node:

srun --mem=16G --time=10:00:00 -n 8 --pty /bin/bash 

Step 2: Prepare and Run the Installation Script

Update the script below by replacing the grant ID pl0000-01 with your own. The script installs the latest version of Miniconda3 into your project space.

Create a file named install_miniconda.sh:

#!/bin/bash
# Grant ID – replace with your actual grant
GRANT_ID="pl0000-01"

# Temporary directories
export TMP="$HOME/$GRANT_ID/scratch/$USER/tmp"
export TMPDIR=$TMP
export TEMP=$TMP
export TEMPDIR=$TMP
mkdir -p "$TMP"

# Destination path for Miniconda3 installation
SOFT_PATH="/soft/miniconda3"
REAL_PATH=$(realpath "$HOME/$GRANT_ID/project_data")
MINICONDA_DIR="$REAL_PATH$SOFT_PATH"

# Diagnostics
echo "REAL_PATH=$REAL_PATH"
echo "SOFT_PATH=$SOFT_PATH"
echo "MINICONDA_DIR=$MINICONDA_DIR"

# Create target directory
mkdir -p "$MINICONDA_DIR" || { echo "Failed to create directory $MINICONDA_DIR"; exit 1; }

# Download latest Miniconda installer
INSTALLER_PATH="$TMP/miniconda_latest.sh"
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O "$INSTALLER_PATH" || { echo "Failed to download Miniconda"; exit 1; }

# Check if installer was downloaded
if [ ! -f "$INSTALLER_PATH" ]; then
    echo "Miniconda installer not found at $INSTALLER_PATH"
    exit 1
fi

# Unset Python path
unset PYTHONPATH

# Run the installer (non-interactive)
bash "$INSTALLER_PATH" -b -u -p "$MINICONDA_DIR" || { echo "Miniconda installation failed"; exit 1; }

# Remove installer after installation
rm -f "$INSTALLER_PATH"

# Verify installation
if [ ! -f "$MINICONDA_DIR/bin/conda" ]; then
    echo "Conda binary not found! Installation failed."
    exit 1
fi

# Initialize conda for bash shell
"$MINICONDA_DIR/bin/conda" init bash || { echo "Conda initialization failed"; exit 1; }

echo "✅ Miniconda installed successfully at $MINICONDA_DIR"

Important

Remember to replace pl0000-01 with your actual grant ID in the installation script before running it.

Step 3: Execute the Script

Save the script to a file, make it executable, and run it:

chmod +x install_miniconda.sh
./install_miniconda.sh

TIP

After installation, log out and log back in, or open a new shell to enable Conda initialization.

Step 4: Create a Conda Environment with Python

Check available Python versions:

conda search python

Example output:

python                        3.12.8      pkgs/main
python                        3.13.0      pkgs/main
python                        3.13.1      pkgs/main

Create a new environment with a selected version (e.g., 3.13.1):

conda create -n my_env python=3.13.1

Activate the environment:

conda activate my_env

Verify the installed Python version:

python --version
# Output: Python 3.13.1