Using cyclone

From gfi
Revision as of 12:46, 7 February 2022 by Csp001 (talk | contribs) (Add instructions for jupyter)

Connection and first steps

Use ssh to connect

ssh -Y <username>@cyclone.hpc.uib.no

or just simply:

ssh -Y cyclone.hpc.uib.no

if you're connecting from a computer where you are logged in under the same name. When you are connecting for the first time, you will be asked to verify a signature. Reply "yes" to this question.

Home directories

cyclone.hpc.uib.no shares home directories with the new hexagon cluster. These home directories are separate from your standard UiB home directory. The contents of your home directories will not be synced, so you will manually need to transfer your scripts to the new system.

How to access software

In contrast to the old skd-cyclone, most scientific software is only available through software modules. Modules need to be loaded manually before the respective software can be used. We use software modules, because they provide are a clean and handy way to offer conflicting software or software versions: Instead of updating software and potentially breaking user scripts and programs new software is then made available in parallel to those that require it.

Start-up scripts

It might be handy to automatically load modules for software that you use frequently every time you login to cyclone. To accomplish that for bash, there are two relevant files, $HOME/.bashrc and $HOME/.bash_profile . .bash_profile should contain things that need to be defined at login time only, like PATH and other environment variables, startup programs, etc. You just need things once, not in every shell you open. In most cases, you also need the things from .bashrc in your login shell. That's why .bash_profile sources .bashrc as well, but .bashrc doesn't usually source .bash_profile. In contrast, .bashrc should typically contain things you want to set in every shell you open, like aliases, functions, etc. These are per shell session items that are not inherited from environment (source and detailed explanation).

An example .bash_profile for inspiration

module load cdo/1.6.9
module load ncarg/6.3.0-login

alias rm="rm -i"
alias cp="cp -i"
alias erai="cd /Data/skd/share/Reanalysis/ERA_INTERIM/6HOURLY/"

if [ -f ~/.bashrc ]; then
   . ~/.bashrc
fi

An example .bash_profile for inspiration

alias rm="rm -i"
alias cp="cp -i"
alias erai="cd /Data/skd/share/Reanalysis/ERA_INTERIM/6HOURLY/"


Available software

Below is a list of frequently used software and which module to load on the new cyclone. Relevant module names are printed like for matlab/2015b. Modules can be loaded by module load <module name>, and unloaded module unload <module name>. More details on the module system.

To list all available modules, use the command

module avail

Hexagon-specific modules are now no longer listed and available on cyclone. All packages listed with this command should work, so please report to support@hpc.uib.no if they do not.

Data analysis and plotting

  • CDO
    CDO, CDO/1.9.5-foss-2018b
  • Dynlib
    See temporary solution for python. Requires in addition OpenBLAS/0.3.1-GCC-7.3.0-2.30 and ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1.
  • ghostview
    <no module required>
  • grads
    <no module required>
  • Matlab
    Matlab, Matlab/2018a, Matlab/2012a
  • ncdump and other netCDF utilities
    netCDF, netCDF/4.6.1-foss-2018b
  • NCL
    NCL, NCL/6.5.0
  • nco
    NCO, NCO/4.7.4-foss-2018b
  • ncview
    ncview, ncview/2.1.7-foss-2018b. Load only when needed! The package is currently known to break some GUI programs like gedit and gvim.
  • python, plus selection of scientific packages
    First choose and load the python module for the python version that you want to use, either Python/2.7.15-foss-2018b for python 2.7.15, or Python/3.7.0-foss-2018b for python 3.7.0.
    Then, as a temporary solution, many scientific python packages are installed in the following virtual environments:
    source /Data/gfi/users/local/share/virtualenv/dynpie2/bin/activate for python 2.7.15.
    source /Data/gfi/users/local/share/virtualenv/dynpie3-2021a/bin/activate for python 3.7.0.
    For the pygrib module, ecCodes/2.7.3-foss-2018b is required in addition.
  • R
    <no module required>

Text editors

  • emacs
    Is installed globally, but requires util-linux/2.32-GCCcore-7.3.0 to work.
  • gedit
    <no module required>
  • gvim
    <no module required>
  • vi(m)
    <no module required>

Compilers and libraries

  • blas
    OpenBLAS/0.3.1-GCC-7.3.0-2.30. Note: The library is called libopenblas, so you'll need to compile with -lopenblas.
  • eccodes (replacement for grib_api)
    ecCodes/2.7.3-foss-2018b
  • gcc, gfortran
    GCC, GCC/7.3.0-2.30
  • grib_api
    grib_api/1.24.0-foss-2018b
  • HDF5 libraries
    HDF5/1.10.2-foss-2018b
  • lapack
    ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1. Note: The library is called libscalapack, so you'll need to compile with -lscalapack.
  • MPI
    OpenMPI/3.1.1-GCC-7.3.0-2.30
  • netcdf
    For C-libaries: netCDF/4.6.1-foss-2018b,
    For Fortran-libraries netCDF-Fortran/4.4.4-foss-2018b, compile with option -I/shared/apps/netCDF-Fortran/4.4.4-foss-2018b/include
  • proj
    PROJ/5.0.0-foss-2018b

Jupyter & Jupyterhub

Here is a step by step guide to setup and use Jupyter based on the dynpie3 environment.

File:Get started cyclone.pdf


Accessing the data from your own machines

Note: From your personal machines you will need to be connected to the UiB VPN to be able to mount the following network drives.

On MacOS you can mount folders through Finder > Go > Connect to Server .... In the dialogue window that pops up, enter

smb://<user name>@leo.hpc.uib.no/gfi

as the Server Address. You might want to replace gfi by skd to access the skd disks if that is your primary affiliation.

On Ubuntu this is very similar: Files > Go > Enter Location .... Enter the address:

smb://<user name>@leo.hpc.uib.no/gfi

and enter your password in the next pop-up window, while keeping "UiB" in the "Domain" field.

Alternatively you can mount the network drive with sshfs to a dedicated directory using sshfs (install with sudo apt install sshfs):

mkdir ~/cyclone_gfi
From UiB network: sshfs -o follow_symlinks <user name>@cyclone.hpc.uib.no:/gfi/ ~/cyclone_gfi/
From ext. networks: sshfs -o follow_symlinks <user name>@sync.uib.no:/gfi/ ~/cyclone_gfi/
Unmount with: fusermount -u ~/cyclone_gfi

On Windows you can mount by navigating to File Explorer > Map network drive and enter \\leo.hpc.uib.no\gfi. Make sure to enable Connect using different credentials and fill in <user name>@uib.no and your PW.


Displaying GUIs on your screen

The ssh-login command at the top of this wiki enables trusted X11 forwarding (the -Y option), which allows you to display Graphical User Interfaces (GUIs) on your screen. This way you may be able to run software on a high performance system in a more userfriendly way than by just operating via the terminal. To make this work properly you may have to edit the ssh configuration file (/etc/ssh/ssh_config on linux systems) by adding or uncommenting the following two lines:

X11Forwarding yes
ForwardX11Trusted yes

Editing this file requires superuser priviliges so you may have to ask the IT department for support. For Mac the solution should be pretty similar, windows requires additional software like PuTTY (for establishing an ssh-connection) and an XServer (e.g. VcXsrv).