.computecanada.ca
}}
Then, copy and paste the first provided HTTP address into your web browser. In the above salloc
example, this would be:
http://node_name.int.cluster.computecanada.ca:8888/lab?token=101c3688298e78ab554ef86d93a196deaf5bcd2728fad4eb
=== From Windows ===
An [[SSH tunnelling|SSH tunnel]] can be created from Windows using [[Connecting_with_MobaXTerm | MobaXTerm]] as follows. Note: this procedure also works from any terminal that supports the ssh
command.
- Once JupyterLab is launched on a compute node (see [[#Starting_JupyterLab|Starting JupyterLab]]), you can extract the
hostname:port
and the token
from the first provided HTTP address. For example:
http://node_name.int.cluster.computecanada.ca:8888/lab?token=101c368829...2728fad4eb
└────────────────────┬────────────────────┘ └──────────┬──────────┘
hostname:port token
- Open a new Terminal tab in MobaXTerm. In the following command, replace
with its corresponding value (refer to the above figure), replace
with your Alliance account username, and replace
with the cluster on which you have launched JupyterLab:{{Command2
|prompt=[name@local ~]$
|ssh -L 8888: @.computecanada.ca
}}
- Open your web browser and go to the following address where
must be replaced with the alphanumerical value extracted from the above figure:
http://localhost:8888/?token=
== Shutting down JupyterLab ==
You can shut down the JupyterLab server before the walltime limit by pressing Ctrl-C twice in the terminal that launched the interactive job.
If you have used MobaXterm to create an SSH tunnel, press Ctrl-D to shut down the tunnel.
= Adding kernels =
It is possible to add kernels for other programming languages, for a different Python version or for a persistent virtual environment that has all required packages and libraries for your project. Refer to [http://jupyter-client.readthedocs.io/en/latest/kernels.html Making kernels for Jupyter] to learn more.
The installation of a new kernel is done in two steps:
# Installation of the packages that will allow the language interpreter to communicate with the Jupyter interface.
# Creation of a file that will indicate to JupyterLab how to initiate a communication channel with the language interpreter. This file is called a kernel spec file, and it will be saved in a subfolder of ~/.local/share/jupyter/kernels
.
In the following sections, we provide a few examples of the kernel installation procedure.
== Julia Kernel ==
Prerequisites:
# The configuration of a Julia kernel depends on a Python virtual environment and a kernels
folder. If you do not have these dependencies, make sure to follow the first few instructions listed in [[#Python_kernel|Python kernel]] (note: no Python kernel required).
# Since the installation of Julia packages requires an access to the internet, the configuration of a Julia kernel must be done in a [[SSH|remote shell session on a login node]].
Once you have a Python virtual environment available and activated, you may configure the Julia kernel:
- Load the [[Julia]] module:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|module load julia
}}
- Install IJulia:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|echo -e 'using Pkg\nPkg.add("IJulia")' {{!}} julia
}}
- Important: start or restart a new JupyterLab session before using the Julia kernel.
For more information, see the [https://github.com/JuliaLang/IJulia.jl IJulia documentation].
=== Installing more Julia packages ===
As in the above installation procedure, it is required to install Julia packages from a login node, but the Python virtual environment could be deactivated:
- Make sure the same Julia module is loaded:{{Command2
|module load julia
}}
- Install any required package. For example with
Glob
:{{Command2
|echo -e 'using Pkg\nPkg.add("Glob")' {{!}} julia
}}
- The newly installed Julia packages should already be usable in a notebook executed by the Julia kernel.
== Python kernel ==
In a terminal with an active session on the remote server,
you may configure a [[Python#Creating_and_using_a_virtual_environment|Python virtual environment]] with all the required [[Available_Python_wheels|Python modules]]
and a custom Python kernel for JupyterLab.
Here are the initial steps for the simplest Jupyter configuration
in a new Python virtual environment:
- If you do not have a Python virtual environment, create one. Then, activate it:
- Start from a clean Bash environment (this is only required if you are using the Jupyter Terminal via [[JupyterHub]] for the creation and configuration of the Python kernel):{{Command2
|env -i HOME{{=}}$HOME bash -l
}}
- Load a Python module:{{Command2
|module load python
}}
- Create a new Python virtual environment:{{Command2
|virtualenv --no-download $HOME/jupyter_py3
}}
- Activate your newly created Python virtual environment:{{Command2
|source $HOME/jupyter_py3/bin/activate
}}
- Create the common
kernels
folder, which is used by all kernels you want to install:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|mkdir -p ~/.local/share/jupyter/kernels
}}
- Finally, install the Python kernel:
- Install the
ipykernel
library:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|pip install --no-index ipykernel
}}
- Generate the kernel spec file. Replace
with a name that will uniquely identify your kernel:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|python -m ipykernel install --user --name --display-name "Python 3.x Kernel"
}}
- Important: start or restart a new JupyterLab session before using the Python kernel.
For more information, see the [http://ipython.readthedocs.io/en/stable/install/kernel_install.html ipykernel documentation].
=== Installing more Python libraries ===
Based on the Python virtual environment configured in the previous section:
- If you are using the Jupyter Terminal via [[JupyterHub]], make sure the activated Python virtual environment is running in a clean Bash environment. See the above section for details.
- Install any required library. For example,
numpy
:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|pip install --no-index numpy
}}
- The newly installed Python libraries can now be imported in any notebook using the
Python 3.x Kernel
.
== R Kernel ==
Prerequisites:
# The configuration of an R kernel depends on a Python virtual environment and a kernels
folder. If you do not have these dependencies, make sure to follow the first few instructions listed in [[#Python_kernel|Python kernel]] (note: no Python kernel required).
# Since the installation of R packages requires an access to [https://cran.r-project.org/ CRAN], the configuration of an R kernel must be done in a [[SSH|remote shell session on a login node]].
Once you have a Python virtual environment available and activated, you may configure the R kernel:
- Load an R module:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|module load r/4.1
}}
- Install the R kernel dependencies (
crayon
, pbdZMQ
, devtools
) - this will take up to 10 minutes, and packages should be installed in a local directory like ~/R/x86_64-pc-linux-gnu-library/4.1
:{{Command2
|prompt=(jupyter_py3) [name@server ~]$
|R --no-save
|result=> install.packages(c('crayon', 'pbdZMQ', 'devtools'), repos{{=}}'http://cran.us.r-project.org')
}}
- Install the R kernel.{{Command2
|prompt=>
|devtools::install_github(paste0('IRkernel/', c('repr', 'IRdisplay', 'IRkernel')))
}}
- Install the R kernel spec file.{{Command2
|prompt=>
|IRkernel::installspec()
}}
- Important: Start or restart a new JupyterLab session before using the R kernel.
For more information, see the [https://irkernel.github.io/docs/ IRkernel documentation].
=== Installing more R packages ===
The installation of R packages cannot be done from notebooks because there is no access to CRAN.
As in the above installation procedure, it is required to install R packages from a login node, but the Python virtual environment could be deactivated:
- Make sure the same R module is loaded:{{Command2
|module load r/4.1
}}
- Start the R shell and install any required package. For example with
doParallel
:{{Command2
|R --no-save
|result=> install.packages('doParallel', repos{{=}}'http://cran.us.r-project.org')
}}
- The newly installed R packages should already be usable in a notebook executed by the R kernel.
= Running notebooks as Python scripts =
For longer run or analysis, we need to submit a [[Running_jobs#Use_sbatch_to_submit_jobs|non-interactive job]]. We then need to convert our notebook to a Python script, create a submission script and submit it.
1. From the login node, create and activate a [[Python#Creating_and_using_a_virtual_environment|virtual environment]], then install nbconvert if not already available.
{{Command
|prompt=(venv) [name@server ~]$
|pip install --no-index nbconvert
}}
2. Convert the notebook (or all notebooks) to Python scripts.
{{Command
|prompt=(venv) [name@server ~]$
|jupyter nbconvert --to python mynotebook.ipynb
}}
3. Create your submission script, and submit your job.
In your submission script, run your converted notebook with:
python mynotebook.py
and submit your non-interactive job:
{{Command
|sbatch my-submit.sh
}}
= References =