Jupyter Lab using Custom Anaconda Environment
Using Custom Conda Environments within Jupyter on Nero
Scenario:
In this example we want to use a custom built anaconda3 environment within Jupyter. Specifically, we want to use R 4.0 from within.
First, we create a custom anaconda 3 environment within our home folder under the “envs” directory. We will then install the desired packages, create a settings file for JupyterHub on Nero, and test it out.
1. SSH to Nero and load Anaconda 3 module
Using an ssh client, ssh to your GCP project using the following command, where
<sunetid> should be replaced by your actual SUNet ID, and <pi-sunetid> should be replaced by your pi’s SUNet ID.
$ ssh <sunetid>@<pi-sunetid>.compute.stanford.edu
$ module load anaconda/3
2. Create your custom environment on Nero.
Here, the name of the environment is custom-jupyter
$ conda create -p ~/envs/custom-jupyter python pip conda nodejs r-base
This environment will have the packages python pip conda nodejs r-base installed
3. Activate the new environment
$ source activate ~/envs/custom-jupyter
The prompt in your terminal will prepend (custom-jupyter) to indicate that you are in your custom Conda environment
4. Install additional Jupyter required packages and fixes
(custom-jupyter)$ pip install jupyterhub jupyterlab jupyter jsonschema jupyterlab-server
Once the packages are installed, create a symlink to use the labhub startup
(custom-jupyter)$ cd ~/envs/custom-jupyter/bin/
(custom-jupyter)$ ln -s jupyterhub-singleuser jupyter-labhub
5. Create Nero setting file ‘override’ to enable usage of custom environment.
If your Jupyter notebook is running, stop it using the Hub Control Panel (located under the File menu).
This file is recognized when you login to Nero and loads the Conda environment you specify under the ‘_CONDA_ROOT=’ setting. In this example we are using ~/envs/custom-jupyter; if you have set up a different environment name, remember to change the value of _CONDA_ROOT. Set the ‘_CONDA_ROOT=
(custom-jupyter)$ mkdir ~/.srcc/jupyter
(custom-jupyter)$ echo "export _CONDA_ROOT=~/envs/custom-jupyter" > ~/.srcc/jupyter/override
Note that you may have to create ~/.srcc if it does not exist.
(custom-jupyter)$ mkdir ~/.srcc/
6. Login to Nero’s Jupyter instance
In a web browser, go to your Nero GCP url (your PI can provide this).
You should now be placed in Jupyterlab using your custom conda environment - you may notice things look different, but you should now be able to launch notebooks, console, and Terminal.
Note: If your Jupyterlab instance was already running, you must stop it via “Hub, Control Panel, Stop My Server”. Then, click “Start My Server” to relaunch it. Then your custom conda environment will load.
Additional steps to get R 4.0 working:
To get the r-irkernel package for Jupyter Notebook usage (and the “shortcut” on Launcher page): Once in your JupyterLab session, open Terminal.
Launch Anaconda R by typing “R”.
Install irkernel with:
install.packages('IRkernel')
Select USA (OR) as the repo. (The Oregon repo is #73)
Things will install….then type:
IRkernel::installspec()
Restart your Jupyter Server once more. (File, Hub Control Panel)
R should now be available as a notebook type.
Note: In this newer version of JupyterLab, the Hub Control panel used for stopping and starting your server has moved to “File, Hub Control Panel”.
If any problems, or to go back to Nero’s default Jupyter set up (conda’s base environment)
SSH back into Nero, and change ~/.srcc/jupyter/override to override.bak
$ mv ~/.srcc/jupyter/override ~/.srcc/jupyter/override.bak
Stop your Jupyter instance via “File, Hub Control Panel”, and start it again.
You should then be placed back into the default Nero Jupyter base environment.
Continue with general Jupyter Lab Instructons