Commit da091da1 authored by Willi Rath's avatar Willi Rath

Merge branch '27-rewrite-using-conda-init' into 'master'

Rewrite docs using conda init

Closes #27

See merge request !27
parents 5bf0019a ce59c0e1
# How to work with JupyterLab everywhere
>
> _This guide **will** walk you through the complete setup of JupyterLab and
> one or more scientific Python computing environments on your local computer
> or on a remote machine._
>
>
> _This guide **will not** explain how to use the shell, how to connect to
> remote machines via SSH, or how to make sure you have access to the relevant
> networks, e.g., via VPN._
>
>
> _Currently, **only Linux / Unix** and **MacOSX** are covered._
>
> - _This guide **will** walk you through the complete setup of JupyterLab and one or more scientific Python computing environments on your local computer or on a remote machine._
> - _This guide **will not** explain how to use the shell, how to connect to remote machines via SSH, or how to make sure you have access to the relevant networks, e.g., via VPN._
> - _Currently, **only Linux / Unix** and **MacOSX** are covered._
## Quick start guide
Clone this repository (only necessary once)
```bash
git clone https://git.geomar.de/python/jupyter_on_HPC_setup_guide.git
```
run
```bash
jupyter_on_HPC_setup_guide/scripts/remote_jupyter_manager.sh
```
and follow the instructions given. It installs a (hopefully) complete Python/Jupyterlab environment on any machine, and starts up Jupyterlab so you can start working immediately.
and follow the instructions given. It installs a (hopefully) complete Python/Jupyterlab environment
on any machine, and starts up Jupyterlab so you can start working immediately.
Please _**continue reading**_ to learn how to specify which software to use and to get a more detailed understanding about your computing environment.
Note that it is highly recommended to get a more detailed understanding on what the
above code actually does! So, plaese, continue reading ...
## Install the base environment and JupyterLab
>
> _This will show how to download the installer, install a minimal Python
> environment, and start JupyerLab. Execute the steps below on the computer
> where the calculation should be performed (e.g., your local computer or a
> computer in a remote computing centre)._
>
> _This will show how to download the installer, install a minimal Python environment, and start JupyerLab. Execute the steps below on the computer where the calculation should be performed (e.g., your local computer or a computer in a remote computing centre)._
### Download and install Miniconda3
>
> _(No matter if you want to work with Python 3, which you probably should, or
> Python 2, the **base environment should be Python 3**.)_
>
> Python 2 won't be supported beyond the end of 2019. So please make sure to **always use Python3** in all following steps.
To download the latest installer
......@@ -70,35 +45,35 @@ To install, run (on both, Linux and MacOSX):
bash Miniconda3.sh -b -p ${HOME}/miniconda3
```
(If this fails, check
<https://conda.io/docs/user-guide/install/#regular-installation>, make sure to
follow the instructions for Miniconda**3** and do _**not**_ make this
installation your standard Python.)
### Initialize your shell (only necessary after installation)
> This will ensure that `conda` is always available, but that no environment is activated automatically.
### Activate the base environment and install JupyterLab
Before continuing make sure you are in a bash shell by typing
To make conda available, run
```bash
echo $SHELL | grep bash || bash
${HOME}/miniconda3/bin/conda init bash
${HOME}/miniconda3/bin/conda config --set auto_activate_base false
```
log out and log back in.
To activate the base environment, run:
### Activate the base environment and install JupyterLab
Before continuing make sure you are in a bash shell by typing
```bash
source ${HOME}/miniconda3/bin/activate base
echo $SHELL | grep bash || bash
```
Then (and in the same terminal), install `jupyterlab` and `nb_conda_kernels`,
by running:
Then (and in the same terminal), install `jupyterlab` and `nb_conda_kernels`, to the `base` environment by running:
```bash
conda install jupyterlab nb_conda_kernels
conda install -n base jupyterlab nb_conda_kernels
```
### Start JupyterLab
After activating the base environment with
```bash
source ${HOME}/miniconda3/bin/activate base
conda activate base
```
run:
```bash
......@@ -114,48 +89,26 @@ Copy/paste this URL into your browser when you connect for the first time,
```
Copy the URL (from your shell!) to your web browser. You should see JupyterLab.
>
> _**Note 1:** At this point, you only have a minimal Python 3 installation with
> a functional JupyterLab frontend. To perform scientific analyses, you need to add one
> or more environments with a selection of scientific software._
>
> _**Note 1:** At this point, you only have a minimal Python 3 installation with a functional JupyterLab frontend. To perform scientific analyses, you need to add one or more environments with a selection of scientific software._
>
> _**Note 2:** If you have trouble connecting with a tunneled browser
> (see [Issue 21](https://git.geomar.de/python/jupyter_on_HPC_setup_guide/issues/21))
> you can use a different IP address. As details may differ from machine to machine,
> we can only list possible solutions here:_
> - _`${HOSTNAME}` or `${HOST}` will most probably work but might make your JupyterLab
> try to listen to the whole internet (which ususally is prevented by the firewall,
> but be aware of this and talk to someone experienced if you're not sure),_
> - to get a list of possible network interfaces, run `ifconfig` in the shell._
>
> _**Note 2:** If you have trouble connecting with a tunneled browser (see [Issue 21](https://git.geomar.de/python/jupyter_on_HPC_setup_guide/issues/21)) you can use a different IP address. As details may differ from machine to machine, we can only list possible solutions here:_
> - _`${HOSTNAME}` or `${HOST}` will most probably work but might make your JupyterLab try to listen to the whole internet (which ususally is prevented by the firewall, but be aware of this and talk to someone experienced if you're not sure),_
> - _to get a list of possible network interfaces, run `ifconfig` in the shell._
## Add a scientific Python environment and use it
>
> _This will show how to add a Python environment that can be used for scientific
> analyses._
>
> _This will show how to add a Python environment that can be used for scientific analyses._
### Add a scientific environment
To add a new environment, first activate the `base` environment:
```bash
source ${HOME}/miniconda3/bin/activate base
```
Then, create an environment (called `py3_std` in this example, and containing
Python 3, `numpy`, `matplotlib`, `scipy`):
```bash
conda create -n py3_std python=3 numpy matplotlib scipy ipykernel
```
>
> _(Do **always** install `ipykernel` into the environment, to make sure that
> JupyterLab recognizes the new environment as a kernel.)_
>
> _(Do **always** install `ipykernel` into the environment, to make sure that JupyterLab recognizes the new environment as a kernel.)_
### Use the environment in JupyterLab
......@@ -164,22 +117,12 @@ Above, we created a new environment `py3_std`. To use it, activate the `base`
environment and start JupyterLab. There, you should be able to create a new
notebook and choose the `py3_std` environment as its kernel.
>
> _At this point, you know how to set up a full Python installation including a
> JupyterLab front end in the `base` environment and one or more scientific
> computing environments that can be used in real scientific analyses. The
> instructions above work on any local or remote Linux / Unix or MacOSX machine.
> Below, you'll learn how to connect to JupyterLab if you installed Python on a
> remote machine that you can only connect to via SSH._
>
> _At this point, you know how to set up a full Python installation including a JupyterLab front end in the `base` environment and one or more scientific computing environments that can be used in real scientific analyses. The instructions above work on any local or remote Linux / Unix or MacOSX machine. Below, you'll learn how to connect to JupyterLab if you installed Python on a remote machine that you can only connect to via SSH._
## Connect to JupyterLab running on a remote machine
>
> _This will show how to connect to Jupyterlab if your web browser cannot
> directly access its address._
>
> _This will show how to connect to Jupyterlab if your web browser cannot directly access its address._
### Start Jupyterlab on a remote machine
......@@ -205,10 +148,8 @@ chromium to use this socket as a proxy server. (The next step provides a script
that bundles this and handles a few caveats like isolating the tunneled browser
session from your other activities on the internet.)
>
> _Note that instead of `54321`, we could have used any free non-privileged (number
> between 1024 and 65535) port._
>
### Wrapped in a script
......@@ -224,26 +165,19 @@ And run it to connect to `host.example.com`:
./run_chromium_through_ssh_tunnel.sh user@host.example.com
```
Note that these steps will only work, if `host.example.com` is a known host in
`.ssh/known_hosts`. (To test this, simply create a ssh connection to
`host.example.com`, if it is not yet a known host, you will be asked whether to
continue and if you do, `host.example.com` will be added to `.ssh/known_hosts`.)
Note that these steps will only work, if `host.example.com` is a known host in `.ssh/known_hosts`. (To test this, simply create a ssh connection to `host.example.com`, if it is not yet a known host, you will be asked whether to continue and if you do, `host.example.com` will be added to `.ssh/known_hosts`.)
### Connect your browser to the remote JupyterLab
Now, you can paste the URL (usually of the form
`http://localhost:8888/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`)
provided by JupyterLab into the tunneled browser.
Now, you can paste the URL (usually of the form `http://localhost:8888/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`) provided by JupyterLab into the tunneled browser.
## Appendix: Trouble Shooting
- If you get `ImportError: [...]: version GLIBC_[...] not found` errors on
relatively old machines, make sure to priorize the `defaults` channel over
`conda-forge`. This is done by specifying `-c defaults` **before**
`-c conda-forge` in `conda install` or `conda create` commands.
- If you get `ImportError: [...]: version GLIBC_[...] not found` errors on relatively old machines, make sure to priorize the `defaults` channel over `conda-forge`. This is done by specifying `-c defaults` **before** `-c conda-forge` in `conda install` or `conda create` commands.
- If your installation fails, check <https://conda.io/docs/user-guide/install/#regular-installation>, make sure to follow the instructions for Miniconda**3** and do _**not**_ make this installation your standard Python.
## Appendix: Jargon
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment