Commit 64798d07 authored by Willi Rath's avatar Willi Rath

Merge branch 'rewrite-more-verbose' into 'master'

Simplify text and formatting

See merge request !29
parents 52dac51c 6a22adb3
# How to work with JupyterLab everywhere
> - _This guide **will** walk you through the complete setup of JupyterLab and one or more scientific Python computing environments on your local computer or on a remote machine._
> - _This guide **will not** explain how to use the shell, how to connect to remote machines via SSH, or how to make sure you have access to the relevant networks, e.g., via VPN._
> - _Currently, **only Linux / Unix** and **MacOSX** are covered._
This guide **will** walk you through the complete setup of JupyterLab and one or more scientific Python computing environments on your local **Linux** or **MacOSX** computer or on a remote **Linux/Unix** machine. This guide **will not** explain how to use the shell, how to connect to remote machines via SSH, or how to make sure you have access to the relevant networks, e.g., via VPN.
## Quick start guide
## Quick start
Clone this repository (only necessary once)
```bash
......@@ -16,48 +14,43 @@ jupyter_on_HPC_setup_guide/scripts/remote_jupyter_manager.sh
```
and follow the instructions given. It installs a (hopefully) complete Python/Jupyterlab environment on any machine, and starts up Jupyterlab so you can start working immediately.
---
Please _**continue reading**_ to learn how to specify which software to use and to get a more detailed understanding about your computing environment.
## Install the base environment and JupyterLab
> _This will show how to download the installer, install a minimal Python environment, and start JupyerLab. Execute the steps below on the computer where the calculation should be performed (e.g., your local computer or a computer in a remote computing centre)._
### Download and install Miniconda3
This will show how to download the installer, install a minimal Python environment, and start JupyerLab on any **Linux/Unix** or **MacOSX** system.
> Python 2 won't be supported beyond the end of 2019. So please make sure to **always use Python3** in all following steps.
Execute the steps below on the computer where the calculation should be performed (e.g., your local computer or a computer in a remote computing centre).
To download the latest installer
- on Linux, run:
```bash
curl https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o Miniconda3.sh
```
### Download, install and initialize Miniconda3 (only necessary once)
- on MaxOSX, run:
```bash
curl https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o Miniconda3.sh
```
_As Python 2 won't be supported beyond the end of 2019, please make sure to **always use Python3** in all following steps._
To install, run (on both, Linux and MacOSX):
Download latest Miniconda3 on **Linux/Unix**:
```bash
bash Miniconda3.sh -b -p ${HOME}/miniconda3
curl https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o Miniconda3.sh
```
### Initialize your shell (only necessary after installation)
> This will ensure that `conda` is always available, but that no environment is activated automatically.
Or download the latest Miniconda3 on **MacOSX**:
```bash
curl https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o Miniconda3.sh
```
To make conda available, run
Then, to install Miniconda3 to your Home directory, initialize conda and make sure that you don't override any Python installation from your operating system, run:
```bash
bash Miniconda3.sh -b -p ${HOME}/miniconda3
${HOME}/miniconda3/bin/conda init bash
${HOME}/miniconda3/bin/conda config --set auto_activate_base false
```
log out and log back in.
After installation and initialization, log out and log back in again.
### Activate the base environment and install JupyterLab
### Install JupyterLab to the base environment
Before continuing make sure you are in a bash shell by typing
```bash
......@@ -69,71 +62,49 @@ Then (and in the same terminal), install `jupyterlab` and `nb_conda_kernels`, to
conda install -n base jupyterlab nb_conda_kernels
```
### Start JupyterLab
### Add a scientific Python environment
After activating the base environment with
At this point, there only is a minimal Python environment called `base` that contains `conda` and JupyterLab. The following explains how to add a Python environment that can be used for scientific analyses.
Use `conda` to create an environment (called `py3_std` in this example, and containing Python 3, `numpy`, `matplotlib`, `scipy`):
```bash
conda activate base
conda create -n py3_std python=3 numpy matplotlib scipy ipykernel
```
run:
Make sure to _**always** install `ipykernel` into the environment_, to make sure that JupyterLab recognizes the new environment as a kernel.)
An opinionated recommendation for a more complete computing environment [can be found in the appendix](#appendix-recommended-environment).
### Use the environment in JupyterLab
Above, we created a new environment `py3_std`. To use it, activate the `base`
environment and start JupyterLab:
```bash
conda activate base
jupyter lab --no-browser --ip 127.0.0.1
```
This will print a few lines to your screen and finally show something like:
If you installed Python to the computer you're sitting in front of, this will automatically start a browser and connect to JypyterLab. If this does not happen, copy the URL given at the end of the output of the above commands into your browser:
```
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://127.0.0.1:8888/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
```
Copy the URL (from your shell!) to your web browser. You should see JupyterLab.
> _**Note 1:** At this point, you only have a minimal Python 3 installation with a functional JupyterLab frontend. To perform scientific analyses, you need to add one or more environments with a selection of scientific software._
> _**Note 2:** If you have trouble connecting with a tunneled browser (see [Issue 21](https://git.geomar.de/python/jupyter_on_HPC_setup_guide/issues/21)) you can use a different IP address. As details may differ from machine to machine, we can only list possible solutions here:_
> - _`${HOSTNAME}` or `${HOST}` will most probably work but might make your JupyterLab try to listen to the whole internet (which ususally is prevented by the firewall, but be aware of this and talk to someone experienced if you're not sure),_
> - _to get a list of possible network interfaces, run `ifconfig` in the shell._
## Add a scientific Python environment and use it
> _This will show how to add a Python environment that can be used for scientific analyses._
### Add a scientific environment
Then, create an environment (called `py3_std` in this example, and containing
Python 3, `numpy`, `matplotlib`, `scipy`):
```bash
conda create -n py3_std python=3 numpy matplotlib scipy ipykernel
```
> _(Do **always** install `ipykernel` into the environment, to make sure that JupyterLab recognizes the new environment as a kernel.)_
> _An opinionated recommendation for a computing env [can be found here](#appendix-recommended-environment)._
### Use the environment in JupyterLab
Above, we created a new environment `py3_std`. To use it, activate the `base`
environment and start JupyterLab. There, you should be able to create a new
notebook and choose the `py3_std` environment as its kernel.
> _At this point, you know how to set up a full Python installation including a JupyterLab front end in the `base` environment and one or more scientific computing environments that can be used in real scientific analyses. The instructions above work on any local or remote Linux / Unix or MacOSX machine. Below, you'll learn how to connect to JupyterLab if you installed Python on a remote machine that you can only connect to via SSH._
In JupyterLab, you should be able to create a new notebook and choose the `py3_std` environment as its kernel.
## Connect to JupyterLab running on a remote machine
> _This will show how to connect to Jupyterlab if your web browser cannot directly access its address._
At this point, you know how to set up a full Python installation including a JupyterLab front end in the `base` environment and one or more scientific computing environments that can be used in real scientific analyses. The instructions above work on any local or remote Linux / Unix or MacOSX machine. Below, you'll learn how to connect to JupyterLab if you installed Python on a remote machine that you can only connect to via SSH.
### Start Jupyterlab on a remote machine
After installing Jupyter and your Python environments on a remote machine,
follow the instructions on how to [Start JupyterLab](#start-jupyterlab) above.
But before pasting the URL provided by Jupyter into your browser, make sure to
follow the next steps.
After installing Jupyter and your Python environments on a remote machine, follow the instructions on how to [Use the environment in JupyterLab](#use-the-environment-in-jupyterlab) above. But before pasting the URL provided by Jupyter into your browser, make sure to follow the next steps.
### The essential steps
......@@ -144,18 +115,12 @@ ssh -f -D localhost:54321 user@host.example.com sleep 15
chromium-browser --proxy-server="socks5://localhost:54321"
```
This will open an SSH session that provides a local network socket listening
on port `54321` that tunnels all traffic through `host.example.com` and then tell
chromium to use this socket as a proxy server. (The next step provides a script
that bundles this and handles a few caveats like isolating the tunneled browser
session from your other activities on the internet.)
> _Note that instead of `54321`, we could have used any free non-privileged (number
> between 1024 and 65535) port._
This will open an SSH session that provides a local network socket listening on port `54321` that tunnels all traffic through `host.example.com` and then tell chromium to use this socket as a proxy server. _(Note that instead of `54321`, we could have used any free non-privileged (number between 1024 and 65535) port.)_
### Wrapped in a script
The following provides a script that bundles this and handles a few caveats like isolating the tunneled browser session from your other activities on the internet.
Download the script (only needed once):
```bash
curl https://git.geomar.de/python/jupyter_on_HPC_setup_guide/raw/master/scripts/run_chromium_through_ssh_tunnel.sh -O
......@@ -164,15 +129,10 @@ chmod a+x run_chromium_through_ssh_tunnel.sh
And run it to connect to `host.example.com`:
```bash
./run_chromium_through_ssh_tunnel.sh user@host.example.com
./run_chromium_through_ssh_tunnel.sh user@host.example.com <URL-from-JupyterLab>
```
Note that these steps will only work, if `host.example.com` is a known host in `.ssh/known_hosts`. (To test this, simply create a ssh connection to `host.example.com`, if it is not yet a known host, you will be asked whether to continue and if you do, `host.example.com` will be added to `.ssh/known_hosts`.)
### Connect your browser to the remote JupyterLab
Now, you can paste the URL (usually of the form `http://localhost:8888/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`) provided by JupyterLab into the tunneled browser.
Note that these steps will only work, if `host.example.com` (or whatever the real address of the remote computer) is a known host in `.ssh/known_hosts`. (To test this, simply create an SSH connection to `host.example.com`, if it is not yet a known host, you will be asked whether to continue and if you do, `host.example.com` will be added to `.ssh/known_hosts`.)
## Appendix: Trouble Shooting
......@@ -229,14 +189,13 @@ If you want to install from Git repositories, you'd also need access to
### How to check if I'm blocked
You can use netcat to see if you can connect to a given port on a given host. To, e.g., tell if you can connect to `repo.continuum.io` via HTTPS (port `443`), run
> ```shell
> nc -v -z repo.continuum.io 443
> ```
```shell
nc -v -z repo.continuum.io 443
```
which should return something like
> ```
> Connection to repo.continuum.io 443 port [tcp/https] succeeded!
> ```
```
Connection to repo.continuum.io 443 port [tcp/https] succeeded!
```
### How to get access?
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment