README.md 6.28 KB
Newer Older
Willi Rath's avatar
Willi Rath committed
1
# How to work with JupyterLab everywhere
Willi Rath's avatar
Willi Rath committed
2

Willi Rath's avatar
Willi Rath committed
3 4 5 6 7 8 9
_This guide **will** walk you through the complete setup of JupyterLab and
one or more scientific Python computing environments on your local computer
or on a remote machine.  Currently, only Linux / Unix and MacOSX is covered._

_This guild **will not** explain how to use the shell, how to connect to 
remote machines via SSH, or how to make sure you have acces to the relevant
networks, e.g., via VPN._
10

Willi Rath's avatar
Willi Rath committed
11

Willi Rath's avatar
Willi Rath committed
12
## Install the base environment and JupyterLab
Willi Rath's avatar
Willi Rath committed
13 14

_This will show how to download the installer, install a minimal Python
15 16 17 18
environment, and start JupyerLab.  Execute the steps below on the computer
where the calculation should be performed (e.g., your local computer or a
computer in a remote computing centre)._

Willi Rath's avatar
Willi Rath committed
19

20 21 22
### Download and install Miniconda3

_(No matter if you want to work with Python 3, which you probably should, or
23
Python 2, the **base environment should be Python 3**.)_
24

25
To download the latest installer
26

27
- on Linux, run:
28 29 30 31
  ```bash
  curl https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o Miniconda3.sh
  ```

32
- on MaxOSX, run:
33 34 35 36 37 38 39 40 41 42 43
  ```bash
  curl https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o Miniconda3.sh
  ```

To install, run (on both, Linux and MacOSX):
```bash
bash Miniconda3.sh -b -p ${HOME}/miniconda3
```

(If this fails, check
<https://conda.io/docs/user-guide/install/#regular-installation>, make sure to
Willi Rath's avatar
Willi Rath committed
44
follow the instructions for Miniconda**3** and do _**not**_ make this
45 46 47
installation your standard Python.)


Willi Rath's avatar
Willi Rath committed
48
### Activate the base environment and install JupyterLab
49 50 51 52 53 54 55 56 57 58 59 60 61

To activate the base environment, run:
```bash
source ${HOME}/miniconda3/bin/activate base
```

Then (and in the same terminal), install `jupyterlab` and `nb_conda_kernels`,
by running:
```bash
conda install jupyterlab nb_conda_kernels
```


Willi Rath's avatar
Willi Rath committed
62
### Start JupyterLab
63

64 65 66 67 68
After activating the base environment with
```bash
source ${HOME}/miniconda3/bin/activate base
```
run:
69 70 71 72 73 74 75 76 77 78 79 80
```bash
jupyter lab --no-browser
```

This will print a few lines to your screen and finally show something like:
```
Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

```

Willi Rath's avatar
Willi Rath committed
81
Copy the URL (from your shell!) to your web browser.  You should see JupyterLab.
82

83 84 85 86 87 88 89 90 91 92
---
---

_**Note** that at this point, you only have a minimal Python 3 installation with
a functional JupyterLab frontend.  To perform scientific analyses, you need to add one
or more environments with a selection of scientific software._

---
---

Willi Rath's avatar
Willi Rath committed
93 94 95 96 97 98 99

## Add a scientific Python environment and use it

_This will show how to add a Python environment that can be used for scientific
analyses._


100 101 102 103 104 105 106 107 108 109 110 111 112 113
### Add a scientific environment

To add a new environment, first activate the `base` environment:
```bash
source ${HOME}/miniconda3/bin/activate base
```

Then, create an environment (called `py3_std` in this example, and containing
Python 3, `numpy`, `matplotlib`, `scipy`):
```bash
conda create -n py3_std python=3 numpy matplotlib scipy ipykernel
```

_(Do **always** install `ipykernel` into the environment, to make sure that
114
JupyterLab recognizes the new environment as a kernel.)_
115 116


Willi Rath's avatar
Willi Rath committed
117
### Use the environment in JupyterLab
118 119

Above, we created a new environment `py3_std`.  To use it, activate the `base`
Willi Rath's avatar
Willi Rath committed
120
environment and start JupyterLab.  There, you should be able to create a new
121 122
notebook and choose the `py3_std` environment as its kernel.

123 124 125 126 127 128 129 130 131 132 133 134 135
---
---

_At this point, you know how to set up a full Python installation including a 
JupyterLab front end in the `base` environment and one or more scientific
computing environments that can be used in real scientific analyses. The
instructions above work on any local or remote  Linux / Unix or MacOSX machine.
Below, you'll learn how to connect to JupyterLab if you installed Python on a
remote machine that you can only connect to via SSH._

---
---

136

Willi Rath's avatar
Willi Rath committed
137
## Connect to JupyterLab running on a remote machine
Willi Rath's avatar
Willi Rath committed
138 139 140 141 142

_This will show how to connect to Jupyterlab if your web browser cannot
directly access its address._


Willi Rath's avatar
Willi Rath committed
143 144
### Start Jupyterlab on a remote machine

145
After installing Jupyter and your Python environments on a remote machine,
146
follow the instructions on how to [Start JupyterLab](#start-jupyterlab) above.
147 148
But before pasting the URL provided by Jupyter into your browser, make sure to
follow the next steps.
Willi Rath's avatar
Willi Rath committed
149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179


### The essential steps

To let your browser see the network like it looks from a remote machine
(called `host.example.com` here), run the following:
```bash
ssh -f -D localhost:54321 user@host.example.com sleep 15
chromium-browser --proxy-server="socks5://localhost:54321"
```

This will open an SSH session that provides a local network socket tunneling
all traffic through `host.example.com` and then tell chromium to use this
socket as aproxy server.  (The next step provides a script that bundles this and
handles a few caveats like isolating the tunneled browser session from your
other activities on the internet.)


### Wrapped in a script

Download the script (only needed once):
```bash
curl https://git.geomar.de/python/jupyter_on_HPC_setup_guide/raw/master/scripts/run_chromium_through_ssh_tunnel.sh -O
chmod a+x run_chromium_through_ssh_tunnel.sh
```

And run it to connect to `host.example.com`:
```bash
./run_chromium_through_ssh_tunnel.sh user@host.example.com
```

180 181 182 183 184
Note that these steps will only work, if `host.example.com` is a known host in 
`.ssh/known_hosts`. (To test this, simply create a ssh connection to 
`host.example.com`, if it is not yet a known host, you will be asked whether to 
continue and if you do, `host.example.com` will be added to `.ssh/known_hosts`.)

Willi Rath's avatar
Willi Rath committed
185

Willi Rath's avatar
Willi Rath committed
186
### Connect your browser to the remote JupyterLab
187

188 189 190
Now, you can paste the URL (usually of the form
`http://localhost:8888/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`) 
provided by JupyterLab into the tunneled browser.
191 192


Willi Rath's avatar
Willi Rath committed
193 194 195 196 197
## Appendix: Jargon

- `conda` is the package manager of the [Anaconda Python
  distribution](https://en.wikipedia.org/wiki/Anaconda_(Python_distribution))

198 199 200 201
- [conda environments](https://conda.io/docs/user-guide/tasks/manage-environments.html)
  are collections of packages / versions that make it possible to switch
  between different sets of software.

Willi Rath's avatar
Willi Rath committed
202 203
- [JupyterLab](https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906) is
  an integrated web-based computing environment