Thoughts on automated jupyter lab management script
(Thanks @sebastian-wahl for triggering this. I like the idea of having all these tasks in a single script!)
A few thoughts on the automation in scripts/run_chromium_through_ssh_tunnel.sh.
-
Pass functions by SSH rather than using heredocs. -
Re-use discovering jupyter lab. -
Clean up chrome user dirs after shutdown. Use mktemp -d
rather than explicit path. -
Specify envs rather than hard-code packages.
Pass functions by SSH rather than using heredocs.
According to https://stackoverflow.com/a/22107893 this
function get_hostname () { echo ${HOSTNAME}; }
ssh <remote_host> "$(typeset -f get_hostname); get_hostname;"
is equivalent to
ssh <remote_host> <<EOF
echo \${HOSTNAME}
EOF
Re-use discovering jupyter lab
What about
function discover_jl_port () {
page=$(${HOME}/miniconda3/bin/python \
${HOME}/miniconda3/bin/jupyter notebook list" \
| grep http | awk '{print$1}')
port=$(echo $page | awk -F: '{print$3}' | awk -F/ '{print$1}')
}
if [[ "$task" == "start" ]] || [[ "$task" == "connect" ]]; then
port=$(ssh $hname "$(typeset -f discover_jl_port); discover_jl_port")
if [[ -z "$port" ]] ; then
…
elif [[ "$task" == "stop" ]]; then
port=$(ssh $hname "$(typeset -f discover_jl_port); discover_jl_port")
if [[ ! -z "$port" ]] ; then
…
Clean up temporary chrome user dir
(Also an issue for the other script.)
tmp_user_data=$(mktemp -d)
chromium-browser --new-window \
--user-data-dir="${tmp_user_data}" \
--proxy-server="socks5://localhost:${socks_5_port}" $page
rm -rf ${tmp_user_data}
Specify envs rather than hard-code packages
Currently, ae3db2fb790b6bb7f8f47604d7bc4f49d3f8c609/scripts/remote_jupyter_manager.sh#L116ff. hard-codes one python env. This should be generalized to allow for easier specification of envs.
With this py3_std.yml
,
name: py3_std
channels:
- defaults
- conda-forge
dependencies:
- numpy
- matplotlib
- scipy
- ipykernel
- git
- python=3
- cdo
- netcdf4
- xarray
- libiconv
- cartopy
the following installs a py3_std
env that is identical to the hard-coded one:
function _conda_create () {
source ${HOME}/miniconda3_20180525/bin/activate base;
conda env create -f /dev/stdin;
}
cat py3_std.yml | ssh $hname "$(typeset -f _conda_create); _conda_create;"