Hanging git clone with exploding ssh window sizes
On Nesh, cloning the large repo https://git.geomar.de/data/ARGO/ quite persistently failed around Receiving objects: 81%
.
Problem
Following https://stackoverflow.com/a/13274606, I tried ping -i 0.7 git.geomar.de
while cloning and it seemed to help.
SSH debug-level 2 (-vv
) output showed that on the attempts that hung, the window size was constantly increased till SSH stopped reacting.
Reproduce
Steps to reproduce this:
# create destiation for logging
log_dir=$HOME/_log
mkdir -p $log_dir
# Clone ARGO repo, skip Git LFS, and modify ssh command to use
# higher debug level and time-stamped log file.
GIT_SSH_COMMAND="ssh -vv -E $log_dir/ssh_\`date -Ins\`.log" \
GIT_LFS_SKIP_SMUDGE=1 \
git clone git@git.geomar.de:data/ARGO.git
Stop / repeat until it hangs (S+
for the respective SSH command in ps haux | grep $USER
).
Then, the $HOME/_log/ssh_*.log
contain the level-2 ssh logs. Those Git sessions that hang will show an exploding window size.
Visualize
With the different _log/ssh_2017-^C-04T19\:47\:59\,285709796+0200.log
, look at:
grep adjust _log/ssh_2017-^C-04T19\:47\:59\,285709796+0200.log | cut -d ' ' -f 5 | grep -v adj | gnuplot -p -e 'plot "/dev/stdin"'