SSH Jump Host Trick

I once had dinner with Bill Dally (the Chief Scientist of Nvidia) at the Palais des Congrès Neptune, in Toulon, France. I asked him point blank

“Why does Apple refuse to put Nvidia’s superior GPUs into their macs?”

Throughout the course of the meal I got the whole story, and in the end it basically boiled down to Apple refusing to pay up. Because of this cheap decision, data scientists were condemned to years of inconvenience!

That is, most work must be on remote linux boxes. We can’t develop gpu-based deep learning models or test them locally, on our beloved macbooks (RocM is still too slow).

SSH'ing Through a Jump Host

SSH (Secure Socket sHell) provides a secure way to access another computer. To reduce the threats, modern compute clouds are hardened by so called Bastion or Jump hosts. On AWS when properly configured through the use of security groups and Network ACLs, the bastion essentially acts as a bridge to your private instances via the internet.

Unfortunately this setup, although more secure, can necessitate one to login with ssh credentials twice - once when connecting to the bastion and again when connecting to the target instance in the VPC. And this annoyance crops up anytime we need to use scp as well. However, there is a clever shortcut we can employ to save time.

The slick way of connecting to an instance guarded by a bastion is to simply add the following entries to your ~/.ssh/config file, changing your machine names, usernames, and private ip addresses as necessary:

Host bastion-host
    Hostname XX.XXX.XXX.XXX
    User <username>
    ForwardAgent yes
    Port 22

Host your-instance
    Hostname XX.X.XX.XX
    User ubuntu
    Port 22
    ProxyCommand ssh bastion-host 'ssh-add /usr/local/etc/keys/bastion-host && nc %h %p'

You can then ssh (or scp) with a single authentication to your instance:

ssh your-instance

The idea here is that the ProxyCommand entry automatically executes ssh commands on the remote host and forwards all traffic through. Of course, you must already have nc installed on the bastion-host.

I haven’t seen this idiom anywhere else on the internet, so I hope it is of some use to my fellow remote developers!