Secure Shell ssh

Installation

Secure shell is the exclusive way to get into the secure environment of the HWW machines. Commands like telnet, ftp as well as the r-cmds are NOT allowed and therefore are rejected by the firewall.

Warning: Access to HWW platforms via ssh will definitely be restricted to ssh V2 ONLY!

On HWW platforms the ssh software is installed. The user is responsible for installation of ssh on his platform.

The source of OpenSSH is available from

• HOME page of OpenSSH Alternatives for other platforms like MS Windows can also be found on this site.
  

For details on the licensing see the external link HOME page of OpenSSH

For details on the licensing see the copyright notice. Please read first the files README and INSTALL.

Please be aware to run the latest version of ssh client software. Most operating systems provide insallable packages and update mechanism.

Tips and hints

if you can't get a connection

this may have 3 causes:

1. Your workstation is not known by the firewall. Please contact Your HLRS contact person (adviser/Betreuer), or submit a bug-report to accounting via the Trouble Ticket Submission Form providing Your username and the static IP Address You want to access from.
2. The HWW platform, you're trying to connect to, is currently not running. Please check the appropriate status page for your platform.
3. Your ssh client is setuid root. Please remove setuid root bit or use -P (for ssh) resp. -L option (for scp).

If you encounter any problems while connecting with ssh, please use -v option to get more detailed messages from ssh.

Filetransfer without password (scp) using Protocol version 2

Login via ssh resp. Filetransfer via scp is also possible without having to enter a password. Authentification is done via a public/private key pair. The following steps have to be accomplished:

(system1 is the machine from which you logon to the target system system2)

    System1 > ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key ($HOME/.ssh/id_rsa): <RETURN>
    Enter passphrase (empty for no passphrase): <RETURN>
    Enter same passphrase again: <RETURN>
    Your identification has been saved in $HOME/.ssh/id_rsa.
    Your public key has been saved in $HOME/.ssh/id_rsa.pub.
    The key fingerprint is:
    2f:5c:18:e7:g0:3b:fb:52:g3:22:1d:3d:a4:cf:b1:57 $LOGNAME@$HOSTNAME

    System1 > cd $HOME/.ssh
    System1 > scp id_rsa.pub System2:sys1.pub
    System1 > chmod 700 $HOME/.ssh
    System1 > chmod 700 $HOME/.ssh/id_rsa

    System2 > cd $HOME/.ssh
    System2 > cat $HOME/sys1.pub >> authorized_keys
    System2 > chmod 700 $HOME/.ssh

The key generation with ssh-keygen has only to be done once. The public key in the file $HOME/.ssh/identity.pub can be used for as many systems as needed.

Further information regarding ssh can be found at external link http://www.cs.hut.fi/ssh/ or at external link DATA FELLOWS - F-Secure SSH A good alternative for Windows platforms is external link putty

ssh-x509 Certificate based ssh access

This access method is implemented and can be used on request. Following Systems provide this service:

• hazelhen.hww.de (Cray XC40)
• clr3fr1.hww.de (nehalem cluster)

Be aware you need a special version of ssh-client with x509 support. Detailed information on how to set up X.509 enabled ssh can be found on our webserver at X.509-SSH.

ssh tunnel

Due to security policies inside the HWW network (all HWW platforms), general internet access is not possible. For example to get access to software repositories (svn servers, git servers, etc.), a ssh tunnel could be useful.

W A R N I N G :

If you create an ssh tunnel, any user on this system is able to access this port. Please be aware of sensitive data on your local network. E.g. if you create an ssh tunnel to your local private software repository, any users on the remote server has access to this port and is able to use this repository!

SVN

Please find the following example to get access to a svn repository.

For Linux, Mac and Unix users:

Assume the svn repository is: svn.mplayerhq.hu listening on port 3690 (default svn port) and your username on the HWW platform hermit1.hww.de is hpchugo. Then on your local workstation login and create the tunnel using following commands:

 ssh -R7777:svn.mplayerhq.hu:3690 hpchugo@hermit1.hww.de

You will be asked for your password on hermit1 and get a login session. The local port on hermit1 into the created tunnel in this example is 7777. Please select a different number in the range of 5000-20000. This number must be unique for each user and each tunnel. Be aware on some platform of HWW (e.g. the cray system hermit1) round robin name service will cause multiple sessions on different login servers. The created tunnel is only available on the login server created by the ssh command above.

To use this tunnel you have to access the selected port (7777) on localhost of hermit1.hww.de:

 svn co svn://localhost:7777/ffmpeg/trunk ffmpeg

on windows using putty you have to open following dialogue:

by klick on the ADD button, the dialogue should look like:

To use this tunnel you have to access the selected port (7778) on localhost of hermit1.hww.de:

 svn co svn://localhost:7778/ffmpeg/trunk ffmpeg

For more information please check:

 man ssh

External links (Wikipedia & Co.): ssh-tunnel german ssh-tunnel wiki german ssh-tunnel wiki english

as a second example following commands are neccessary to transfer an external WEB page into the current working dirtory on the hermit1 system:

 ##   login and create the tunnel 
 ssh -R7890:ftp.scientificlinux.org:80 hpchugo@hermit1.hww.de
 ##  fetch one file from this server 
 wget http://localhost:7890/linux/scientific/6x/x86_64/iso/readme

for tunnelling svn+ssh connections:

If you use the svn+ssh protocol to check out files you have to use a bit more workarounds since you are already using ssh to do the tunneling.

A possible solution is to start a tunnel from your laptop to the frontend through the firewall and access the repository through this tunnel. Log in to the cluster and see if the port is free

 ssh cl3fr1.hww.de "netstat -l | grep 10022"

If it is in use, just pick another one for the following steps.

Make a tunnel from your laptop to the repository:

 ssh -R10022:your.svnserver.com:22 -N -t -x cl3fr1.hww.de 

Test the tunnel (this makes sure it works and that an entry of localhost:10022 will be added to your ssh known_hosts file such that ssh-agent works like a charm) by doing the following on the cluster:

   ssh -p 10022 username@localhost

(Note that the username is the username at the other end of the tunnel!)

Add a custom protocol to your .subversion/config file on the cluster. Under the heading [tunnels] add

tunnel_ssh = $TUNNEL_SSH ssh -p 10022

Maybe you have to transfer ssh-keys in order to be able to log in password-free. Be aware that there are other users on the same host who can use this tunnel as well, so be careful how much you open. Maybe it is a better idea to temporarily genearate a separate ssh-keypair which you throw away after you are done with the svn stuff.

Finally, you can test the connection, or simply check out your sources:

svn co svn+tunnel_ssh://localhost/repos/...

Git

Git supports four protocols to access remote repositories: Local, HTTP(S), SSH and git (cf. [1] for the respective pros and cons). The method required to use Git on the HWW-systems depends on the chosen protocol:

Local

no modifications of the standard procedure required in this case

HTTP(S)

Unfortunately, just using a SSH tunnel as with the SSH and git protocols is not sufficient in this case (cf. below for a detailed description of the reason). Instead, one has to connect via an additional SOCKS proxy on a machine that has unlimited access to the internet, e.g. your local machine.

In order to do so, establish a proxy by using a special feature of OpenSSH:

  ssh   -N   -D 1080   localhost

This will establish some kind of a "loopback" SSH connection from your local machine to itself which will not execute any command (-N) but act as an SOCKS proxy on port 1080 (-D 1080).

On a second shell, now login to the desired HWW-system and forward a port on the remote machine (e.g. 7777) to the port on your local machine where the newly established SOCKS proxy is listening on (1080):

  ssh   -R 7777:localhost:1080   <system-name>.hww.de

By doing so, you have a SOCKS proxy listening on port 7777 of the HWW-system. Hence you can use this proxy for accessing remote git repositories. Unfortunately, the default versions of git installed on the HWW-systems are not capable of doing this. You hence have to load an appropriate version first:

  module load tools/git

In order to use the proxy, you can now add "-c http.proxy='socks5://localhost:7777'" to your git commands, e.g.:

  git   -c http.proxy='socks5://localhost:7777'   clone http://github.com/llvm-mirror/llvm.git

In order to avoid typing this in every git call, you can also set the respective port to be used whenever git talks to a remote repository via HTTP by

  git config   --global http.proxy 'socks5://localhost:7777'

In order to use HTTPS instead of HTTP, just replace 'http' by 'https' in the above commands.

Description of the underlying problem: During interaction with the remote side, "normal" URLs (e.g. http://github.com/llvm-mirror/llvm.git) seem to be transferred to the client, which - however - can't be accessed from within the HWW-systems. Instead, the host needs to be replaced by the local endpoint of the tunnel pointing to this host (i.e. http://localhost:7777/llvm-mirror/llvm.git if we have a tunnel from localhost:7777 to github.com:80). This translation, however, can't be done by Git itself because it is not aware of accessing the remote side via an SSH tunnel. The described workaround hence uses a SOCKS proxy outside of the HWW-network in order to avoid this address translation. Unfortunately, "socks[*]://"-prefixes are only supported in libcurl versions >= 7.21.7 (cf. [2]). The default git versions on HWW-systems, however, are linked against older versions of libcurl. Hence, one has to use the module tools/git which provides libcurl 7.48.0 and Git 2.8.0 linked against these.

SSH

Let's assume we like to clone LLVM from GitHub with the SSH protocol, i.e. we like to access the repository git@github.com:llvm-mirror/llvm.git. In order to do so, establish a tunnel from a port (e.g. 7777) on the HWW-system to port 22 on the repositories host (github.com) by

  ssh   -R 7777:github.com:22   <system-name>.hww.de

and use the following syntax to access the repository:

  git clone ssh://git@localhost:7777/llvm-mirror/llvm.git

Keep in mind that the usage of SSH to transfer Git data is fully independent of the used SSH tunnel! The latter is rather utilized to host a further SSH connection to a repository somewhere on the internet.

git

Let's assume we like to clone LLVM from GitHub with the git protocol, i.e. we like to access the repository at git://github.com/llvm-mirror/llvm.git. In order to do so, establish a tunnel from a port (e.g. 7777) on the HWW-system to port 9418 on the repositories host (e.g. github.com) by

  ssh   -R 7777:github.com:9418   <system-name>.hww.de

and just replace the server in the original URL by localhost:7777, i.e.

 git clone git://localhost:7777/llvm-mirror/llvm.git

With all protocols, don't forget to re-establish the tunnel (and proxy, in case of HTTP(S)) when again attempting to access the remote repository!

pip (Python installer)

Note: You are probably not allowed to install Python modules system-wide. Hence, please use the flag "--user" in order to install the module in your home.

Python's module installer pip also requires access to HTTP-resources. Hence, a similar workaround as described for git can be deployed. However, the SOCKS-proxy included in OpenSSH is not sufficient in this case. So you have to use a HTTP-proxy like Squid instead. Below, we assume such a proxy to be already running on your local machine and listening on port 3128 (which is the default, at least for Squid).

Login to the desired HWW-system and forward a port on the remote machine (e.g. 7777) to port 3128:

  ssh   -R 7777:localhost:3128   <system-name>.hww.de

By doing so, you have a HTTP-proxy listening on port 7777 of the HWW-system. Hence you can use this proxy with pip by:

  pip --proxy http://localhost:7777 ...