Changes between Initial Version and Version 1 of User Guide/Portal/SSH

Timestamp:

Mar 30, 2026, 5:53:57 PM (6 hours ago)

Author:

editor

Comment:

New: SSH Access sub-page

User Guide/Portal/SSH

@@ +1 @@
+[[Include(WikiToC)]]
+= SSH Access to Testbed Nodes =
+
+Once you have an approved reservation and your SSH public key uploaded to the portal, you can connect to testbed nodes using SSH. This page explains the testbed's SSH access model, connection patterns, tunneling techniques, and configuration tips.
+
+----
+
+== Access Model ==
+
+The COSMOS/ORBIT testbed uses a '''two-hop SSH access model''':
+
+1. '''Console server (jump host)''' — you first SSH into the console server for the testbed domain you reserved. You authenticate with your '''portal username''' and your '''SSH key'''.
+2. '''Testbed node''' — from the console server, you hop to individual testbed nodes. On baseline images, you log in as '''root'''. The console server automatically has root SSH access to all nodes in its domain.
+
+This two-hop model exists because testbed nodes are on internal networks (10.x.x.x addresses) that are not directly reachable from the Internet. The console servers act as secure gateways.
+
+'''Important terminology:'''
+* Your '''portal username''' (e.g., `jsmith`) is used on '''console servers'''
+* The '''root''' account is used on '''testbed nodes''' when running baseline images
+* If you load a custom image, the node's SSH configuration depends on how you configured that image
+
+----
+
+== Console Servers ==
+
+Each testbed domain has its own console server. The console server hostname follows the pattern `console.<domain>` or simply `<domain>`:
+
+||= Domain =||= Console Server =||= Location =||
+|| grid.orbit-lab.org || console.grid.orbit-lab.org || ORBIT Main Grid (400 nodes) ||
+|| sb1.orbit-lab.org || console.sb1.orbit-lab.org || ORBIT Sandbox 1 ||
+|| sb2.orbit-lab.org || console.sb2.orbit-lab.org || ORBIT Sandbox 2 ||
+|| sb3.orbit-lab.org || console.sb3.orbit-lab.org || ORBIT Sandbox 3 (USRP/GNURadio) ||
+|| sb4.orbit-lab.org || console.sb4.orbit-lab.org || ORBIT Sandbox 4 (RF Matrix) ||
+|| sb7.orbit-lab.org || console.sb7.orbit-lab.org || ORBIT Sandbox 7 ||
+|| sb9.orbit-lab.org || console.sb9.orbit-lab.org || ORBIT Sandbox 9 (Wired/SDN) ||
+|| outdoor.orbit-lab.org || console.outdoor.orbit-lab.org || ORBIT Outdoor Network ||
+|| sb1.cosmos-lab.org || console.sb1.cosmos-lab.org || COSMOS Sandbox 1 ||
+|| sb2.cosmos-lab.org || console.sb2.cosmos-lab.org || COSMOS Sandbox 2 (Columbia) ||
+|| bed.cosmos-lab.org || console.bed.cosmos-lab.org || COSMOS BED (West Harlem, NYC) ||
+|| weeks.cosmos-lab.org || console.weeks.cosmos-lab.org || COSMOS Weeks Hall (Industry 4.0) ||
+
+For a gateway connection (file transfer, tunneling) without a specific domain reservation, you can use:
+* `gw.cosmos-lab.org` — COSMOS gateway
+* `gw.orbit-lab.org` — ORBIT gateway
+
+----
+
+== Basic Connection ==
+
+The simplest way to connect to a testbed node is using SSH's `-J` flag (ProxyJump), which routes your connection through the console server in a single command:
+
+{{{
+ssh -J <username>@<console> root@<node>
+}}}
+
+For example, to connect to node1-1 in COSMOS Sandbox 1:
+{{{
+ssh -J jsmith@console.sb1.cosmos-lab.org root@node1-1
+}}}
+
+Or to connect to node5-5 in the ORBIT Grid:
+{{{
+ssh -J jsmith@console.grid.orbit-lab.org root@node5-5
+}}}
+
+The `-J` flag tells SSH to first connect to the console server (authenticating with your portal username and SSH key), then transparently hop to the target node as root. You do not need to manually log into the console and then SSH to the node — it all happens in one step.
+
+'''Shorthand:''' For many domains, you can use the domain name directly as the console (without the `console.` prefix):
+{{{
+ssh -J jsmith@sb1.cosmos-lab.org root@node1-1
+}}}
+
+----
+
+== SSH Config File ==
+
+For frequent use, configuring your `~/.ssh/config` file avoids typing long SSH commands. Here is a recommended configuration:
+
+{{{
+# COSMOS/ORBIT Gateway (for file transfer and general access)
+Host cosmos-gw
+    HostName gw.cosmos-lab.org
+    User jsmith
+
+# COSMOS SB1 Console
+Host cosmos-sb1
+    HostName console.sb1.cosmos-lab.org
+    User jsmith
+
+# Quick access to SB1 nodes (usage: ssh sb1-node1-1)
+Host sb1-node*
+    ProxyJump cosmos-sb1
+    User root
+
+# ORBIT Grid Console
+Host orbit-grid
+    HostName console.grid.orbit-lab.org
+    User jsmith
+
+# Quick access to Grid nodes (usage: ssh grid-node5-5)
+Host grid-node*
+    ProxyJump orbit-grid
+    User root
+
+# General settings for all testbed hosts
+Host cosmos-* orbit-* sb1-node* grid-node*
+    StrictHostKeyChecking no
+    UserKnownHostsFile /dev/null
+    ServerAliveInterval 60
+    ServerAliveCountMax 3
+}}}
+
+With this configuration, connecting to a node is as simple as:
+{{{
+ssh sb1-node1-1
+}}}
+
+The `StrictHostKeyChecking no` and `UserKnownHostsFile /dev/null` settings prevent SSH from complaining about changed host keys, which is expected behavior on testbed nodes that are frequently reimaged with different operating systems.
+
+The `ServerAliveInterval` and `ServerAliveCountMax` settings send keepalive packets to prevent idle connections from being dropped by firewalls.
+
+----
+
+== SSH Tunneling == #Tunneling
+
+SSH tunnels let you securely access network services running on testbed nodes (such as web interfaces, Jupyter notebooks, or custom dashboards) from your local machine. All traffic flows encrypted through the SSH connection.
+
+=== Local Port Forwarding ===
+
+Local port forwarding maps a port on a remote testbed node to a port on your local machine. This is the most common type of tunnel.
+
+'''Example: Access a Jupyter notebook running on port 8888 of node1-1:'''
+{{{
+ssh -L 8888:localhost:8888 -J jsmith@console.sb1.cosmos-lab.org root@node1-1
+}}}
+
+After running this command, open `http://localhost:8888` in your web browser to access the Jupyter notebook. The traffic is encrypted and routed through the SSH connection.
+
+'''Example: Access a web dashboard on port 3000:'''
+{{{
+ssh -L 3000:localhost:3000 -J jsmith@console.sb1.cosmos-lab.org root@node1-1
+}}}
+
+You can forward multiple ports simultaneously by adding additional `-L` flags:
+{{{
+ssh -L 8888:localhost:8888 -L 3000:localhost:3000 -J jsmith@console.sb1.cosmos-lab.org root@node1-1
+}}}
+
+=== Dynamic SOCKS Proxy ===
+
+A dynamic SOCKS proxy creates a general-purpose encrypted tunnel that you can route any TCP traffic through:
+{{{
+ssh -D 1080 -J jsmith@console.sb1.cosmos-lab.org root@node1-1
+}}}
+
+Then configure your web browser to use `localhost:1080` as a SOCKS5 proxy. All web traffic will be routed through the testbed node, letting you access internal testbed services and web interfaces as if you were on the testbed network.
+
+=== Remote Port Forwarding ===
+
+Remote port forwarding makes a service running on your local machine accessible from the testbed node. This is useful for sending data from the testbed to a service on your laptop:
+{{{
+ssh -R 5000:localhost:5000 -J jsmith@console.sb1.cosmos-lab.org root@node1-1
+}}}
+
+After this, processes on node1-1 can connect to `localhost:5000` and reach the service running on port 5000 of your local machine.
+
+----
+
+== File Transfer ==
+
+To copy files between your local machine and testbed nodes, use `scp` or `rsync` with the ProxyJump option:
+
+'''Upload a file to a node:'''
+{{{
+scp -J jsmith@console.sb1.cosmos-lab.org localfile.txt root@node1-1:/root/
+}}}
+
+'''Download a file from a node:'''
+{{{
+scp -J jsmith@console.sb1.cosmos-lab.org root@node1-1:/root/results.csv ./
+}}}
+
+'''Sync a directory:'''
+{{{
+rsync -avz -e "ssh -J jsmith@console.sb1.cosmos-lab.org" root@node1-1:/root/data/ ./data/
+}}}
+
+For large file transfers, consider using the gateway (`gw.cosmos-lab.org`) and the NFS-shared filesystem at `/mnt/datasets/` which is accessible from all console servers and nodes.
+
+----
+
+== Troubleshooting ==
+
+'''Connection refused or timeout'''
+* Verify that you have an '''active''' (approved) reservation for the domain — pending reservations do not grant access
+* Check that your SSH public key is uploaded in the portal (not the private key)
+* Ensure you are using your '''portal username''' (not root) for the console server
+
+'''Permission denied (publickey)'''
+* Make sure the public key on the portal matches the private key on your machine
+* Try specifying the key explicitly: `ssh -i ~/.ssh/id_ed25519 -J jsmith@console.sb1.cosmos-lab.org root@node1-1`
+* Check that your private key file has correct permissions: `chmod 600 ~/.ssh/id_ed25519`
+
+'''Host key verification failed'''
+* Testbed nodes are frequently reimaged, changing their SSH host keys. Add these lines to your `~/.ssh/config` for testbed hosts: `StrictHostKeyChecking no` and `UserKnownHostsFile /dev/null`
+
+'''Connection drops after idle time'''
+* Add `ServerAliveInterval 60` to your SSH config to send keepalive packets every 60 seconds
+
+=== Further Reading ===
+
+* [https://www.ssh.com/academy/ssh/tunneling SSH Academy: SSH Tunneling] — explanation of local, remote, and dynamic forwarding
+* [https://man.openbsd.org/ssh OpenBSD: ssh manual page] — the authoritative reference for all SSH options
+* [https://www.digitalocean.com/community/tutorials/how-to-configure-custom-connection-options-for-your-ssh-client DigitalOcean: SSH Config Tutorial] — guide to setting up `~/.ssh/config`