Configuration of RCPS

RCPS is composed of the following:

  • A linux server (DEV)
  • A windows physics simulation machine (SIM)
  • 32 Beaglebone Blacks (BBB)
  • A configurable (smart) network switch
  • A basic network switch

DEV machine (linux server)

The linux server provides the following roles:

  • NGINX web-server for providing port routing to
    • rcps.isis.vanderbilt.edu : a static page site
    • rosmod.rcps.isis.vanderbilt.edu : a ROSMOD webgme server
    • tes.rcps.isis.vanderbilt.edu : a TES GridlabD webgme server
  • Internet gateway (with NAT and masquerading) for the BBB cluster
  • Remote access (through key-authenticated ssh) for DEV and the cluster

NGINX setup

The NGINX serer provides the ability to resolve the various DNS names (*.rcps.isis.vanderbilt.edu) to specific servers on the same machine. The configuration file for the NGINX server manager can be found in /etc/nginx/sites-available/default.

Security certificate management

Since the webgme servers that we run (TES, ROSMOD) must support authentication, they must be able to run over HTTPS. This in turn requires that they have valid security certificates.

The security certificates are managed by letsencrypt, which can be run on the server to update the security certificates for the associated domains whenever they expire (usually every 6 months).

Creation

First you must stop the NGINX server (which redirects rcps.isis.vanderbilt.edu, tes.rcps.isis.vanderbilt.edu, and rosmod.rcps.isis.vanderbilt.edu):

sudo service nginx stop

To renew the keys for those sites (including both secure and insecure versions):

cd /home/jeb/Repositories/letsencrypt
./letsencrypt-auto -d rcps.isis.vanderbilt.edu -d tes.rcps.isis.vanderbilt.edu -d rosmod.rcps.isis.vanderbilt.edu -d www.rcps.isis.vanderbilt.edu -d www.tes.rcps.isis.vanderbilt.edu -d www.rosmod.rcps.isis.vanderbilt.edu

Now that the certificates are renewed, restart the NGINX server:

sudo service nginx start

Renewal

Make sure you stop the nginx server (as above) and then you can simply run

cd /home/jeb/Repositories/letsencrypt
./letsencrypt-auto renew

RCPS site setup

The RCPS site is a set of statically hosted webpages (which were generated from ReStructured Text (RST) files that were compiled into the site webpages using python-sphinx. The site itself exists within the gh-pages branch of the webgme-rosmod repository; this branch has been checked out into /home/jeb/docs/webgme-rosmod/docs.

ROSMOD server setup

The ROSMOD server is able to be run as a system service using commands like:

sudo start rosmod
sudo restart rosmod
sudo stop rosmod

The ability to run it as a service (and also its configuration) is provided by the file /etc/init/rosmod.conf. The actual code for the ROSMOD server is found in /home/jeb/Repositories/webgme-rosmod, which is a clone of github.com/rosmod/webgme-rosmod.

The ROSMOD server also has an ancillary configuration file, /etc/init/rosmod-dev.conf, which allows you to run it in dev mode, under which the META is able to be edited. This is possible only if the regular rosmod service is not currently running. If so, then the rosmod-dev server can be controlled by

sudo start rosmod-dev
sudo restart rosmod-dev
sudo stop rosmod-dev

No matter what mode it runs in, the webgme-rosmod server always outputs its log to /var/log/rosmod.log.

TES server setup

The tes server is able to be run as a system service using commands like:

sudo start tes
sudo restart tes
sudo stop tes

The ability to run it as a service (and also its configuration) is provided by the file /etc/init/tes.conf. The actual code for the TES server is found in /home/jeb/Repositories/webgme-gridlabd, which is a clone of github.com/finger563/webgme-gridlabd.

The TES server always outputs its log to /var/log/tes.log.

Gateway setup

The gateway functionality for the server is configured by

  • setting the ip_forward control to 1 so that it forwards relevant traffic for other nodes
echo 1 > /proc/sys/net/ipv4/ip_forward
  • Setting up IPTables to properly forward traffic between the DEV machine’s two interfaces (eth0: internet side, eth1: BBB cluster side). NOTE: this command already exists within /etc/rc.local which is run every time the machine reboots.
iptables-restore < /home/jeb/utils/iptables.conf

Where /home/jeb/utils/iptables.conf contains

# Generated by iptables-save v1.4.21 on Thu Jul 28 13:39:12 2016
*filter
:INPUT ACCEPT [232:21394]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [59:4684]
-A FORWARD -i eth1 -j ACCEPT
COMMIT
# Completed on Thu Jul 28 13:39:12 2016
# Generated by iptables-save v1.4.21 on Thu Jul 28 13:39:12 2016
*nat
:PREROUTING ACCEPT [138:21945]
:INPUT ACCEPT [79:16517]
:OUTPUT ACCEPT [2:140]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Thu Jul 28 13:39:12 2016

which was created by configuring iptables using:

iptables --append FORWARD --in-interface eth1 -j ACCEPT
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
iptables-save > /home/jeb/utils/iptables.conf

To see the current setting of the NAT iptables listing, just run

jeb@rcps:~$ sudo iptables -t nat -n -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

Remote access setup

The server is configured (through /etc/ssh/sshd_config) to accept only key-based remote logins, using the key provided in /home/jeb/.ssh/id_rsa_jetsontk1, under the username jeb.

With this same key you can log into any of the nodes (conveniently defined as node1 through node32 in the /etc/hosts file) using either debian or root as the username, e.g.

ssh -i /home/jeb/.ssh/id_rsa_jetsontk1 root@node1
ssh -i /home/jeb/.ssh/id_rsa_jetsontk1 debian@node1

The connection to the BBBs from the server goes through eth1 to the smart / programmable switch. The network that the server and the BBBs are on together is the 10.1.1.0/24 network, with the BBBs having address 1-32, and the server having 249.

SIM machine (windows)

The SIM machine connects to the BBBs through the basic network switch, sharing the 191.168.127.0/24 network with them. The SIM machine runs physics simulators such as Kerbal Space Program, Cities Skylines, SUMO, etc.

BBB Cluster

The BBB cluster all run the same 16 GB debian image, using a 4.x kernel version. The image can be flashed to the card from the server (using dd) or from a windows machine (using win32 disk imager).

On the BBB, the high speed Gigabit ethernet jack (eth0) provides access to the 10.1.1.0/24 network which connects to the DEV machine, and through which their ROSMOD code travels. A USB-Ethernet adapter (eth1) provides slower access to the physics machine through their 191.168.127.0/24 network.

For the BBB to be able to use the linux server (DEV) as a gateway for the internet, they must have their default route configured to go through the gateway, e.g.

route add default gw 10.1.1.249 dev eth0

If you’re unsure about what the routes that are currently defined are, just check using (where master is defined in /etc/hosts as 10.1.1.249):

root@node1:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         master          0.0.0.0         UG    0      0        0 eth0
default         *               0.0.0.0         U     0      0        0 eth1
10.1.1.0        *               255.255.255.0   U     0      0        0 eth0
link-local      *               255.255.0.0     U     0      0        0 eth1
191.168.127.0   *               255.255.255.0   U     0      0        0 eth1
192.168.7.0     *               255.255.255.252 U     0      0        0 usb0

Published: August 30 2016

blog comments powered by Disqus