nepi-ng - B - wireless
Prerequisites
For this series of experiments, we make the same assumptions as in the
previous series. In a nutshell, we expect you have a valid
reservation, and the 2 nodes fit01
and fit02
are properly loaded with
the default image, and turned on of course.
Please visit this page to review how to reach that point, if necessary.
Objectives
In this series we will start playing with the wireless interfaces. Of course a minimal background is assumed in terms of dealing with wireless interfaces under linux; further information can be found at the following locations :
we recommend to see this quick introduction to
iw
, that is instrumental in these tasksas well as this helpful page on how to use
iw
if you were more familiar withiwconfig
that is now considered obsolete - much likeip
has now replacedifconfig
.
nepi-ng
Now as far as nepi-ng
is concerned, in the previous series we have
seen ways to run remotely simple commands. In this section we will see
simple means to come up with more complex logic, simply by using shell
scripts that are pushed remotely under the hood before they are
triggered. This is what the RunString
and RunScript
classes are
about. Let us see them in action right away.
Objective
We are going to run the exact same experiment as in the previous run
A5, that is to say a simple ping triggered on
fit01
towards fit02
, but this time on one of the wireless
interfaces.
What changes then, as compared with our previous experiment, is that
we cannot anymore simply run the predefined convenience command
turn-on-data
, and we are going to have to put a little more work in this
step.
We are going to configure a small ad hoc network between both nodes,
using their Atheros WiFi card, and this is the purpose of the bash
shell script named turn_on_wireless_script
; as you will see, this
script uses some hard-wired settings like the channel and SSID, but
you can easily tweak the code so as to make this a parameter.
New features
The B1 code
below exhibits the use of a new class, RunString
,
which is a very convenient variation around the Run
class.
Instead of remotely invoking a command that is supposed to be
available there already, like we have done so far when e.g. invoking
ping
or even turn-on-data
, RunString
allows you to invoke a command
that we provide as a python variable.
This means that we can write our own shell snippet, in charge of
creating a small ad-hoc network, and embed this script right inside
our nepi-ng script; see the turn_on_wireless_script
variable in the
code below.
See also how the init_node_01
job now uses RunString
to pass it
python variables as arguments. For example this is how
turn_on_wireless_script
gets passed the right IP address for each node.
As you will see in the sample output below, it takes some time for the IP connectivity to be established after the driver is configured.
The code
#!/usr/bin/env python3 from argparse import ArgumentParser from asynciojobs import Scheduler from apssh import SshNode, SshJob from apssh import Run, RunString ########## gateway_hostname = 'faraday.inria.fr' gateway_username = 'inria_r2lab.tutorial' verbose_ssh = False wireless_driver="ath9k" wireless_interface="atheros" parser = ArgumentParser() parser.add_argument("-s", "--slice", default=gateway_username, help="specify an alternate slicename, default={}" .format(gateway_username)) parser.add_argument("-v", "--verbose-ssh", default=False, action='store_true', help="run ssh in verbose mode") args = parser.parse_args() gateway_username = args.slice verbose_ssh = args.verbose_ssh ########## faraday = SshNode(hostname = gateway_hostname, username = gateway_username, verbose = verbose_ssh) node1 = SshNode(gateway = faraday, hostname = "fit01", username = "root", verbose = verbose_ssh) node2 = SshNode(gateway = faraday, hostname = "fit02", username = "root", verbose = verbose_ssh) ########## # create an orchestration scheduler scheduler = Scheduler() ########## check_lease = SshJob( # checking the lease is done on the gateway node = faraday, # this means that a failure in any of the commands # will cause the scheduler to bail out immediately critical = True, command = Run("rhubarbe leases --check"), scheduler = scheduler, ) #################### # This is our own brewed script for setting up a wifi network # it run on the remote machine - either sender or receiver # and is in charge of initializing a small ad-hoc network # # Thanks to the RunString class, we can just define this as # a python string, and pass it arguments from python variables # turn_on_wireless_script = """#!/bin/bash # we expect the following arguments # * wireless driver name (iwlwifi or ath9k) # * wireless interface name (intel or atheros) # * IP-address/mask for that interface # * the wifi network name to join # * the wifi frequency to use driver=$1; shift ifname=$1; shift ipaddr_mask=$1; shift netname=$1; shift freq=$1; shift # load the r2lab utilities - code can be found here: # https://github.com/fit-r2lab/r2lab-embedded/blob/master/shell/nodes.sh source /etc/profile.d/nodes.sh # make sure to use the latest code on the node git-pull-r2lab turn-off-wireless echo loading module $driver modprobe $driver # some time for udev to trigger its rules sleep 1 echo configuring interface $ifname # make sure to wipe down everything first so we can run again and again ip address flush dev $ifname ip link set $ifname down # configure wireless iw dev $ifname set type ibss ip link set $ifname up # set to ad-hoc mode iw dev $ifname ibss join $netname $freq ip address add $ipaddr_mask dev $ifname """ ########## # setting up the wireless interface on both fit01 and fit02 init_node_01 = SshJob( node = node1, command = RunString( # first argument is a string containing # the script to be run remotely turn_on_wireless_script, # and now its arguments wireless_driver, wireless_interface, "10.0.0.1/24", "foobar", 2412, # verbose=True, ), required = check_lease, scheduler = scheduler, ) init_node_02 = SshJob( node = node2, command = RunString( turn_on_wireless_script, wireless_driver, wireless_interface, "10.0.0.2/24", "foobar", 2412, ), required = check_lease, scheduler = scheduler, ) # the command we want to run in node1 is as simple as it gets ping = SshJob( node = node1, required = (init_node_01, init_node_02), command = Run( 'ping', '-c', '20', '10.0.0.2', '-I', wireless_interface, # verbose=True, ), scheduler = scheduler, ) ########## # run the scheduler ok = scheduler.orchestrate() # give details if it failed ok or scheduler.debrief() success = ok and ping.result() == 0 # producing a dot file for illustration scheduler.export_as_dotfile("B1.dot") # return something useful to your OS exit(0 if success else 1)
Sample output
fit02:From https://github.com/fit-r2lab/r2lab-embedded
fit02: e496bfd..b8050d8 master -> origin/master
fit02: * [new branch] oai-5g-latest -> origin/oai-5g-latest
fit02: * [new branch] using-library -> origin/using-library
fit01:From https://github.com/fit-r2lab/r2lab-embedded
fit01: e496bfd..b8050d8 master -> origin/master
fit01: * [new branch] oai-5g-latest -> origin/oai-5g-latest
fit01: * [new branch] using-library -> origin/using-library
fit02:Already on 'master'
fit01:Already on 'master'
fit02:From https://github.com/fit-r2lab/r2lab-embedded
fit02: * branch master -> FETCH_HEAD
fit02:turn-off-wireless: shutting down device intel
fit02:turn-off-wireless: removing driver iwlwifi
fit01:From https://github.com/fit-r2lab/r2lab-embedded
fit01: * branch master -> FETCH_HEAD
fit01:turn-off-wireless: shutting down device intel
fit01:turn-off-wireless: removing driver iwlwifi
fit02:turn-off-wireless: shutting down device atheros
fit02:turn-off-wireless: removing driver ath9k
fit01:turn-off-wireless: shutting down device atheros
fit01:turn-off-wireless: removing driver ath9k
faraday.inria.fr:Checking current reservation for inria_r2lab.tutorial : OK
fit02:========== Updating /root/r2lab-embedded for branch master
fit02:HEAD is now at e496bfd bugs fixed
fit02:Fetching origin
fit01:========== Updating /root/r2lab-embedded for branch master
fit01:HEAD is now at e496bfd bugs fixed
fit01:Fetching origin
fit02:Your branch is behind 'origin/master' by 29 commits, and can be fast-forwarded.
fit02: (use "git pull" to update your local branch)
fit01:Your branch is behind 'origin/master' by 29 commits, and can be fast-forwarded.
fit01: (use "git pull" to update your local branch)
fit02:Updating e496bfd..b8050d8
fit02:Fast-forward
fit02: backgrounds/moto-e-4g.jpg | Bin 0 -> 102057 bytes
fit02: imaging/all-images.sh | 15 ++++-
fit02: nightly/nightly.py | 22 +++++---
fit02: shell/faraday.sh | 6 +-
fit02: shell/imaging.sh | 5 +-
fit02: shell/nodes.sh | 4 +-
fit02: shell/oai-common.sh | 18 ++++++
fit02: shell/oai-enb.sh | 138 ++++++++++++++++++++++++++++++----------------
fit02: shell/oai-gw.sh | 5 +-
fit02: 9 files changed, 145 insertions(+), 68 deletions(-)
fit02: create mode 100644 backgrounds/moto-e-4g.jpg
fit01:Updating e496bfd..b8050d8
fit01:Fast-forward
fit01: backgrounds/moto-e-4g.jpg | Bin 0 -> 102057 bytes
fit01: imaging/all-images.sh | 15 ++++-
fit01: nightly/nightly.py | 22 +++++---
fit01: shell/faraday.sh | 6 +-
fit01: shell/imaging.sh | 5 +-
fit01: shell/nodes.sh | 4 +-
fit01: shell/oai-common.sh | 18 ++++++
fit01: shell/oai-enb.sh | 138 ++++++++++++++++++++++++++++++----------------
fit01: shell/oai-gw.sh | 5 +-
fit01: 9 files changed, 145 insertions(+), 68 deletions(-)
fit01: create mode 100644 backgrounds/moto-e-4g.jpg
fit02:loading module ath9k
fit01:loading module ath9k
fit02:configuring interface atheros
fit01:configuring interface atheros
fit01:PING 10.0.0.2 (10.0.0.2) from 10.0.0.1 atheros: 56(84) bytes of data.
fit01:From 10.0.0.1 icmp_seq=1 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=2 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=3 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=4 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=5 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=6 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=7 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=8 Destination Host Unreachable
fit01:From 10.0.0.1 icmp_seq=9 Destination Host Unreachable
fit01:64 bytes from 10.0.0.2: icmp_seq=10 ttl=64 time=4.55 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=11 ttl=64 time=2.53 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=12 ttl=64 time=2.47 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=13 ttl=64 time=2.54 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=14 ttl=64 time=2.52 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=15 ttl=64 time=2.52 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=16 ttl=64 time=2.46 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=17 ttl=64 time=2.52 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=18 ttl=64 time=1.69 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=19 ttl=64 time=1.73 ms
fit01:64 bytes from 10.0.0.2: icmp_seq=20 ttl=64 time=1.73 ms
fit01:
fit01:--- 10.0.0.2 ping statistics ---
fit01:20 packets transmitted, 11 received, +9 errors, 45% packet loss, time 19063ms
fit01:rtt min/avg/max/mdev = 1.696/2.483/4.555/0.742 ms, pipe 3
Next
Let us see some variants on that theme in the next tab
Objective
In this new variant, we are going to illustrate a few convenient tricks:
we add a new option to the script so that one can choose, right on the command line, whether we want to use the intel or the atheros WiFi card; this simply relies on the standard argparse module that we have already used for other options, the only new thing being the use of the
choices
keyword, so that only the 2 supported driver names can be used;more interestingly, we can remove the
wireless_interface
variable from that script altogether, thanks to a shell utilily available on all nodes and namedwait-for-interface-on-driver
. This shell function returns the name of the network interface associated with a given driver name, so it is enough to load, say, moduleath9k
and let this function figure out that the actual interface name isatheros
;in much the same way, we will remove the need to pass an IP address to the
turn_on_wireless_script
, by computing this address based on the node rank in r2lab, using another convenience tool namedr2lab-ip
; there actually are 2 similar functions available on each noder2lab-ip
: will return the number of the current node, under 1 or 2 digits; for example on node 8, this returns8
; this is suitable to build IP addresses;r2lab-id
: will return the number of the current node, but always in 2 digits; for example on node 8, this returns08
; this is suitable to build hostnames; so for example you would do
#
my_data_hostname="data$(r2lab-id)"
echo $my_data_hostname
data08
my_wireless_ip_address="10.0.0.$(r2lab-ip)"
echo my_wireless_ip_address
10.0.0.8
- finally, this new script displays the outputs of the ssh commands
with a slightly different format, in that every line will now
receive a timestamp in addition to the hostname. This is achieved
through the use of the
TimeColonFormatter()
class; we create one instance of this class for each instance ofSshNode
.
So for example we would issue lines like
14-37-28:fit01:turn-off-wireless: shutting down device intel
instead of just
fit01:turn-off-wireless: shutting down device intel
About that last point, note that other types of formatters are available, that will let you store output in a given directory, and in files named after each individual hostname. See this page for more info on other available formatters.
The code
#!/usr/bin/env python3 from argparse import ArgumentParser from asynciojobs import Scheduler from apssh import SshNode, SshJob from apssh import Run, RunString from apssh import TimeColonFormatter ########## gateway_hostname = 'faraday.inria.fr' gateway_username = 'inria_r2lab.tutorial' verbose_ssh = False parser = ArgumentParser() parser.add_argument("-s", "--slice", default=gateway_username, help="specify an alternate slicename, default={}" .format(gateway_username)) parser.add_argument("-v", "--verbose-ssh", default=False, action='store_true', help="run ssh in verbose mode") parser.add_argument("-d", "--driver", default='ath9k', choices = ['iwlwifi', 'ath9k'], help="specify which driver to use") args = parser.parse_args() gateway_username = args.slice verbose_ssh = args.verbose_ssh wireless_driver = args.driver ########## faraday = SshNode(hostname = gateway_hostname, username = gateway_username, verbose = verbose_ssh, formatter = TimeColonFormatter()) node1 = SshNode(gateway = faraday, hostname = "fit01", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) node2 = SshNode(gateway = faraday, hostname = "fit02", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) ########## # create an orchestration scheduler scheduler = Scheduler() ########## check_lease = SshJob( # checking the lease is done on the gateway node = faraday, # this means that a failure in any of the commands # will cause the scheduler to bail out immediately critical = True, command = Run("rhubarbe leases --check"), scheduler = scheduler, ) #################### # This is our own brewed script for setting up a wifi network # it run on the remote machine - either sender or receiver # and is in charge of initializing a small ad-hoc network # # Thanks to the RunString class, we can just define this as # a python string, and pass it arguments from python variables # turn_on_wireless_script = """#!/bin/bash # we expect the following arguments # * wireless driver name (iwlwifi or ath9k) # * the wifi network name to join # * the wifi frequency to use driver=$1; shift netname=$1; shift freq=$1; shift # load the r2lab utilities - code can be found here: # https://github.com/fit-r2lab/r2lab-embedded/blob/master/shell/nodes.sh source /etc/profile.d/nodes.sh # make sure to use the latest code on the node git-pull-r2lab turn-off-wireless # local IP address to use is computed on the 10.0.0.0/24 # subnet and based on current node number (using r2lab-ip) ipaddr_mask=10.0.0.$(r2lab-ip)/24 echo loading module $driver modprobe $driver # some time for udev to trigger its rules sleep 1 # use r2lab tools to figure out the interface name ifname=$(wait-for-interface-on-driver $driver) echo configuring interface $ifname # make sure to wipe down everything first so we can run again and again ip address flush dev $ifname ip link set $ifname down # configure wireless iw dev $ifname set type ibss ip link set $ifname up # set to ad-hoc mode iw dev $ifname ibss join $netname $freq ip address add $ipaddr_mask dev $ifname """ ########## # setting up the wireless interface on both fit01 and fit02 init_node_01 = SshJob( node = node1, command = RunString( turn_on_wireless_script, wireless_driver, "foobar", 2412, # setting a remote_name allows to # improve the graphical rendering remote_name = 'turn-on-wireless' # verbose=True, ), required = check_lease, scheduler = scheduler, ) init_node_02 = SshJob( node = node2, command = RunString( turn_on_wireless_script, wireless_driver, "foobar", 2412, remote_name = 'turn-on-wireless' ), required = check_lease, scheduler = scheduler, ) # the command we want to run in node1 is as simple as it gets ping = SshJob( node = node1, required = (init_node_01, init_node_02), command = Run( 'ping', '-c', '20', '10.0.0.2', # verbose=True, ), scheduler = scheduler, ) ########## # run the scheduler ok = scheduler.orchestrate() # give details if it failed ok or scheduler.debrief() success = ok and ping.result() == 0 # producing a dot file for illustration scheduler.export_as_dotfile("B2.dot") # return something useful to your OS exit(0 if success else 1)
Sample output
11-00-55:fit01:Already on 'master'
11-00-55:fit02:Already on 'master'
11-00-55:fit01:From https://github.com/fit-r2lab/r2lab-embedded
11-00-55:fit01: * branch master -> FETCH_HEAD
11-00-55:fit01:turn-off-wireless: driver iwlwifi not used
11-00-55:fit02:From https://github.com/fit-r2lab/r2lab-embedded
11-00-55:fit02: * branch master -> FETCH_HEAD
11-00-55:fit01:turn-off-wireless: shutting down device atheros
11-00-55:fit02:turn-off-wireless: driver iwlwifi not used
11-00-55:fit02:turn-off-wireless: shutting down device atheros
11-00-55:fit01:turn-off-wireless: removing driver ath9k
11-00-55:fit02:turn-off-wireless: removing driver ath9k
11-00-56:fit01:Using id=01 and fitid=fit01 - from hostname
11-00-56:fit02:Using id=02 and fitid=fit02 - from hostname
11-00-58:fit01:Using device atheros
11-00-58:fit02:Using device atheros
11-00-53:faraday.inria.fr:Checking current reservation for inria_r2lab.tutorial : OK
11-00-54:fit01:========== Updating /root/r2lab-embedded for branch master
11-00-54:fit01:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
11-00-54:fit01:Fetching origin
11-00-54:fit02:========== Updating /root/r2lab-embedded for branch master
11-00-54:fit02:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
11-00-54:fit02:Fetching origin
11-00-55:fit01:Your branch is up-to-date with 'origin/master'.
11-00-55:fit02:Your branch is up-to-date with 'origin/master'.
11-00-55:fit01:Already up-to-date.
11-00-55:fit02:Already up-to-date.
11-00-56:fit01:loading module ath9k
11-00-56:fit02:loading module ath9k
11-00-58:fit01:configuring interface atheros
11-00-58:fit02:configuring interface atheros
11-01-01:fit01:PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
11-01-01:fit01:From 10.0.0.1 icmp_seq=1 Destination Host Unreachable
11-01-01:fit01:From 10.0.0.1 icmp_seq=2 Destination Host Unreachable
11-01-01:fit01:From 10.0.0.1 icmp_seq=3 Destination Host Unreachable
11-01-04:fit01:From 10.0.0.1 icmp_seq=4 Destination Host Unreachable
11-01-04:fit01:From 10.0.0.1 icmp_seq=5 Destination Host Unreachable
11-01-04:fit01:From 10.0.0.1 icmp_seq=6 Destination Host Unreachable
11-01-06:fit01:64 bytes from 10.0.0.2: icmp_seq=7 ttl=64 time=2004 ms
11-01-06:fit01:64 bytes from 10.0.0.2: icmp_seq=8 ttl=64 time=1006 ms
11-01-06:fit01:64 bytes from 10.0.0.2: icmp_seq=9 ttl=64 time=6.51 ms
11-01-07:fit01:64 bytes from 10.0.0.2: icmp_seq=10 ttl=64 time=1.33 ms
11-01-08:fit01:64 bytes from 10.0.0.2: icmp_seq=11 ttl=64 time=1.34 ms
11-01-09:fit01:64 bytes from 10.0.0.2: icmp_seq=12 ttl=64 time=1.34 ms
11-01-10:fit01:64 bytes from 10.0.0.2: icmp_seq=13 ttl=64 time=1.35 ms
11-01-11:fit01:64 bytes from 10.0.0.2: icmp_seq=14 ttl=64 time=1.34 ms
11-01-12:fit01:64 bytes from 10.0.0.2: icmp_seq=15 ttl=64 time=1.34 ms
11-01-13:fit01:64 bytes from 10.0.0.2: icmp_seq=16 ttl=64 time=1.34 ms
11-01-14:fit01:64 bytes from 10.0.0.2: icmp_seq=17 ttl=64 time=1.34 ms
11-01-15:fit01:64 bytes from 10.0.0.2: icmp_seq=18 ttl=64 time=1.39 ms
11-01-16:fit01:64 bytes from 10.0.0.2: icmp_seq=19 ttl=64 time=1.22 ms
11-01-17:fit01:64 bytes from 10.0.0.2: icmp_seq=20 ttl=64 time=1.28 ms
11-01-17:fit01:
11-01-17:fit01:--- 10.0.0.2 ping statistics ---
11-01-17:fit01:20 packets transmitted, 14 received, +6 errors, 30% packet loss, time 19015ms
11-01-17:fit01:rtt min/avg/max/mdev = 1.228/216.582/2004.461/558.972 ms, pipe 3
Next
In the next variant, we are going to see how we can better esimate the time it takes for the ad hoc network to actually come up.
Objective
In this variant, we will see how we can even design our experiment so that all the shell code goes in a single file, working as a companion for the python script that will only deal with overall logic.
Our pretext for doing so is that we would like to better understand
the startup sequence observed in the previous runs B1 and B2. And for
doing that it is not good enough to just run ping
like we did, but
instead we would need run something a little more elaborate - that
we'll write in shell.
As much as it is convenient to have the ability to insert shell script right in the python code, when things become more complex, the shell code tends to become in the way.
So, two features are at work in the following code
Using
RunScript
instead ofRunString
as a way to define commands allows us to separate the shell script in a separate file.Also you can note at the end of the shell script, a very simple trick that lets you group any number of functions in a shell script, and call each individual function by just stating its name and arguments.
In other words, it means that if you write the following in a file named myscript.sh
:
#!/bin/bash function foo () { echo "in function foo" for arg in "$@"; do echo "foo arg: " $arg; done } function bar () { echo "in function bar" for arg in "$@"; do echo "bar arg: " $arg; done } "$@"
and then you invoke myscript.sh foo 1 2 "3 4"
you will get this ouput
$ myscript.sh foo 1 2 "3 4"
in function foo
foo arg: 1
foo arg: 2
foo arg: 3 4
The code
Make sure you download both files in the same location before trying to run the python script.
#!/usr/bin/env python3 from argparse import ArgumentParser from asynciojobs import Scheduler from apssh import SshNode, SshJob from apssh import Run, RunString, RunScript from apssh import TimeColonFormatter ########## gateway_hostname = 'faraday.inria.fr' gateway_username = 'inria_r2lab.tutorial' verbose_ssh = False parser = ArgumentParser() parser.add_argument("-s", "--slice", default=gateway_username, help="specify an alternate slicename, default={}" .format(gateway_username)) parser.add_argument("-v", "--verbose-ssh", default=False, action='store_true', help="run ssh in verbose mode") parser.add_argument("-d", "--driver", default='ath9k', choices = ['iwlwifi', 'ath9k'], help="specify which driver to use") args = parser.parse_args() gateway_username = args.slice verbose_ssh = args.verbose_ssh wireless_driver = args.driver ########## faraday = SshNode(hostname = gateway_hostname, username = gateway_username, verbose = verbose_ssh, formatter = TimeColonFormatter()) node1 = SshNode(gateway = faraday, hostname = "fit01", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) node2 = SshNode(gateway = faraday, hostname = "fit02", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) ########## # create an orchestration scheduler scheduler = Scheduler() ########## check_lease = SshJob( # checking the lease is done on the gateway node = faraday, # this means that a failure in any of the commands # will cause the scheduler to bail out immediately critical = True, command = Run("rhubarbe leases --check"), scheduler = scheduler, ) # the shell script has gone into B3-wireless.sh #################### ########## # setting up the wireless interface on both fit01 and fit02 init_node_01 = SshJob( node = node1, # RunString is replaced with RunScript command = RunScript( # first argument is the local filename # where to find the script to run remotely "B3-wireless.sh", # then its arguments "init-ad-hoc-network", wireless_driver, "foobar", 2412, # no need to set remote_name with RunScript ), required = check_lease, scheduler = scheduler, ) init_node_02 = SshJob( node = node2, command = RunScript( "B3-wireless.sh", "init-ad-hoc-network", wireless_driver, "foobar", 2412, # setting a label to gain space in the graphical output label="ditto", ), required = check_lease, scheduler = scheduler, ) # the command we want to run in node1 is as simple as it gets ping = SshJob( node = node1, required = (init_node_01, init_node_02), command = RunScript( "B3-wireless.sh", "my-ping", '10.0.0.2', 20, # verbose=True, ), scheduler = scheduler, ) ########## # run the scheduler ok = scheduler.orchestrate() # give details if it failed ok or scheduler.debrief() success = ok and ping.result() == 0 # producing a dot file for illustration scheduler.export_as_dotfile("B3.dot") # return something useful to your OS exit(0 if success else 1)
#!/bin/bash #################### # This is our own brewed script for setting up a wifi network # it run on the remote machine - either sender or receiver # and is in charge of initializing a small ad-hoc network # # Thanks to the RunString class, we can just define this as # a python string, and pass it arguments from python variables # # we expect the following arguments # * wireless driver name (iwlwifi or ath9k) # * the wifi network name to join # * the wifi frequency to use function init-ad-hoc-network (){ driver=$1; shift netname=$1; shift freq=$1; shift # load the r2lab utilities - code can be found here: # https://github.com/fit-r2lab/r2lab-embedded/blob/master/shell/nodes.sh source /etc/profile.d/nodes.sh # make sure to use the latest code on the node git-pull-r2lab turn-off-wireless ipaddr_mask=10.0.0.$(r2lab-ip)/24 echo loading module $driver modprobe $driver # some time for udev to trigger its rules sleep 1 ifname=$(wait-for-interface-on-driver $driver) echo configuring interface $ifname # make sure to wipe down everything first so we can run again and again ip address flush dev $ifname ip link set $ifname down # configure wireless iw dev $ifname set type ibss ip link set $ifname up # set to ad-hoc mode iw dev $ifname ibss join $netname $freq ip address add $ipaddr_mask dev $ifname ### addition - would be cool to come up with something along these lines that # works on both cards # a recipe from Naoufal for Intel # modprobe iwlwifi # iwconfig wlan2 mode ad-hoc # ip addr add 10.0.0.41/16 dev wlan2 # ip link set wlan2 up # iwconfig wlan2 essid mesh channel 1 } function my-ping (){ dest=$1; shift maxwait=$1; shift start=$(date +%s) while true; do # allow for one second timeout only; try one packet if ping -w 1 -c 1 $dest >& /dev/null; then end=$(date +%s) duration=$(($end - $start)) echo "$(hostname) -> $dest: SUCCESS after ${duration}s" return 0 else echo "$dest not reachable" end=$(date +%s) duration=$(($end - $start)) if [ "$duration" -ge "$maxwait" ]; then echo "$(hostname) -> $dest: FAILURE after ${duration}s" return 1 fi fi done } ######################################## # just a wrapper so we can call the individual functions. so e.g. # B3-wireless.sh tracable-ping 10.0.0.2 20 # results in calling tracable-ping 10.0.0.2 20 "$@"
Sample output
11-01-22:fit01:Already on 'master'
11-01-22:fit02:Already on 'master'
11-01-22:fit01:From https://github.com/fit-r2lab/r2lab-embedded
11-01-22:fit01: * branch master -> FETCH_HEAD
11-01-22:fit01:turn-off-wireless: driver iwlwifi not used
11-01-22:fit01:turn-off-wireless: shutting down device atheros
11-01-22:fit01:turn-off-wireless: removing driver ath9k
11-01-22:fit02:From https://github.com/fit-r2lab/r2lab-embedded
11-01-22:fit02: * branch master -> FETCH_HEAD
11-01-22:fit02:turn-off-wireless: driver iwlwifi not used
11-01-22:fit02:turn-off-wireless: shutting down device atheros
11-01-22:fit02:turn-off-wireless: removing driver ath9k
11-01-22:fit02:Using id=02 and fitid=fit02 - from hostname
11-01-22:fit01:Using id=01 and fitid=fit01 - from hostname
11-01-25:fit02:Using device atheros
11-01-25:fit01:Using device atheros
11-01-20:faraday.inria.fr:Checking current reservation for inria_r2lab.tutorial : OK
11-01-21:fit01:========== Updating /root/r2lab-embedded for branch master
11-01-21:fit01:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
11-01-21:fit01:Fetching origin
11-01-21:fit02:========== Updating /root/r2lab-embedded for branch master
11-01-21:fit02:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
11-01-21:fit02:Fetching origin
11-01-22:fit01:Your branch is up-to-date with 'origin/master'.
11-01-22:fit02:Your branch is up-to-date with 'origin/master'.
11-01-22:fit01:Already up-to-date.
11-01-22:fit02:Already up-to-date.
11-01-22:fit02:loading module ath9k
11-01-22:fit01:loading module ath9k
11-01-25:fit02:configuring interface atheros
11-01-25:fit01:configuring interface atheros
11-01-26:fit01:10.0.0.2 not reachable
11-01-27:fit01:10.0.0.2 not reachable
11-01-28:fit01:10.0.0.2 not reachable
11-01-29:fit01:10.0.0.2 not reachable
11-01-30:fit01:10.0.0.2 not reachable
11-01-31:fit01:10.0.0.2 not reachable
11-01-32:fit01:10.0.0.2 not reachable
11-01-33:fit01:10.0.0.2 not reachable
11-01-34:fit01:10.0.0.2 not reachable
11-01-34:fit01:fit01 -> 10.0.0.2: SUCCESS after 9s
Next
Let us proceed on this series with an example that adds a cyclic task to this scenario.
Objective
In this example, we will just for fun add an infinite cyclic task in the scheduler. Here we will simply write a TICK mark every second, but this technique is most useful for consuming events in a message queue, or any other similar approach.
The trick is to use the plain Job
class from asynciojobs
, which
expects a regular asyncio
coroutine object, that we implement as
infinite_clock()
. We just need to define the associated job
clock_job
with forever = True
, which tells the scheduler that this
job never ends, so that it knows that there is no point in waiting for
it to complete.
More on graphical views
Note in this example that, with respect to the graphical representation of schedulers:
each job needs a specific label to be displayed in such a graph - as well as in
list()
anddebrief()
by the waythe system does its best to provide a meaningful label, but
you are always free to define your own label on a given job, like we do here with the
infinite clock
job.
The code
#!/usr/bin/env python3 from argparse import ArgumentParser from asynciojobs import Scheduler from asynciojobs import Job from apssh import SshNode, SshJob from apssh import Run, RunString, RunScript from apssh import TimeColonFormatter ########## gateway_hostname = 'faraday.inria.fr' gateway_username = 'inria_r2lab.tutorial' verbose_ssh = False parser = ArgumentParser() parser.add_argument("-s", "--slice", default=gateway_username, help="specify an alternate slicename, default={}" .format(gateway_username)) parser.add_argument("-v", "--verbose-ssh", default=False, action='store_true', help="run ssh in verbose mode") parser.add_argument("-d", "--driver", default='ath9k', choices = ['iwlwifi', 'ath9k'], help="specify which driver to use") args = parser.parse_args() gateway_username = args.slice verbose_ssh = args.verbose_ssh wireless_driver = args.driver ########## faraday = SshNode(hostname = gateway_hostname, username = gateway_username, verbose = verbose_ssh, formatter = TimeColonFormatter()) node1 = SshNode(gateway = faraday, hostname = "fit01", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) node2 = SshNode(gateway = faraday, hostname = "fit02", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) ########## # create an orchestration scheduler scheduler = Scheduler() ########## check_lease = SshJob( # checking the lease is done on the gateway node = faraday, # this means that a failure in any of the commands # will cause the scheduler to bail out immediately critical = True, command = Run("rhubarbe leases --check"), scheduler = scheduler, ) ########## # setting up the wireless interface on both fit01 and fit02 init_node_01 = SshJob( node = node1, command = RunScript( "B3-wireless.sh", "init-ad-hoc-network", wireless_driver, "foobar", 2412, ), required = check_lease, scheduler = scheduler, ) init_node_02 = SshJob( node = node2, command = RunScript( "B3-wireless.sh", "init-ad-hoc-network", wireless_driver, "foobar", 2412, label = "ditto", ), required = check_lease, scheduler = scheduler, ) # the command we want to run in node1 is as simple as it gets ping = SshJob( node = node1, required = (init_node_01, init_node_02), command = RunScript( "B3-wireless.sh", "my-ping", '10.0.0.2', 20, # verbose=True, ), scheduler = scheduler, ) ######## # for the fun of it, let's add a job that runs forever and writes # current time every second import time import asyncio async def infinite_clock(): while True: print("--- TICK - {}".format(time.strftime("%H:%M:%S"))) await asyncio.sleep(1) # a forever job is not expected to end, instead # it gets killed when the rest of the flock is done with clock_job = Job( infinite_clock(), forever=True, scheduler = scheduler, # for the illustrated graph label = "infinite clock", ) ########## # run the scheduler ok = scheduler.orchestrate() # give details if it failed ok or scheduler.debrief() success = ok and ping.result() == 0 # producing a dot file for illustration scheduler.export_as_dotfile("B4.dot") # return something useful to your OS exit(0 if success else 1)
Sample output
11-01-39:fit01:Already on 'master'
11-01-39:fit02:Already on 'master'
11-01-39:fit01:From https://github.com/fit-r2lab/r2lab-embedded
11-01-39:fit01: * branch master -> FETCH_HEAD
11-01-39:fit01:turn-off-wireless: driver iwlwifi not used
11-01-39:fit01:turn-off-wireless: shutting down device atheros
11-01-40:fit01:turn-off-wireless: removing driver ath9k
11-01-40:fit02:From https://github.com/fit-r2lab/r2lab-embedded
11-01-40:fit02: * branch master -> FETCH_HEAD
11-01-40:fit02:turn-off-wireless: driver iwlwifi not used
11-01-40:fit02:turn-off-wireless: shutting down device atheros
11-01-40:fit02:turn-off-wireless: removing driver ath9k
11-01-40:fit01:Using id=01 and fitid=fit01 - from hostname
11-01-40:fit02:Using id=02 and fitid=fit02 - from hostname
11-01-42:fit01:Using device atheros
11-01-42:fit02:Using device atheros
--- TICK - 11:01:35
--- TICK - 11:01:36
--- TICK - 11:01:37
11-01-37:faraday.inria.fr:Checking current reservation for inria_r2lab.tutorial : OK
--- TICK - 11:01:38
11-01-38:fit02:========== Updating /root/r2lab-embedded for branch master
11-01-38:fit02:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
11-01-38:fit02:Fetching origin
11-01-38:fit01:========== Updating /root/r2lab-embedded for branch master
11-01-38:fit01:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
11-01-38:fit01:Fetching origin
--- TICK - 11:01:39
11-01-39:fit01:Your branch is up-to-date with 'origin/master'.
11-01-39:fit02:Your branch is up-to-date with 'origin/master'.
11-01-39:fit01:Already up-to-date.
11-01-40:fit02:Already up-to-date.
--- TICK - 11:01:40
11-01-40:fit01:loading module ath9k
11-01-40:fit02:loading module ath9k
--- TICK - 11:01:41
--- TICK - 11:01:42
11-01-42:fit01:configuring interface atheros
11-01-42:fit02:configuring interface atheros
--- TICK - 11:01:43
11-01-44:fit01:10.0.0.2 not reachable
--- TICK - 11:01:44
11-01-45:fit01:10.0.0.2 not reachable
--- TICK - 11:01:45
11-01-46:fit01:10.0.0.2 not reachable
--- TICK - 11:01:46
11-01-47:fit01:10.0.0.2 not reachable
--- TICK - 11:01:47
11-01-48:fit01:10.0.0.2 not reachable
--- TICK - 11:01:48
11-01-49:fit01:10.0.0.2 not reachable
--- TICK - 11:01:49
11-01-50:fit01:10.0.0.2 not reachable
--- TICK - 11:01:50
11-01-51:fit01:10.0.0.2 not reachable
11-01-51:fit01:fit01 -> 10.0.0.2: SUCCESS after 8s
Next
In the next example,
we will see a convenience class named Watch
, that comes in
handy in most experiments as it allows to keep track of elapsed time since the beginning of an experiment.
Objective
In this final example in the series, we will take the chance to illustrate
a convenience class named Watch
from asynciojobs
, that allows to keep
track of elapsed time since some reference point in time, generally the
beginning of the experiment.
So here, we create a Watch
instance, and use it when the clock is ticking,
instead of showing plain wall clock time.
The code
#!/usr/bin/env python3 from argparse import ArgumentParser from asynciojobs import Scheduler from asynciojobs import Job from asynciojobs import Watch from apssh import SshNode, SshJob from apssh import Run, RunString, RunScript from apssh import TimeColonFormatter ########## gateway_hostname = 'faraday.inria.fr' gateway_username = 'inria_r2lab.tutorial' verbose_ssh = False parser = ArgumentParser() parser.add_argument("-s", "--slice", default=gateway_username, help="specify an alternate slicename, default={}" .format(gateway_username)) parser.add_argument("-v", "--verbose-ssh", default=False, action='store_true', help="run ssh in verbose mode") parser.add_argument("-d", "--driver", default='ath9k', choices = ['iwlwifi', 'ath9k'], help="specify which driver to use") args = parser.parse_args() gateway_username = args.slice verbose_ssh = args.verbose_ssh wireless_driver = args.driver ########## faraday = SshNode(hostname = gateway_hostname, username = gateway_username, verbose = verbose_ssh, formatter = TimeColonFormatter()) node1 = SshNode(gateway = faraday, hostname = "fit01", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) node2 = SshNode(gateway = faraday, hostname = "fit02", username = "root", verbose = verbose_ssh, formatter = TimeColonFormatter()) ########## # create an orchestration scheduler scheduler = Scheduler() ########## check_lease = SshJob( # checking the lease is done on the gateway node = faraday, # this means that a failure in any of the commands # will cause the scheduler to bail out immediately critical = True, command = Run("rhubarbe leases --check"), scheduler = scheduler, ) ########## # setting up the wireless interface on both fit01 and fit02 init_node_01 = SshJob( node = node1, command = RunScript( "B3-wireless.sh", "init-ad-hoc-network", wireless_driver, "foobar", 2412, ), required = check_lease, scheduler = scheduler, ) init_node_02 = SshJob( node = node2, command = RunScript( "B3-wireless.sh", "init-ad-hoc-network", wireless_driver, "foobar", 2412, label = "ditto", ), required = check_lease, scheduler = scheduler, ) # the command we want to run in node1 is as simple as it gets ping = SshJob( node = node1, required = (init_node_01, init_node_02), command = RunScript( "B3-wireless.sh", "my-ping", '10.0.0.2', 20, # verbose=True, ), scheduler = scheduler, ) ######## # for the fun of it, let's add a job that runs forever and writes # current time every second import time import asyncio async def infinite_clock(watch): while True: print("--- TICK - {}".format(watch.elapsed())) await asyncio.sleep(1) # create a Watch instance for keeping track of elapsed time watch = Watch() # a forever job is not expected to end, instead # it gets killed when the rest of the flock is done with clock_job = Job( infinite_clock(watch), forever=True, scheduler = scheduler, # for the illustrated graph label = "infinite stopwatch", ) ########## # run the scheduler ok = scheduler.orchestrate() # give details if it failed ok or scheduler.debrief() success = ok and ping.result() == 0 # producing a dot file for illustration scheduler.export_as_dotfile("B5.dot") # return something useful to your OS exit(0 if success else 1)
Sample output
12-35-47:fit01:Already on 'master'
12-35-47:fit02:Already on 'master'
12-35-48:fit01:From https://github.com/fit-r2lab/r2lab-embedded
12-35-48:fit01: * branch master -> FETCH_HEAD
12-35-48:fit02:From https://github.com/fit-r2lab/r2lab-embedded
12-35-48:fit02: * branch master -> FETCH_HEAD
12-35-48:fit01:turn-off-wireless: driver iwlwifi not used
12-35-48:fit01:turn-off-wireless: shutting down device atheros
12-35-48:fit02:turn-off-wireless: driver iwlwifi not used
12-35-48:fit02:turn-off-wireless: shutting down device atheros
12-35-48:fit01:turn-off-wireless: removing driver ath9k
12-35-48:fit02:turn-off-wireless: removing driver ath9k
12-35-48:fit02:Using id=02 and fitid=fit02 - from hostname
12-35-48:fit01:Using id=01 and fitid=fit01 - from hostname
12-35-50:fit02:Using device atheros
12-35-50:fit01:Using device atheros
000.000
--- TICK - 000.000
--- TICK - 001.005
12-35-46:faraday.inria.fr:Checking current reservation for inria_r2lab.tutorial : OK
12-35-46:fit01:========== Updating /root/r2lab-embedded for branch master
12-35-46:fit01:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
12-35-46:fit01:Fetching origin
12-35-46:fit02:========== Updating /root/r2lab-embedded for branch master
12-35-46:fit02:HEAD is now at b8050d8 Merge branch 'master' of github.com:fit-r2lab/r2lab-embedded
12-35-46:fit02:Fetching origin
--- TICK - 002.008
12-35-47:fit01:Your branch is up-to-date with 'origin/master'.
12-35-47:fit02:Your branch is up-to-date with 'origin/master'.
--- TICK - 003.011
12-35-48:fit01:Already up-to-date.
12-35-48:fit02:Already up-to-date.
12-35-48:fit02:loading module ath9k
12-35-48:fit01:loading module ath9k
--- TICK - 004.016
--- TICK - 005.017
12-35-50:fit02:configuring interface atheros
12-35-50:fit01:configuring interface atheros
--- TICK - 006.018
12-35-51:fit01:10.0.0.2 not reachable
--- TICK - 007.020
12-35-52:fit01:10.0.0.2 not reachable
--- TICK - 008.020
12-35-53:fit01:10.0.0.2 not reachable
--- TICK - 009.025
12-35-54:fit01:10.0.0.2 not reachable
--- TICK - 010.027
12-35-55:fit01:10.0.0.2 not reachable
--- TICK - 011.029
12-35-56:fit01:10.0.0.2 not reachable
--- TICK - 012.030
12-35-57:fit01:10.0.0.2 not reachable
--- TICK - 013.034
12-35-58:fit01:10.0.0.2 not reachable
--- TICK - 014.035
12-35-59:fit01:10.0.0.2 not reachable
12-35-59:fit01:fit01 -> 10.0.0.2: SUCCESS after 9s
Next
It is now time to wrap up this series.
We now know how to:
have local scripts executed remotely; we have seen that we can either
- write them as plain shell scripts - using
RunScript
, - or embed them right in python strings - using
RunString
;
- write them as plain shell scripts - using
obtain remote outputs using alternate formats, using e.g.
TimeColonFormatter
; see this page for more info on other available formatters;run infinite jobs, that will get properly terminated when all the finite jobs in the scenario are done;
use the
Watch
class to keep track of elapsed time, as opposed to displaying wall clock time.
In the next series of tutorials, we will learn more about transferring files back and forth.