ECOC 2018 - Demo session

Background

One of the submitted ECOC 2018 demo session entries ( https://www.ecoc2018.org/programme/demo-session/ ) will feature TSN solution with SDN management based on YANG/NETCONF and the yuma123 toolchain. Testsuite with scripts validating the setup, allowing management of traffic generation and visualization of traffic parameters in the topology are published under BSD-3-clause license and are a good application example https://sourceforge.net/projects/ecoc-demo-2018

Below an excerpt of the README is presented:

Files

• topology.svg - topology diagram
• topology.xml - ietf-network model based network spec. Edit to change network addresses, authorization credentials etc.
• configure.ac - autoconf build script
• Makefile.am - automake build script (contains the list of all testcases)

Running

apt-get update
apt-get -y upgrade

apt-get -y install software-properties-common
add-apt-repository -s "deb http://yuma123.org/repos/apt/debian stretch main"
wget -O - http://yuma123.org/repos/yuma123.gpg.key | apt-key add -
apt-get update

apt-get -y install testsuite-h100-ecoc-demo
cd /opt/testsuite-h100-ecoc-demo
autoreconf -i -f
./configure

Make sure your topology.xml has correct server/port/username/password configuration:

make check

This should complete with OKs (some scripts report fail if there is external traffic generator incrementing counters)

To start the continuous monitor (updates the files referenced by monitor/index.html for graphical representation of the traffic and periodic reports):

./test-monitor

Here is example of the output of the monitor:

Report time: 20:25:19

1. Period:
+------------+---------------------+---------------------+--------------+
|    node    |       start         |         stop        |     period   |
+------------+---------------------+---------------------+--------------+
| middle     | 2018-09-22T18:24:20 | 2018-09-22T18:25:00 |       40.000 |
| local      | 2018-09-22T18:24:20 | 2018-09-22T18:25:00 |       40.000 |
| remote     | 2018-09-22T18:24:20 | 2018-09-22T18:25:00 |       40.000 |
+------------+---------------------+---------------------+--------------+
2. Data transfer rate:
+------------+--------------+-------+
|  if name   |       MB/s   |   %   |
+------------+--------------+-------+
| middle.xe0 |       3412   |   29  |
| remote.xe0 |       6773   |   57  |
+------------+--------------+-------+

3. Timing:
+-----------+--------------+-------------+--------------+---------------+
|  if name  |  delay-min   |  delay-max  |      PDV     |     samples   |
+-----------+--------------+-------------+--------------+---------------+
| local.ge1 |      7353 ns |      7456 ns|       103 ns |    228883040  |
| local.ge2 |      7347 ns |      7449 ns|       102 ns |    229662467  |
| local.ge3 |      7398 ns |      7494 ns|        96 ns |    230612730  |
| local.ge4 |         0 ns |         0 ns|         0 ns |            0  |
+-----------+--------------+-------------+--------------+---------------+

Device side

There are 3 scripts in ./setup (run-netconfd-local.sh, run-netconfd-middle.sh, run-netconfd-remote.sh) that can start the netconfd instances for each node with the correct parameters. Use those to either connect to the VCU-110 cards directly or to connect to another netconfd providing register access. In case there is external host that runs netconfd make sure the parameters are identical e.g. all expected modules are loaded.

Reports

• test-suite.log - overview
• test-*.log - individual test reports

Test Network Setup

2x H100 1.6 prototype VCU-110 systems, 1x H100 1.0 prototype and an optional traffic analyzer.

Further in the document the elements will be referred to as:

• local - the left H100 1.6
• remote - the right H100 1.6
• middle - the H100 1.0 in the middle
• analyzer (optional)

Details about the actual networking addresses and authorization are specified in the topology.xml file and if there is a change in the setup this file needs to be updated before running.

References

1. Transpacket AS, "VCU-110 Prototype r1.6"
2. RFC6241 Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, <http://www.rfc-editor.org/info/rfc6241>.

Test specification

Test cases included in the suite:

1. validate-topology-config
2. netconf-node-access
3. setup-idle
4. setup-fusion-streams-yangcli
5. setup-network-bridge-yangcli
6. get-net-[1,2-1,3,2-2]
7. setup-traffic-analyzers
8. setup-loopbacks
9. setup-traffic-generators
10. diff-net
11. set-net-2-1

Every test case is implemented as executable named test-<testcase-name> and can be started individually e.g ./test-setup-idle

The test cases produce output with formal NETCONF transactions data (state and configuration) so that every single action taken by the test case can be traced back and reported results according to the "Reported results" section described in each test case. All the information is printed to the standard output (no other files containing test report data are created). The return status of the executable is indication of the requirements for the specified validations and acceptance criteria were met. Following the common Unix convention 0=success, not(0)=error-code.

A detailed specification for each test case part of the suite follows with this information structure:

+ Objective
+ Procedure
+ DUT Configuration(s)
+ Steps
+--- []
   +--- Validates ...
   +--- Returns ...
+ Reported results
+ Acceptance criteria

Testcase: validate-topology-config

Objective

Validate the topology.xml has correct XML format and no YANG syntactic or semantic errors.

Procedure

Call netconfd in --validate-config-only mode.

Testcase: netconf-node-access

Objective

Validate all nodes part of the test setup network are accessible over NETCONF.

Procedure

Establish connections to all nodes specified in the topology.xml file using the common testapi API.

DUT configuration

N.A

Steps

1. Connect to all nodes specified in the topology.xml with testapi.network_connect(network)
   1. Validate the operation succeeds.

Testcase: setup-idle

Objective

Load idle configurations to local, remote and middle and verify idle devices behave as expected.

Procedure

After committing idle configurations validate not counters are running.

DUT configuration

Idle configuration e.g. delete /fusion-streams and other top level containers.

Steps

1. Delete /fusion-streams on local and remote and commit.
   1. No counters are being incremented.

Testcase: setup-*

Objective

Each of the setup-* testcases carry out a incremental step in the sequence of creating a complete benchmark test.

Testcase: get-net-[0-12]

Objective

Using the get-net commannd line tool create snapshot of the network for each transaction.

Testcase: set-net-*

Objective

Using the set-net commannd line tool restore configuration snapshot of the network of a previously captured with get-net configuration.

Testcase: diff-net

Objective

Compare and print all counters of 2 consequtive network snapshot copies.