How to connect OpenFlow switches to multiple controllers (Mininet)

Original post is here.


from import Mininet
from mininet.node import Controller, RemoteController
from mininet.cli import CLI
from mininet.log import setLogLevel, info

def myNet():

    #OpenDayLight controller

    #Floodlight controller

    net = Mininet( topo=None, build=False)

    # Create nodes
    h1 = net.addHost( 'h1', mac='01:00:00:00:01:00', ip='' )
    h2 = net.addHost( 'h2', mac='01:00:00:00:02:00', ip='' )

    # Create switches
    s1 = net.addSwitch( 's1', listenPort=6634, mac='00:00:00:00:00:01' )
    s2 = net.addSwitch( 's2', listenPort=6634, mac='00:00:00:00:00:02' )

    print "*** Creating links"
    net.addLink(h1, s1, )
    net.addLink(h2, s2, )   
    net.addLink(s1, s2, )  

    # Add Controllers
    odl_ctrl = net.addController( 'c0', controller=RemoteController, ip=ODL_CONTROLLER_IP, port=6633)

    fl_ctrl = net.addController( 'c1', controller=RemoteController, ip=FL_CONTROLLER_IP, port=6633)

    # Connect each switch to a different controller
    s1.start( [odl_ctrl] )
    s2.start( [fl_ctrl] )

    s1.cmdPrint('ovs-vsctl show')

    CLI( net )

if __name__ == '__main__':
    setLogLevel( 'info' )

Openvswitch Module

If you’re getting kernel module installation errors, probably the module is not compiled for the current kernel or its dependencies are not loaded.

# insmod datapath/linux/openvswitch.ko


datapath/linux/openvswitch.ko: Unknown symbol in module

By using the command modinfo, you can take clues about the module, such as its dependencies and kernel version that was used to compile.

If the problem is dependencies, instead of using insmod to install openvswitch.ko module, you can first type the command in the OVS root directory (after compiling):

# make install

This will install all compiled modules in the Linux system. Then, type:

# modprobe openvswitch

The command should automatically load openvswitch module and its dependencies.

OVS kernel’s table entry timeout

How long a kernel’s table entry can be inactive? The OVS kernel’s table can inactive, by default, during a time interval less than 5 seconds. After 5 seconds being inactive, the flow entry is removed from the table. This time is dictated by ovs-vswitchd.

From the original post:

”ovs-vswitchd determines how long the flow should stay in the kernel flow table. By default, it expires a flow after five seconds of inactivity. It can be more aggressive if the flow table becomes large.”

ovs-dpctl: description ‘used=never’

Original post here.

When using ovs-dpctl, there are flow descriptions with “used=never”. What does it mean? It means that a packet went up to the userspace (ovs-vswitchd), a match was found and it was copied to the kernel space. However, there were no more following packets that matched the rule cached in the kernel’s table.


Differences between ovs-ofctl and ovs-dpctl

Basically, ovs-ofctl is used to look up for the actual flow table entries. And, ovs-dpctl can be used to see what is inside of the OVS’s kernel table, which can be considered as flow table, but OVS kernel module actually does not speak OpenFlow, i.e., it doesn’t handle OpenFlow messages.

The number of entries retrieved by ovs-ofctl and ovs-dpctl can be different. That is because only the matched entries in user-space are copied to the kernel-space. More details may be found in the OVS official website.

From the original post.
ovs-ofctl dump-flows” prints OpenFlow flow table entries.

ovs-dpctl dump-flows” is different.

As the manpage says:

This command is primarily useful for debugging Open vSwitch. The flow table entries that it displays are not OpenFlow flow entries. Instead, they are different and considerably simpler flows maintained by the Open vSwitch kernel module.

In a little more detail, the flows that ovs-dpctl prints are always exact-match. They reflect packets that have actually passed through the system in the last 5 seconds or so.

If you want to know what OpenFlow sees, use ovs-ofctl. If you want to verify that the OpenFlow flows are being implemented in the way you expect, ovs-dpctl is the right tool.

A FAQ about OVS flow table

Text extracted from the Open vSwitch’s FAQ.

"### Q: Can OVS populate the kernel flow table in advance instead of in reaction to packets?

A: No.  There are several reasons:

  - Kernel flows are not as sophisticated as OpenFlow flows, which
    means that some OpenFlow policies could require a large number of
    kernel flows.  The "conjunctive match" feature is an extreme
    example: the number of kernel flows it requires is the product of
    the number of flows in each dimension.

  - With multiple OpenFlow flow tables and simple sets of actions, the
    number of kernel flows required can be as large as the product of
    the number of flows in each dimension.  With more sophisticated
    actions, the number of kernel flows could be even larger.

  - Open vSwitch is designed so that any version of OVS userspace
    interoperates with any version of the OVS kernel module.  This
    forward and backward compatibility requires that userspace observe
    how the kernel module parses received packets.  This is only
    possible in a straightforward way when userspace adds kernel flows
    in reaction to received packets.

  For more relevant information on the architecture of Open vSwitch,
  please read "The Design and Implementation of Open vSwitch",
  published in USENIX NSDI 2015."

How to configure ONOS

ONOS is a carrier-grade network operating system (OpenFlow controller). Here, I use the version 1.3 and the configurations below, worked for me.

Download ONOS
$ cd ~; mkdir sdn; cd sdn
$ git clone -b onos-1.3
Installation – Java, Maven, and Karaf
Maven and Karaf
$ cd ~/sdn
$ wget
$ wget
$ tar -zxvf apache-karaf-3.0.3.tar.gz -C ~/sdn
$ tar -zxvf apache-maven-3.3.1-bin.tar.gz -C ~/sdn
Oracle Java 8:
$ sudo apt-get install software-properties-common -y
$ sudo add-apt-repository ppa:webupd8team/java -y
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer oracle-java8-set-default -y
Setting Environment Variables (~/.bashrc). The environment variables can be optional (I’m not sure, but I confired such vars in the .bashrc).
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export JRE_HOME=/usr/lib/jvm/java-8-oracle/jre
export KARAF_ROOT=$HOME/sdn/apache-karaf-3.0.3
export M2_HOME=$HOME/sdn/apache-maven-3.3.1
Development Environment Setup (~/.bashrc)
$ export ONOS_ROOT=$HOME/sdn/onos
$ source $ONOS_ROOT/tools/dev/bash_profile
Building and packaging ONOS 
$ cd ~/sdn/onos
$ mvn clean install
Selecting IP address (~/.bashrc)
Put your IP address. 
$ export ONOS_IP=A.B.C.D
Starting ONOS
$ ok clean # or onos-karaf
GUI: karaf/karaf
http://<Your IP address>:8181/onos/ui/login.html