How to build an OpenFlow testbed on an Ubuntu 12.04 VM in VirtualBox

Installing Ubuntu

I started with an Ubuntu Server 12.04.2 64 bit iso, and a VirtualBox VM with 1024MB of RAM and 8GB of hard disk. My version of VirtualBox is 4.0.10r72479 on Windows 7 x64 Professional. The install is pretty normal - if you've never installed Ubuntu Server before, you shouldn't find this too hard - just follow the prompts and keep pressing enter.

This would be a good time to pour yourself a single malt coffee

Don't be too fussy about packages, but as a rule I tend to want to install the OpenSSH server just because it's a good habit to get into - and something that'll trip you up if you forget to just that one time when it's really important.

Pro tip ™

This will take a couple of minutes to finish, and then you'll have an Ubuntu install ready to go. Remember to eject the ISO, and then turn the machine off. Time for the ugly stuff.

Configuring the network stuff

I've set up my testbed with 3x Ubuntu servers, 2 of which were set up like this and left, and a third that we did some special stuff with. For all of them, we'll need to set up extra adaptors on Internal Networks - the two client machines each get a single new adaptor with their own intnet, and the OVS machine (the third one) gets two new adaptors - the first one goes onto intnet1 (to connect to client 1), and the second goes onto intnet2. I've left the original adaptor untouched on all of the machines so we can add packages later without having to break networking.

Edit our new OFSwitch2 VM

Keep Adapter 1 as is so we can download stuff

Intnet1 matches up with client VM 1

Intnet2 to client VM 2

Once you've set this up, find the vbox file for your VM and open it up in your text editor, Make sure you close VirtualBox first - otherwise it won't take your changes. You'll want to add the following lines:

<ExtraDataItem name="VBoxInternal/Devices/e1000/1/LUN#0/Config/IfPolicyPromisc" value="allow-all"/>
<ExtraDataItem name="VBoxInternal/Devices/e1000/2/LUN#0/Config/IfPolicyPromisc" value="allow-all"/>

The secret sauce

That last part is super important - I spend a few hours today and last night trying to figure out why some packets would hit the bridge and others wouldn't - VirtualBox by default will accept broadcasts and unicasts to your address, but not other MAC addresses. Being a switch, you generally want to accept every MAC address except your own, so this is fairly important.

Installing OpenVSwitch

I've used version 1.10 because it's the coolest. Download it to a folder on your VM, untar, and read the INSTALL file because that's what cool kids do. In actual fact, there's a INSTALL.Debian, but that didn't work for me, so I just built it the generic way.

Packages to install (so you don't spend the next hour chasing dependencies):

• build-essential
• pkg-config
• autoconf
• automake
• python-qt-dev
• python-dev
• python-twisted-conch
• libtool

Then run the install

./boot.sh

./configure

make

sudo make install

I'm pleasantly surprised to say that this all worked the first time - just make sure you install all of those packages in one go and it'll work perfectly from the start :)

Running OpenVSwitch

Now is a good time to start up OpenVSwitch to test that everything is working as you should expect - if we do this right, then the OpenFlow part will be easy. Fire up your two client machines, and set up eth1 on both of them to IPs in the same range - I've used 10.1.1.1/24 and 10.1.1.2/24, but use something else if this would clash with your other network.

Once you have them up, start up OpenVSwitch with the following stuff - I've kept them in separate screens to make it easier

Start a screen (screen)

Screen 0:

sudo modprobe openvswitch

sudo ovsdb-tool create

sudo ovsdb-server --remote=ptcp:9999:127.0.0.1

New screen(CTRL+A, C)

Screen 1

sudo ovs-vswitchd tcp:127.0.0.1:9999

Screen 2

ovs-vsctl --db=tcp:127.0.0.1:9999 add-br br0

ovs-vsctl --db=tcp:127.0.0.1:9999 add-port br0 eth1

ovs-vsctl --db=tcp:127.0.0.1:9999 add-port br0 eth2
ovs-vsctl --db=tcp:127.0.0.1:9999 set bridge br0 protocols=OpenFlow12

sudo ifconfig eth1 up

sudo ifconfig eth2 up

If you bring up your client VMs you should be able to ping between them now. If you can, then great - we'll move onto getting OpenFlow working. You need one more line of code, assuming the controller is (or will be) on the same machine:

ovs-vsctl --db=tcp:127.0.0.1:9999 set-controller br0 tcp:127.0.0.1:6633

Getting OpenFlow going

We're on the home straight here. You can install the controller of your choosing, or you can install Ryu with the following instructions:

sudo apt-get install git python-setuptools

git clone http://github.com/osrg/ryu

cd ryu

sudo python setup.py install

You can then sit and watch as it downloads its dependencies from pypi. When it's done, fire up the controller with an app, and you're ready to go.

ryu-manager ryu/app/simple_switch.py

You can check the flow tables (in another screen) with the following command:

sudo ovs-ofctl dump-flows br0

Check out the manpages if you want to learn more:

http://openvswitch.org/cgi-bin/ovsman.cgi?page=utilities%2Fovs-ofctl.8

http://openvswitch.org/cgi-bin/ovsman.cgi?page=utilities%2Fovs-vsctl.8

What next?

You've got an OpenFlow testbed now, you can do what you want with it. Play with different controllers, or different versions of OpenFlow - it's all up to you.

ARP and ping in POX - Building a POX-based OpenFlow router

What are we doing?

Today, we're going to look at how to handle ARP and ICMP ping messages in the OpenFlow controller POX. The results aren't amazing - latency is between 5 and 50 milliseconds (using pypy makes no difference) - but it's an important feature for any layer 3 device.

Why is it important?

If we want to make native router modules in OpenFlow, we need to be able to assign IP addresses to interfaces on our device. This means the router can talk IP to other devices on the network, a vital step towards building an OpenFlow router.

What about RouteFlow?

RouteFlow is a fully-functional OpenFlow router that you can use today, that translates the physical ports on your OpenFlow device to interfaces on a virtual machine. This virtual machine runs a software router daemon like Quagga or BIRD, meaning you can leverage a mature software router instead of making your own.

RouteFlow represents an important step in OpenFlow routing, but I think we can do better. RouteFlow polls the RIB on a virtual machine and translates that to OpenFlow, which means the router daemons don't know that they're talking to the controller.

If we build a clean interface, we can write POX modules for OSPF, IS-IS, BGP and the like, and let them talk directly to the controller.

How to make packets in POX

I love the packet library in POX, it's clean and easy to use. To make your own packets, just do what your network stack normally does - create the payload, wrap that in the layer below, then the layer below that, and once you're at Ethernet you're finished.

Step 1: ARP replies

I've started with the forwarding.l2_learning module from POX, and added some code to the _handle_PacketIn function, just under self.macToPort[packet.src] = event.port (so that MAC addresses are still stored for each new port).

match = of.ofp_match.from_packet(packet)

if ( match.dl_type == packet.ARP_TYPE and

match.nw_proto == arp.REQUEST and

match.nw_dst == IPAddr("10.1.1.253")):

  self.RespondToARP(packet, match, event)

  return

This checks for ARP requests for our hardcoded IP 10.1.1.253, and responds. The code to respond is as follows:

  def RespondToARP(self, packet, match, event):

    # reply to ARP request

    r = arp()

    r.opcode = arp.REPLY

    r.hwdst = match.dl_src

    r.protosrc = IPAddr("10.1.1.253")

    r.protodst = match.nw_src

    r.hwsrc = EthAddr("00:12:34:56:78:90")

    e = ethernet(type=packet.ARP_TYPE, src=r.hwsrc, dst=r.hwdst)

    e.set_payload(r)

    log.debug("%i %i answering ARP for %s" %

     ( event.dpid, event.port,

       str(r.protosrc)))

    msg = of.ofp_packet_out()

    msg.data = e.pack()

    msg.actions.append(of.ofp_action_output(port =

                                          of.OFPP_IN_PORT))

    msg.in_port = event.port

    event.connection.send(msg)

We build an ARP packet by calling the arp() function from pox.lib.packet, and it initialises the packet as follows:

def __init__(self, raw=None, prev=None, **kw):

        packet_base.__init__(self)

        self.prev = prev

        self.hwtype     = arp.HW_TYPE_ETHERNET

        self.prototype  = arp.PROTO_TYPE_IP

        self.hwsrc      = ETHER_ANY

        self.hwdst      = ETHER_ANY

        self.hwlen      = 6

        self.opcode     = 0

        self.protolen   = 4

        self.protosrc   = IP_ANY 

        self.protodst   = IP_ANY

        self.next       = b''

        if raw is not None:

            self.parse(raw)

        self._init(kw)

We just need to set the OPCODE, HWSRC, HWDST, PROTOSRC and PROTODST fields of this. I've done this in the body of the code, but we can simplify it by passing extra arguments as follows:

r = arp( opcode=arp.REPLY, 

         hwsrc=EthAddr("00:12:34:56:78:90"),

         hwdst=match.dl_src,

         protosrc = IPAddr("10.1.1.253"),

         protodst = match.nw_src)

Once we've created the ARP packet, we need to create an Ethernet packet to put this into. This isn't perfect (we should check for VLAN tags and add them, or steal the body of the original packet and modify that), but it works if we're just dealing with a straight Ethernet network.

e = ethernet(type=packet.ARP_TYPE, src=r.hwsrc, dst=r.hwdst)

e.set_payload(r)

Then we send this off to the controller, which sends it out the port it came through. Now we have an IP address that people can find, let's make it respond to something.

Step 2: Ping replies

If we can reply to ARP requests, we can reply to pings. This has a few more layers, but that just makes the code a little longer, not any more complicated.

The ARP reply is easy - we make an ARP packet, then put that in an Ethernet packet. For ping reply, this is what we do:

1. Get the payload from the echo request (ping)
2. Create an echo reply packet, insert the old payload
3. Create an ICMP packet, insert the echo reply
4. Create an IPv4 packet, insert the ICMP
5. Create an Ethernet packet, insert the IPv4

Here's what the code looks like:

  def RespondToPing(self, ping, match, event):

    p = ping

    # we know this is an ICMP Echo packet, so loop through

    # maybe this needs a try... except?

    while not isinstance(p, echo):

      p = p.next

    

    r = echo(id=p.id, seq=p.seq)

    r.set_payload(p.next)

    i = icmp(type=0, code=0)

    i.set_payload(r)

    ip = ipv4(protocol=ipv4.ICMP_PROTOCOL,

              srcip=IPAddr("10.1.1.253"),

              dstip=match.nw_src)

    ip.set_payload(i)

    e = ethernet(type=ping.IP_TYPE,

                 src=match.dl_dst,

                 dst=match.dl_src)

    e.set_payload(ip)

    log.debug("%i %i answering PING for %s" % (

              event.dpid, event.port,

              str(match.nw_src)))

    msg = of.ofp_packet_out()

    msg.data = e.pack()

    msg.actions.append(of.ofp_action_output(port =

                                          of.OFPP_IN_PORT))

    msg.in_port = event.port

    event.connection.send(msg)

Simple, just slightly longer than the ARP code.

Pictures

Here's a look at the controller output, and the view from Wireshark.

The OpenFlow dissector for Wireshark is part of the OpenFlow reference switch. It's a few years old, and uses an obselete API call - I can put up a patch if anyone gets stuck.

Next steps

• ARP tables - if we're going to route traffic, we need to find the MAC addresses of destination IPs so that we send traffic to them
• Routing protocol - RIP and OSPF will be fairly easy, BGP will be a bit harder due to relying on TCP. These can all be added to POX as modules
• TUN/TAP support - we can create TUN/TAP interfaces and let the linux TCP stack do the hard work for us. This means a BGP module would create the TUN/TAP interface and handle OpenFlow encapsulation/decapsulation, but could offload the TCP to the Linux stack.

DjangoFlow - Web UI for the POX OpenFlow controller

The Plan

OpenFlow is a simple concept - an open interface to switch and router hardware. Can we tie this into an open web framework to create a foundation for a really simple Web UI?

The tools

I've been learning how to use Django recently, and it seems like a perfect choice for this task. It's written in Python, so it integrates easily with the POX controller. I've also used virtualenv to make it more portable - this means the project is largely self-contained and can be copied to a new system by simply copying the folder.

Implementation

To start, I created a virtualenv called "djangoflow" on an Ubuntu machine and installed django in it. There are lots of ways to do this - do a google for "django setup virtualenv ubuntu" and you'll get tons of hits.

I made a project called mysite, and an app called flew (NZ english for "flow"), and left that bit for the time being.

The folder structure then looks something like this:

djangoflow

- bin

- include

- lib

- local

- mysite

--- flew

--- mysite

I then did a git clone of the latest build of POX from the NOXREPO github, into the djangoflow folder, and then copied everything from the base mysite folder into the pox folder. The folder structure then looks like:

djangoflow

- bin

- include

- lib

- local

- pox

--- .git

--- flew

--- mysite

--- pox

This is great - now back to Django.

The model that I used was really simple, here's what I've got so far:

from django.db import models

# Create your models here.

class Flow(models.Model):

internalip = models.CharField(max_length=200)

externalip = models.CharField(max_length=200)

idletime = models.IntegerField()

hardtime = models.IntegerField()

def __unicode__(self):

return "Internal: " + self.internalip + ", External: " + self.externalip

class User(models.Model):

name = models.CharField(max_length=200)

class Device(models.Model):

dpid = models.CharField(max_length=200)

We'll only use the Flow model today, the others are for expansion later. To make this work, we'll need to edit the settings in mysite.settings - set up the databases and installed_apps sections as follows:

DATABASES = {

    'default': {

        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.

        'NAME': 'flew.db',                      # Or path to database file if using sqlite3.

        'USER': '',                      # Not used with sqlite3.

        'PASSWORD': '',                  # Not used with sqlite3.

        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.

        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.

    }

}

INSTALLED_APPS = (

    'django.contrib.auth',

    'django.contrib.contenttypes',

    'django.contrib.sessions',

    # 'django.contrib.sites',

    'django.contrib.messages',

    'django.contrib.staticfiles',

    # Uncomment the next line to enable the admin:

    'django.contrib.admin',

    # Uncomment the next line to enable admin documentation:

    'django.contrib.admindocs',

    'flew',

)

Awesome, now run syncdb (check against whatever tutorial you use to see how this is done) and populate the database. Now, fire up the shell and create your first flow:

python manage.py shell

from flew.models import Flow

f = Flow(internalip = "10.1.10.2", externalip = "10.1.20.2", idletime = 300, hardtime = 3600)

f.save()

If that all worked, then you'll have a new entry in your database. We are never going to access the database directly though - we can access all the Django goodness from POX.

Open up pox/forwarding/l2_learning.py and have a look through - if you've used this before, then good, if not, then see what it all does.

I've hijacked this just for this example, but it sets a starting point for any POX-Django integrated tools. Make sure your imports section looks like this:

# import django stuff

from django.core.management import setup_environ

from mysite import settings

setup_environ(settings)

from flew.models import Flow

from pox.core import core

import pox.openflow.libopenflow_01 as of

from pox.lib.revent import *

from pox.lib.util import dpidToStr

from pox.lib.util import str_to_bool

from pox.lib.packet import ethernet

from pox.lib.packet import ipv4

import time

Then go down to the __init__ function for LearningSwitch and add this to the end:

    # add new flow by default

    flow1 = Flow.objects.all()[0]

    msg = of.ofp_flow_mod()

    msg.match = of.ofp_match()

    msg.match.dl_type = ethernet.IP_TYPE

    msg.match.nw_src = str(flow1.internalip)

    msg.match.nw_dst = str(flow1.externalip)

    msg.idle_timeout = flow1.idletime

    msg.hard_timeout = flow1.hardtime

    msg.actions.append(of.ofp_action_output(port = 1))

    #msg.buffer_id = event.ofp.buffer_id # 6a

    self.connection.send(msg)

What does this do? It takes the first Flow out of our database, creates a flow to allow traffic from internalip, going to externalip, to go out port 1, which should be pointing at the outside world. This flow will stay in the switch for an hour, or 5 minutes without being triggered - whichever happens first.

Question time

Q: Is it really this easy?

A: Yes. Python is easy, Django is easy, Virtualenv is easy, POX is easy. It's just a little tricky making them all work together - that's what this guide is for.

Q: Why add flows this way when we already have a CLI?

A: Django has a clean admin interface that I haven't covered here (check the setup tutorial on the Django website) - you can set flows in there, and every time a switch connects, it will use those flows.

Q: Could I start using this right now?

A: Totally. If you want your switches to retrieve their configuration from the controller automatically when they start up, you can set a bunch of super specific flows here and roll it out now. If you want something a bit more clever, then you can jump into the Django and POX API and make it happen yourself.

Q: What next?

A: There are two extra models that we didn't use - one for user authentication, and one for managing devices. If your OpenFlow devices connect by SSL (which they should in the real world) then you can create models for them that hold particular flows - this can all be managed via a web interface. As for user authentication, there are millions of ways to do this - you can set privileges for who can set certain flows, make requests that admins can approve - the possibilities are endless!

Polishing pyswitch

Polishing pyswitch

I've had my modified version of pyswitch running on NOX for a couple of weeks, and it's working fine. The key to OpenFlow is the controller - if your controller is processing a lot of packets, then it's a bottleneck; but if all your traffic is matching flows in the switch, then it will work at line speed.

As I've been using the switch for more and more test servers, I've noticed that my modifications have oversimplified things a little. Here's a summary of the current pyswitch logic:

1. If a packet doesn't match a flow in the switch, send to the controller
2. For each packet sent to the controller, save the source address and source port
3. If the controller gets a packet with a destination address it knows, it sends it to that port and installs a new flow into the switch

Do you see the problem? It's fine with two computers on the switch, but here's how it works with three:

1. PC A sends a packet to PC B. No flows in the switch so the controller gets the packet, saves the address and port of A, and floods the packet
2. PC B replies. No flow matched, controller gets the packet, saves the address and port of B, and recognises PC A. Controller then forwards the packet to the port that PC A was seen on, and installs a flow into the switch
3. PC A sends another packet to PC B. No flow matched, controller gets the packet, recognises address of PC B so it forwards the packet and stores a flow in the switch.
4. Flows are in the switch for both PC A and PC B, so packets to them are sent at line speed without touching the controller

What happens when PC C comes along?

1. PC C sends a packet to PC A. There is a flow for this, so it is forwarded at line speed in the switch
2. PC A replies to PC C. No flow, so the controller gets the packet, saves the source details (address and port of PC A), doesn't have details of PC C so it floods the packet

Do you see the problem? The source details of PC C never get stored, because all its outbound packets match flows in the switch. This is a serious problem - it means that all of the traffic back to PC C goes through the OpenFlow controller at about 10 packets per second, breaking the network.

The original pyswitch didn't have this problem - it created very specific flows based on all the source and destination attributes. I could have fixed it up to handle VLANS better (by making it recognise ethertype 0x8100 as VLAN and move up the header for the actual ethertype), but this isn't efficient - a connection to a website would have 2 flows for the original arp requests, another 2 for the dns lookup, and another 2 for the TCP connection - 6 flows for a single web page?

We could strike a compromise and set flows based on the source and destination MAC addresses, but I still don't like that. It means that for N MAC addresses on the switch, you go from N flows to NxN flows; for a 48-port switch, this is from 48 flows to 2,304 flows. It may be a case of trading extra flows for simpler code, but I think I have a better solution.

My new addition to pyswitch adds a flow to the switch whenever it has to flood a packet. The idea is, when PC C comes along and sends a packet, we want that to go to the controller, even if we know the destination. Here's the new code:

# --

# If we've learned the destination MAC set up a flow and

# send only out of its inport.  Else, flood.

# --

def forward_l2_packet(dpid, inport, packet, buf, bufid):    

    dstaddr = packet.dst.tostring()

    if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr):

        prt = inst.st[dpid][dstaddr]

        if  prt[0] == inport:

            log.err('**warning** learned port = inport', system="pyswitch")

            logger.info('**warning** learned port = inport')

            inst.send_openflow(dpid, bufid, buf, openflow.OFPP_ALL, inport)

        else:

            # We know the outport, set up a flow

            log.msg('installing flow for ' + mac_to_str(packet.dst), system="pyswitch")

            logger.info('installing flow for ' + mac_to_str(packet.dst))

            # delete src flow if exists

            delflow = {}

            delflow[core.DL_SRC] = packet.dst

            inst.delete_datapath_flow(dpid, delflow)

            # sam edit - just load dest address, the rest doesn't matter

            flow = create_l2_out_flow(packet)

            actions = [[openflow.OFPAT_OUTPUT, [0, prt[0]]]]

            inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT, 

                                       openflow.OFP_FLOW_PERMANENT, actions,

                                       bufid, openflow.OFP_DEFAULT_PRIORITY,

                                       inport, buf)

    else:    

        # haven't learned destination MAC. Flood 

        if ord(dstaddr[0]) & 1:

            logger.info('broadcast/multicast packet to ' + mac_to_str(packet.dst) + ', flooding')

            inst.send_openflow(dpid, bufid, buf, openflow.OFPP_ALL, inport)

        else:

            logger.info('no MAC known for ' + mac_to_str(packet.dst) + ', flooding')

            # set up flow to capture source packet

            flow = {}

            flow[core.DL_SRC] = packet.dst

            actions = [[openflow.OFPAT_OUTPUT, [65535, openflow.OFPP_CONTROLLER]]]

            inst.send_openflow(dpid, bufid, buf, openflow.OFPP_ALL, inport)

            inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT,

                                       1, actions,

                                       None, openflow.OFP_DEFAULT_PRIORITY+1,

                                       None, None)

Pay attention to the install_datapath_flow() functions. If we start from the bottom, you'll see that the else statement is a lot larger. Broadcast/multicast packets get flooded, but unknown packets also install a flow (at default priority+1) so that any packets from this unknown host come to the controller. This is matched by a delete_datapath_flow() call further up the function, so that when a new flow is installed, it tries to delete any flows that match the source address.

How does it perform? Each new flow sends roughly 3 packets to the controller (the first unknown, and a couple because of our source-match flow - it doesn't get deleted before the next queued packet comes through), but we get our O(N) amount of flows in the table. If we look at our ARP + UDP + TCP example from before, it performs way better - for 6 flows the controller gets 6 packets, but for our 2 flows the controller also gets 6 packets. This means it uses the controller as much as the old, specific pyswitch, but uses a fraction of the flows.

OFPP_FLOOD vs OFPP_ALL

One extra note for those of you who haven't spotted it - I've changed the action from OFPP_FLOOD to OFPP_ALL. The Pronto 3290 we have at work has always responded to FLOOD messages weirdly - it looks like it sets up individual flows for each active port, and after trolling through the OpenFlow spec I've figured out why:

OpenFlow-only switches support only the required actions below, while OpenFlow-

enabled switches, routers, and access points may also support the NORMAL

action. Either type of switch can also support the FLOOD action.

Required Action: Forward. OpenFlow switches must support forwarding

the packet to physical ports and the following virtual ones:

• ALL: Send the packet out all interfaces, not including the incoming in-

terface.

• CONTROLLER: Encapsulate and send the packet to the controller.

• LOCAL: Send the packet to the switchs local networking stack.

• TABLE: Perform actions in flow table. Only for packet-out messages.

• IN PORT: Send the packet out the input port.

Optional Action: Forward. The switch may optionally support the following

virtual ports:

• NORMAL: Process the packet using the traditional forwarding path

supported by the switch (i.e., traditional L2, VLAN, and L3 processing.)

The switch may check the VLAN field to determine whether or not to

forward the packet along the normal processing route. If the switch can-

not forward entries for the OpenFlow-specific VLAN back to the normal

processing route, it must indicate that it does not support this action.

• FLOOD: Flood the packet along the minimum spanning tree, not includ-

ing the incoming interface.

See the difference? FLOOD is an optional action, that activates any spanning-tree code in the switch. It's not as intensive as NORMAL (which only true hybrid switches will support), but it isn't what pyswitch is supposed to do. Changing the code to use OFPP_ALL instead of OFPP_FLOOD seems to make the switch work less on each packet that comes back from the controller - and this means the controller can handle even more flows per second!

Here's a code dump of my latest version, I may polish it and send it back to the NOX dudes later if I get the time:

# Copyright 2008 (C) Nicira, Inc.

# 

# This file is part of NOX. Additions from Sam Russell for

# compatibility with OVS on Pronto 3920

# 

# NOX is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

# 

# NOX is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

# 

# You should have received a copy of the GNU General Public License

# along with NOX.  If not, see <http://www.gnu.org/licenses/>.

# Python L2 learning switch 

#

# ----------------------------------------------------------------------

#

# This app functions as the control logic of an L2 learning switch for

# all switches in the network. On each new switch join, it creates 

# an L2 MAC cache for that switch. 

#

# In addition to learning, flows are set up in the switch for learned

# destination MAC addresses.  Therefore, in the absence of flow-timeout,

# pyswitch should only see one packet per flow (where flows are

# considered to be unidirectional)

#

from nox.lib.core     import *

from nox.lib.packet.ethernet     import ethernet

from nox.lib.packet.packet_utils import mac_to_str, mac_to_int

from twisted.python import log

import logging

from time import time

from socket import htons

from struct import unpack

logger = logging.getLogger('nox.coreapps.examples.pyswitch')

# Global pyswitch instance 

inst = None

# Timeout for cached MAC entries

CACHE_TIMEOUT = 5

# Modified extract_flow except just dest address - another sam edit

def create_l2_out_flow(ethernet):

    attrs = {}

    attrs[core.DL_DST] = ethernet.dst

#    attrs[core.DL_SRC] = ethernet.src

    return attrs

# --

# Given a packet, learn the source and peg to a switch/inport 

# --

def do_l2_learning(dpid, inport, packet):

    global inst 

    logger.info('learning MAC for incoming packet...' + mac_to_str(packet.src))

    # learn MAC on incoming port

    srcaddr = packet.src.tostring()

    if ord(srcaddr[0]) & 1:

        log.msg('MAC is null', system='pyswitch')

        logger.info('MAC is null')

        return

    if inst.st[dpid].has_key(srcaddr):

        dst = inst.st[dpid][srcaddr]

        if dst[0] != inport:

            log.msg('MAC has moved from '+str(src)+'to'+str(inport), system='pyswitch')

            logger.info('MAC has moved from '+str(src)+'to'+str(inport))

        else:

            return

    else:

        logger.info('learned MAC '+mac_to_str(packet.src)+' on %d %d'% (dpid,inport))

    # learn or update timestamp of entry

    inst.st[dpid][srcaddr] = (inport, time(), packet)

    # Replace any old entry for (switch,mac).

    mac = mac_to_int(packet.src)

# --

# If we've learned the destination MAC set up a flow and

# send only out of its inport.  Else, flood.

# --

def forward_l2_packet(dpid, inport, packet, buf, bufid):    

    dstaddr = packet.dst.tostring()

    if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr):

        prt = inst.st[dpid][dstaddr]

        if  prt[0] == inport:

            log.err('**warning** learned port = inport', system="pyswitch")

            logger.info('**warning** learned port = inport')

            inst.send_openflow(dpid, bufid, buf, openflow.OFPP_ALL, inport)

        else:

            # We know the outport, set up a flow

            log.msg('installing flow for ' + mac_to_str(packet.dst), system="pyswitch")

            logger.info('installing flow for ' + mac_to_str(packet.dst))

            # delete src flow if exists

            delflow = {}

            delflow[core.DL_SRC] = packet.dst

            inst.delete_datapath_flow(dpid, delflow)

            # sam edit - just load dest address, the rest doesn't matter

            flow = create_l2_out_flow(packet)

            actions = [[openflow.OFPAT_OUTPUT, [0, prt[0]]]]

            inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT, 

                                       openflow.OFP_FLOW_PERMANENT, actions,

                                       bufid, openflow.OFP_DEFAULT_PRIORITY,

                                       inport, buf)

    else:    

        # haven't learned destination MAC. Flood 

        if ord(dstaddr[0]) & 1:

            logger.info('broadcast/multicast packet to ' + mac_to_str(packet.dst) + ', flooding')

            inst.send_openflow(dpid, bufid, buf, openflow.OFPP_ALL, inport)

        else:

            logger.info('no MAC known for ' + mac_to_str(packet.dst) + ', flooding')

            # set up flow to capture source packet

            flow = {}

            flow[core.DL_SRC] = packet.dst

            actions = [[openflow.OFPAT_OUTPUT, [65535, openflow.OFPP_CONTROLLER]]]

            inst.send_openflow(dpid, bufid, buf, openflow.OFPP_ALL, inport)

            inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT,

                                       1, actions,

                                       None, openflow.OFP_DEFAULT_PRIORITY+1,

                                       None, None)

        

# --

# Responsible for timing out cache entries.

# Is called every 1 second.

# --

def timer_callback():

    global inst

    curtime  = time()

    for dpid in inst.st.keys():

        for entry in inst.st[dpid].keys():

            if (curtime - inst.st[dpid][entry][1]) > CACHE_TIMEOUT:

                log.msg('timing out entry'+mac_to_str(entry)+str(inst.st[dpid][entry])+' on switch %x' % dpid, system='pyswitch')

                inst.st[dpid].pop(entry)

    inst.post_callback(1, timer_callback)

    return True

def datapath_leave_callback(dpid):

    logger.info('Switch %x has left the network' % dpid)

    if inst.st.has_key(dpid):

        del inst.st[dpid]

def datapath_join_callback(dpid, stats):

    logger.info('Switch %x has joined the network' % dpid)

# --

# Packet entry method.

# Drop LLDP packets (or we get confused) and attempt learning and

# forwarding

# --

def packet_in_callback(dpid, inport, reason, len, bufid, packet):

    if not packet.parsed:

        log.msg('Ignoring incomplete packet',system='pyswitch')

        

    if not inst.st.has_key(dpid):

        log.msg('registering new switch %x' % dpid,system='pyswitch')

        inst.st[dpid] = {}

    # don't forward lldp packets    

    if packet.type == ethernet.LLDP_TYPE:

        return CONTINUE

    # learn MAC on incoming port

    do_l2_learning(dpid, inport, packet)

    forward_l2_packet(dpid, inport, packet, packet.arr, bufid)

    return CONTINUE

class pyswitch(Component):

    def __init__(self, ctxt):

        global inst

        Component.__init__(self, ctxt)

        self.st = {}

        inst = self

    def install(self):

        inst.register_for_packet_in(packet_in_callback)

        inst.register_for_datapath_leave(datapath_leave_callback)

        inst.register_for_datapath_join(datapath_join_callback)

        inst.post_callback(1, timer_callback)

    def getInterface(self):

        return str(pyswitch)

def getFactory():

    class Factory:

        def instance(self, ctxt):

            return pyswitch(ctxt)

    return Factory()

Thugs at New Zealand skateparks

This is totally off-topic, but I saw this and it made me really angry. I used to love skateboarding when I was a kid, and the thought of grown adults shoulder-charging young kids off skateboards is just disgusting.

 
Today At Vic Park from NZskate.com on Vimeo.

In other news, I've been playing with the source code to BIRD to see if I can put in some hooks to make it work with NOX. It has a set of OS-specific "kernel" modules which install routes into the OS routing tables, so it shouldn't be hard to make a NOX version of this.

Another option would be to monitor the MRT dump file, or point it at a named pipe - then get NOX to use that to pick up updates.

One thing that Nick Buraglio pointed out was that BIRD is in need of an IS-IS implementation. IPv6 has been a good excuse to move from OSPF to IS-IS because it all runs in one instance - not requiring OSPFv2 and v3 instances for IPv4 and IPv6. Time will tell if there's still a place for decentralised IGP's in an OpenFlow world, but if we're going to see an influx of software routers, then IS-IS will definitely be added in the next year or two. (edit - somebody's beaten me to add this to quagga http://code.google.com/p/google-quagga/)

More details once I get BIRD talking to NOX though!