SDS/IC2E 2015 - An Expressive Simulator for Dynamic Network Flows

2nd IEEE International Workshop on Software Defined Systems (SDS -2015) was held at the Arizona State University Tempe Campus, USA in conjunction with 2015 IEEE International Conference on Cloud Engineering (IC2E 2015) from 09-12 March 2015. I presented my paper titled, "An Expressive Simulator for Dynamic Network Flows". The presentation slides and abstract are given below.

Abstract -

Software-defined networking (SDN) increases the network programmability, promoting an effective development of networked systems of cloud scale. As the scale of the networks and systems is growing larger and larger with time, programmability of the systems and networks is researched intensively. Many emulators are proposed and implemented to emulate large and complex networks inside a single computer, or a cluster of computers in the research lab. However, the emulators lack the ability to represent large systems such as data center networks or content delivery networks. Many of the network algorithms and design choices can also be tested for their functionality and efficiency in a simulator environment. While network emulators and simulators exist, a generic network flow simulator that is easy to program a variety of highly distributed and gigantic systems is still lacking. This paper presents xSDN , an expressive simulator for dynnamic network flows. Adhering to the principles of software-defined networking paradigm from the design, xSDN focuses to be lean, light-weight, easy to learn and configure, and efficient, that can simulate networks of a scale of million nodes within a few seconds.

SDNSim, A Compact Simulator for Software-Defined Networking

Magnificent view from Cacilhas..

The weather is getting colder. I am reaching my third winter in Lisbon. Time indeed goes fast. I am currently building SDNSim, a compact and expressive simulator for Software-Defined Networking (SDN) built using Java, extending the xSDN, an Expressive Simulator for Dynamic Network Flows.

Today I marked all the emails in Gmail as "Read". It was slightly more than 6000, just in the inbox, excluding the filters. This was my first time to have zero-unread-email, since the emails were adding up since late 2008, in the scales of 1000s. Never marked the emails read in bulk before. Gmail took a bit of time to make this transaction, and Gmail was unavailable for a minute or two. Having a cleaner looking Gmail now. :D You may read this post to learn how to mark all your unread emails in your Gmail account "read" in one shot.

Building Open Daylight SDN Controller from Source on Ubuntu 14.04

As of the early research efforts on xSDN, an (extensible/extended/extensive/..) Software-Defined Networking (SDN) solution, I have been revising the literature on SDN while building and testing the related core technologies. 

Open Daylight is an open source consortium of projects lead by the Linux Foundation. Recently, Linux Foundation also announced OPNFV, an open platform to accelerate NFV (Network Functions Virtualization). These technologies are open and are considered to have the potential to change the future of networking.

There is a list of projects under Open Daylight. You may check out some or all of these projects. To build the projects from the source code (trunk), first you should check out the code, and install the controller. Each of the components can then be installed following the respective README files inside the respective modules. 

The build was successful with Oracle jdk1.7.0_55 and Apache Maven 3.1.1, with the tests

$ mvn clean install

Challenges

The controller build failed, with tests for the java version, Oracle jdk1.8.0. The error logs are given below.

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 9.895s

[INFO] Finished at: Tue Oct 07 14:24:21 WEST 2014

[INFO] Final Memory: 136M/362M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test (default-test) on project sal: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test failed: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?

[ERROR] Command was/bin/sh -c cd /home/pradeeban/OpenDayLight/controller/opendaylight/sal/api && /home/pradeeban/programs/jdk1.8.0/jre/bin/java -Xmx1024m -XX:MaxPermSize=256m -javaagent:/home/pradeeban/.m2/repository/org/jacoco/org.jacoco.agent/0.6.2.201302030002/org.jacoco.agent-0.6.2.201302030002-runtime.jar=destfile=/home/pradeeban/OpenDayLight/controller/opendaylight/sal/api/target/code-coverage/jacoco.exec -jar /home/pradeeban/OpenDayLight/controller/opendaylight/sal/api/target/surefire/surefirebooter4105410834640395132.jar /home/pradeeban/OpenDayLight/controller/opendaylight/sal/api/target/surefire/surefire2666028028053803617tmp /home/pradeeban/OpenDayLight/controller/opendaylight/sal/api/target/surefire/surefire_02245232938517059667tmp

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn -rf :sal

Also, the code still failed even after skipping the tests (mvn clean install -DskipTests). The error logs are given below.

 

[ERROR] Failed to execute goal org.codehaus.enunciate:maven-enunciate-plugin:1.28:docs (default) on project connectionmanager.northbound: Execution default of goal org.codehaus.enunciate:maven-enunciate-plugin:1.28:docs failed: A required class was missing while executing org.codehaus.enunciate:maven-enunciate-plugin:1.28:docs: com/sun/mirror/apt/AnnotationProcessorFactory
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>org.codehaus.enunciate:maven-enunciate-plugin:1.28
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
[ERROR] urls[0] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/maven-enunciate-plugin/1.28/maven-enunciate-plugin-1.28.jar
[ERROR] urls[1] = file:/home/pradeeban/OpenDayLight/controller/opendaylight/sal/api/target/sal-0.9.0-SNAPSHOT.jar
[ERROR] urls[2] = file:/home/pradeeban/.m2/repository/org/apache/commons/commons-lang3/3.1/commons-lang3-3.1.jar
[ERROR] urls[3] = file:/home/pradeeban/.m2/repository/org/apache/felix/org.apache.felix.dependencymanager/3.1.0/org.apache.felix.dependencymanager-3.1.0.jar
[ERROR] urls[4] = file:/home/pradeeban/.m2/repository/org/osgi/org.osgi.compendium/4.2.0/org.osgi.compendium-4.2.0.jar
[ERROR] urls[5] = file:/home/pradeeban/.m2/repository/org/osgi/org.osgi.core/5.0.0/org.osgi.core-5.0.0.jar
[ERROR] urls[6] = file:/home/pradeeban/.m2/repository/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.jar
[ERROR] urls[7] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-swagger/1.28/enunciate-swagger-1.28.jar
[ERROR] urls[8] = file:/home/pradeeban/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.3/jackson-mapper-asl-1.9.3.jar
[ERROR] urls[9] = file:/home/pradeeban/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.3/jackson-core-asl-1.9.3.jar
[ERROR] urls[10] = file:/home/pradeeban/OpenDayLight/controller/opendaylight/commons/logback_settings/target/commons.logback_settings-0.1.0-SNAPSHOT.jar
[ERROR] urls[11] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/maven-enunciate-slim-plugin/1.28/maven-enunciate-slim-plugin-1.28.jar
[ERROR] urls[12] = file:/home/pradeeban/.m2/repository/org/apache/maven/plugins/maven-install-plugin/2.3.1/maven-install-plugin-2.3.1.jar
[ERROR] urls[13] = file:/home/pradeeban/.m2/repository/org/codehaus/plexus/plexus-digest/1.0/plexus-digest-1.0.jar
[ERROR] urls[14] = file:/home/pradeeban/.m2/repository/org/apache/maven/plugins/maven-deploy-plugin/2.7/maven-deploy-plugin-2.7.jar
[ERROR] urls[15] = file:/home/pradeeban/.m2/repository/org/apache/maven/enforcer/enforcer-api/1.0/enforcer-api-1.0.jar
[ERROR] urls[16] = file:/home/pradeeban/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
[ERROR] urls[17] = file:/home/pradeeban/.m2/repository/org/apache/maven/shared/maven-filtering/1.0/maven-filtering-1.0.jar
[ERROR] urls[18] = file:/home/pradeeban/.m2/repository/org/apache/maven/reporting/maven-reporting-api/2.0.6/maven-reporting-api-2.0.6.jar
[ERROR] urls[19] = file:/home/pradeeban/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.0-alpha-7/doxia-sink-api-1.0-alpha-7.jar
[ERROR] urls[20] = file:/home/pradeeban/.m2/repository/commons-cli/commons-cli/1.0/commons-cli-1.0.jar
[ERROR] urls[21] = file:/home/pradeeban/.m2/repository/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-4/plexus-interactivity-api-1.0-alpha-4.jar
[ERROR] urls[22] = file:/home/pradeeban/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.12/plexus-interpolation-1.12.jar
[ERROR] urls[23] = file:/home/pradeeban/.m2/repository/org/sonatype/plexus/plexus-build-api/0.0.4/plexus-build-api-0.0.4.jar
[ERROR] urls[24] = file:/home/pradeeban/.m2/repository/org/codehaus/plexus/plexus-utils/3.0/plexus-utils-3.0.jar
[ERROR] urls[25] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-core/1.28/enunciate-core-1.28.jar
[ERROR] urls[26] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-core-rt/1.28/enunciate-core-rt-1.28.jar
[ERROR] urls[27] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-core-annotations/1.28/enunciate-core-annotations-1.28.jar
[ERROR] urls[28] = file:/home/pradeeban/.m2/repository/javax/annotation/jsr250-api/1.0/jsr250-api-1.0.jar
[ERROR] urls[29] = file:/home/pradeeban/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar
[ERROR] urls[30] = file:/home/pradeeban/.m2/repository/javax/servlet/servlet-api/2.3/servlet-api-2.3.jar
[ERROR] urls[31] = file:/home/pradeeban/.m2/repository/org/freemarker/freemarker/2.3.19/freemarker-2.3.19.jar
[ERROR] urls[32] = file:/home/pradeeban/.m2/repository/net/sf/apt-jelly/apt-jelly-freemarker/2.15/apt-jelly-freemarker-2.15.jar
[ERROR] urls[33] = file:/home/pradeeban/.m2/repository/net/sf/apt-jelly/apt-jelly-core/2.15/apt-jelly-core-2.15.jar
[ERROR] urls[34] = file:/home/pradeeban/.m2/repository/commons-digester/commons-digester/2.1/commons-digester-2.1.jar
[ERROR] urls[35] = file:/home/pradeeban/.m2/repository/commons-beanutils/commons-beanutils/1.8.3/commons-beanutils-1.8.3.jar
[ERROR] urls[36] = file:/home/pradeeban/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
[ERROR] urls[37] = file:/home/pradeeban/programs/jdk1.8.0/jre/../lib/tools.jar
[ERROR] urls[38] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-top/1.28/enunciate-top-1.28.jar
[ERROR] urls[39] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-jaxws-support/1.28/enunciate-jaxws-support-1.28.jar
[ERROR] urls[40] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-jaxws-ri/1.28/enunciate-jaxws-ri-1.28.jar
[ERROR] urls[41] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-docs/1.28/enunciate-docs-1.28.jar
[ERROR] urls[42] = file:/home/pradeeban/.m2/repository/org/jdom/jdom/1.1.2/jdom-1.1.2.jar
[ERROR] urls[43] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-jersey/1.28/enunciate-jersey-1.28.jar
[ERROR] urls[44] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-jersey-rt/1.28/enunciate-jersey-rt-1.28.jar
[ERROR] urls[45] = file:/home/pradeeban/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.9.3/jackson-jaxrs-1.9.3.jar
[ERROR] urls[46] = file:/home/pradeeban/.m2/repository/org/codehaus/jackson/jackson-xc/1.9.3/jackson-xc-1.9.3.jar
[ERROR] urls[47] = file:/home/pradeeban/.m2/repository/com/sun/jersey/jersey-server/1.11/jersey-server-1.11.jar
[ERROR] urls[48] = file:/home/pradeeban/.m2/repository/asm/asm/3.1/asm-3.1.jar
[ERROR] urls[49] = file:/home/pradeeban/.m2/repository/com/sun/jersey/jersey-core/1.11/jersey-core-1.11.jar
[ERROR] urls[50] = file:/home/pradeeban/.m2/repository/com/sun/jersey/jersey-servlet/1.11/jersey-servlet-1.11.jar
[ERROR] urls[51] = file:/home/pradeeban/.m2/repository/javax/mail/mail/1.4.4/mail-1.4.4.jar
[ERROR] urls[52] = file:/home/pradeeban/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar
[ERROR] urls[53] = file:/home/pradeeban/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3/jaxb-impl-2.2.3.jar
[ERROR] urls[54] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-xml/1.28/enunciate-xml-1.28.jar
[ERROR] urls[55] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-java-client/1.28/enunciate-java-client-1.28.jar
[ERROR] urls[56] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-c/1.28/enunciate-c-1.28.jar
[ERROR] urls[57] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-obj-c/1.28/enunciate-obj-c-1.28.jar
[ERROR] urls[58] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-csharp/1.28/enunciate-csharp-1.28.jar
[ERROR] urls[59] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-ruby/1.28/enunciate-ruby-1.28.jar
[ERROR] urls[60] = file:/home/pradeeban/.m2/repository/org/codehaus/enunciate/enunciate-php/1.28/enunciate-php-1.28.jar
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import  from realm ClassRealm[project>org.opendaylight.controller:forwarding.staticrouting:0.6.0-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]]
[ERROR]
[ERROR] -----------------------------------------------------: com.sun.mirror.apt.AnnotationProcessorFactory
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn -rf :connectionmanager.northbound

I changed to version 1.7.x. The build started to progress further, and eventually it hanged due to the memory limitation. Ctrl + C produced the below error.
^C^CJava HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGINT to handler- the VM may need to be forcibly terminated
I had to kill the process manually, using kill -9 .

After increasing the memory allocation to maven, the build succeeded, even with the tests!
MAVEN_OPTS="-Xmx2048m -XX:MaxPermSize=1024m"