NetKarma

The NetKarma project has ended, but its tools are still maintained.

About

As computer network experiments increase in complexity and size, it becomes increasingly difficult to fully understand the circumstances under which the experiment was run, particularly when these results are shared for purposes of reproducibility. The provenance of an experiment is its lineage or historical trace that can capture experiment conditions, time ordering, and relationships within the experiment and across the experiment and infrastructure layer.

The Global Environment for Network Innovations (GENI) Provenance Registry (NetKarma) project, funded in October 2009, provides a tool for capturing the workflow of GENI slice creation, topology of the slice, operational status and other measurement statistics and correlate it with the experimental data.

What can NetKarma do for you?

The tool, NetKarma, will allow researchers to see the exact state of the network and store configuration of the experiment and its slice. The provenance of the data will be stored and visualized through a data portal. The provenance data can be used by the researcher to analyze their data, allow for the suspension and resumption of an experiment and provide a single reference to find the details and data collected in an experiment.

How it works

NetKarma is based on the Karma provenance architecture that has been used to collect scientific workflows in diverse domains such as meteorology and life science.

The NetKarma Provenance System and the NetKarma GMOC Adaptor are running as a persistent service on servers at Indiana University. This allows GENI users to process a GUSH log through the NetKarma Adaptor and visualize the resulting provenance graph, including the automated GMOC annotations, without having to install the NetKarma server. To try out the NetKarma provenance visualization, GENI users can process their own GUSH logs or the sample log file included with the NetKarma adaptor.

Projects & downloads

Overview

Released August 26, 2011, NetKarma version 2.5 builds on the NetKarma Adaptor and NetKarma visualization plug-ins for Cytoscape to provide more detailed provenance data from the GUSH logs of experiments run on GENI slices.

In addition to extracting more detailed provenance from GUSH logs, version 2.5 of the NetKarma plug-in includes richer annotations of the provenance visualized and is more configurable by the user, allowing greater flexibility in visualizing and comprehending the provenance of an experiment.

Version 2.5 also includes the NetKarma GMOC Adaptor, which automates the annotation of NetKarma provenance graphs with network data from GMOC.

New features

  • Enhancements to the NetKarma Adaptor used to process GUSH logs and ingest the provenance of GENI experiments run on GUSH into the NetKarma Provenance System.
  • Enhancements to the NetKarma Cytoscape plug-in for NetKarma that includes more detailed provenance and the addition of annotations to processes, artifacts, and the edges that connect the processes and artifacts in the OPM graphs generated using NetKarma.
  • This version of the visualization plug-in also has increased configurability by the user as to the details displayed when visualizing provenance graphs.
  • The new NetKarma GMOC Adaptor is being released which extracts additional metadata from the GMOC database and annotates the GUSH provenance graphs with data from the GMOC database. This Adaptor is configured to run as a background process and automatically annotates the provenance ingested into NetKarma without any additional steps required by GENI users.

Download

Additional versions

Karma Adaptor is one of the collection tools that make up the Karma provenance collection toolkit to harvest provenance from log files. It uses a rule file specific to an application to map raw data into Karma specific provenance events.

The provenance of the data is stored into a relational database which can be visualized through various plugins. The provenance data can be used by the researcher to analyze their data, allow for the suspension and resumption of an experiment and provide references to find the details and data collected in an experiment.

The most current version of NetKarma Adaptor includes enhancements in rule-files for processing gush command-line logs and gush experiment logs to include additional rules for capturing provenance. It also allows the user to select the appropriate rule-file automatically, based on the type of log-file to be processed.

Downloads

The Cytyoscape visualization plug-in version 1.2.1 includes new visualzation algorithms for the capture of provenance from experiments run on the NS2 network simulator. The NetKarma NS2 Adaptor provides the ability to capture provenance from NS2 trace files and has a Tcl extension to capture the network topology of your NS2 experiment. It also allows you to ingest provenance for NS2 experiments from these files and then visualize it from NetKarma using version 1.2.1 or later of the NetKarma visualization plug-ins. 

Downloads

Version 2.5 of NetKarma included version 1.1.0 of the NetKarma visualization plug-in for Cytoscape. This version of the plug-in includes upgrades to the visualization layout algorithms and new visualization styles. These were added as part of our efforts to capture provenance for additional GENI experiments using the NS2 simulator. The visualization plug-in manual includes further details, but the features added in this release of the plug-in include:

  • New layout algorithms that handle provenance graphs generated based on Network Simulator (NS2) experiments.
  • New visualization style for NS2 simulations.

Downloads

Contributors

  • Beth Plale, PI, main contact
  • Christopher Small, Co-PI, main contact
  • Scott Jensen, post-doctoral researcher
  • Mehmet Aktas, post-doctoral researcher
  • Peng Chen
  • Devarshi Ghoshal
  • Yuan Luo
  • Abhijeet Kodgire
  • David Ripley

Sponsors

Data to Insight Center
Indiana University Pervasive Technology Institute
2010-present

BBN Technologies: Global Environment for Network Innovations (GENI)
NetKarma
2009-July 2012