How I Setup my NFD Development Machine

I'm the lead developer of NDN Forwarding Daemon (NFD). In this article, I want to share how my development machine is setup.

Everything in Virtual Machines

I do all NFD development work in virtual machines. There are many benefits in using VMs: I can have a clean operating system, I can test out different OS versions if necessary, and I can work on a different change on another VM when "my code's compiling".

My VM was setup using Vagrant, using the following Vagrantfile:

$vmname = "devbox"
$sshhostport = 2222

$deps = <<SCRIPT
apt-get update
apt-get dist-upgrade -y -qq
apt-get install -y -qq build-essential doxygen gdb git graphviz libboost-all-dev libcrypto++-dev libpcap-dev libsqlite3-dev libssl-dev pkg-config python-pip python-sphinx valgrind
pip install sphinxcontrib-doxylink sphinxcontrib-googleanalytics
SCRIPT

Vagrant.configure(2) do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.network :forwarded_port, guest: 22, host: $sshhostport, id: "ssh"
  config.vm.provider "virtualbox" do |vb|
    vb.name = $vmname
    vb.memory = 4096
    vb.cpus = 4
  end
  config.vm.provision "deps", type: "shell", inline: $deps
  config.vm.provision "hostname", type: "shell", inline: "echo " + $vmname + " > /etc/hostname; hostname " + $vmname
  config.vm.provision "sshpvtkey", type: "file", source: "~/.ssh/id_rsa", destination: ".ssh/id_rsa"
  config.vm.provision "sshpubkey", type: "file", source: "~/.ssh/id_rsa.pub", destination: ".ssh/id_rsa.pub"
  config.vm.provision "sshauth", type: "shell", inline: "cd .ssh; cat id_rsa.pub >> authorized_keys"
  config.vm.provision "gitconfig", type: "file", source: "~/.gitconfig", destination: ".gitconfig"
end

May 2018 update: NFD's minimum supported platform is now Ubuntu 16.04. Here's the new Vagrantfile for NFD development in Ubuntu 16.04.

I chose Ubuntu 14.04 64-bit as the base image. Ubuntu 14.04 is the oldest OS that NFD must support. Developing in this OS allows me to avoid accidental use of compiler and library features that are only present in newer systems. A drawback of using Ubuntu 14.04 is that AddressSanitizer and UndefinedBehaviorSanitizer are unavailable in GCC. Therefore, I also have a Ubuntu 16.04 box for those.

Provisioners in this Vagrantfile install NFD's build dependencies, and uploads my SSH key pair to the VM, so that I can jump in development right away. The repositories are then cloned off Gerrit into the home directory. My actual Vagrantfile clones the repositories in a provisioning step, but it is not included here because clone commands differ per account.

./waf configure

NFD and related projects use Waf the meta build system. Build options are passed to ./waf configure command line, and then a simple ./waf command builds the project. The default ./waf configure without options prepares a release build without test cases and without full debugging support, but a developer needs more than that. The command line gets considerably longer after adding all the options.

To avoid typing a long ./waf configure command line every time, I have a configure.sh script in the home directory:

PROJ=$(basename $PWD)
if [[ $PROJ == ndn-cxx ]]; then
  ./waf configure --debug
elif [[ $PROJ == ndn-cxx-dev ]]; then
  ./waf configure --enable-static --disable-shared --with-tests --debug --without-pch
elif [[ $PROJ == ndn-cxx-san ]]; then
  ./waf configure --enable-static --disable-shared --with-tests --without-pch --with-sanitizer=address,undefined
elif [[ $PROJ == NFD ]]; then
  ./waf configure --with-tests --debug --without-pch
elif [[ $PROJ == ndn-tools ]]; then
  ./waf configure --with-tests --debug
elif [[ $PROJ == repo-ng ]]; then
  ./waf configure --with-tests --debug
else
  echo unknown project
  exit 1
fi

This allowed me to simply run ../configure.sh from within the repository, and the script executes the appropriate ./waf configure line.

You may have noticed that "ndn-cxx" appeared three times in the above script. I cloned ndn-cxx into three directories: ndn-cxx clone is sync'ed with the master branch and installed system-wide, which serves as a basis of NFD and ndn-tools development. ndn-cxx-dev clone is used for making changes to ndn-cxx itself. ndn-cxx-san clone (only on Ubuntu 16.04) has AddressSanitizer and UndefinedBehaviorSanitizer enabled and is used for fixing memory-related bugs in ndn-cxx. The latter two clones are not installed system-wide, and they are built as static libraries so that the unit tests would not look for the installed libndn-cxx.so.

Repository Branches

Within a repository, I keep the master branch in sync with origin/master, and create a local branch for each change. Local branches are named after the Redmine issue number. In case the same Redmine issue requires more than one commit, I also add a suffix to the branch name telling myself the primary change of each commit. To keep track of all the local branches, I have a little branch-list.sh script in the home directory to display the commit title of each local branch:

for B in $(git branch | sed 's/[*]//g'); do echo $B $(git log $B 2>/dev/null | head -5 | tail -1); done

Its output looks like:

vagrant@m0212:~/NFD$ ../branch-list.sh 
4191 tests: avoid misaligned memory access in CS test
4194 docs: corrections in Getting Started page
master tests: use ndn::util::Sha256 instead of ndn::crypto

Unless there is a dependency between two commits, I start every change from the master branch. After a change has been merged into master, I rebase the local branch onto master. The key to not mess up rebasing is to use git cherry-pick instead of git rebase. I have a script in the home directory for that, cherry-pick.sh:

BRANCH=$1
SHA1=$(git log $BRANCH | head -1 | awk '{print $2}')
git branch -D $BRANCH
git checkout -b $BRANCH
git cherry-pick $SHA1

After a commit was merged on Gerrit, I run the script from master branch:

vagrant@m0212:~/NFD$ git checkout master
Switched to branch 'master'
vagrant@m0212:~/NFD$ git pull
Updating fc2e13d..bb6146e
Fast-forward
 10 files changed, 9 insertions(+), 340 deletions(-)
vagrant@m0212:~/NFD$ ../cherry-pick.sh 4194
Deleted branch 4194 (was e3b245c).
Switched to a new branch '4194'
[4194 4da9e4f] docs: corrections in Getting Started page
 1 file changed, 20 insertions(+), 13 deletions(-)

In case there is a code conflict, I would have to manually fix the conflict, and run git add . followed by git cherry-pick --continue. Since I'm dealing with one commit at a time, this wouldn't be too much trouble.

License Boilerplate

The policy at NFD is to update the license boilerplate only if a file is being changed. The official license boilerplate changes once a year to reflect the new copyright end year, and code review process would reject a change if license boilerplate is outdated. I also have a script for that, update-license.sh.

It can be invoked as ../update-license.sh before committing code, or as ../update-license.sh HEAD^1 after committing code (then a git commit -a --amend is needed). It only works with NFD and ndn-cxx repositories, and cannot handle files with a different set of copyright holders (which needs to be reverted manually).

Uploading to Gerrit

The standard command of uploading a change to Gerrit is:

git push origin HEAD:refs/for/master

This is too long to type every time, isn't it?

The shortcut is to modify .git/config within each repository, and add two sections like this: (the url value should be copied from [remote "origin"] section)

[remote "gerrit"]
	url = ssh://sunny@gerrit.named-data.net:29418/NFD
	push = HEAD:refs/for/master
[remote "draft"]
	url = ssh://sunny@gerrit.named-data.net:29418/NFD
	push = HEAD:refs/drafts/master

With these in place, I can upload a change for review with:

git push gerrit

Or I can upload a change as a draft with:

git push draft

Gerrit's draft feature makes code invisible to others, and does not trigger a build on Jenkins. It is particularly useful for code backups, and for viewing a change on the nice interface of Gerrit without bothering reviewers.

Conclusion

This article reveals how I setup my NFD development box as a virtual machine, and provides some bash scripts to help me work more efficiently in the VM. I hope these insights can help you reducing repetitive tasks during NFD development so you can focus on the code.