yoursunny blog
computer, Internet, and life

The Reality of NDN Video Streaming

Junxiao Shi

As of 2019, video accounts for over 60% of downstream traffic on the Internet. It is believed that video streaming could benefit from the in-network caching feature of Named Data Networking (NDN), which would reduce the total traffic volume and bring cost savings for Internet service providers and content publishers. Far Cry: Will CDNs Hear NDN's Call?, a paper published at ACM-ICN 2020 conference, is the latest attempt on NDN video streaming.

How iViSA Works

In Far Cry, the authors implemented iViSA, a browser-based video streaming application that runs on the global NDN testbed, and then performed some comparison study between this application and similar HTTP video streaming application deployed on commercial CDN services.

It's said that if you want reproducible science, the software needs to be open source. The authors released most of their source code, and we can get a peek into how iViSA actually works.

The backend server repository contains:

(read more)

NFD nightly went APT

Junxiao Shi

NOTICE Instructions in this article are outdated. Please see latest information in NFD nightly usage.

Last year, I started building NFD nightly packages in GitHub Actions. So far, installation is a manual procedure: the user must manually download the ZIP files from nfd-nightly.ndn.today, decompress them, and figure out the dependency among various .deb packages. Starting today, I'm publishing NFD nightly packages in an APT repository, and you can install them with apt-get command.

Add the NFD nightly APT repository

Add the repository with the command that matches your platform:

# Ubuntu 18.04 (bionic), amd64 (laptops and servers)
echo "deb [trusted=yes] https://nfd-nightly-apt.ndn.today/ubuntu bionic main" \
  | sudo tee /etc/apt/sources.list.d/nfd-nightly.list

# Ubuntu 20.04 (focal), amd64 (laptops and servers)
echo "deb [trusted=yes] https://nfd-nightly-apt.ndn.today/ubuntu focal main" \
  | sudo tee /etc/apt/sources.list.d/nfd-nightly.list

# Debian 10 (buster), amd64 (laptops and servers)
echo "deb [trusted=yes] https://nfd-nightly-apt.ndn.today/debian buster main" \
  | sudo tee /etc/apt/sources.list.d/nfd-nightly.list

# Debian 10 (buster), armv7 (Raspberry Pi 3 or 4)
echo "deb [trusted=yes] https://nfd-nightly-apt.ndn.today/debian buster main" \
  | sudo tee /etc/apt/sources.list.d/nfd-nightly.list

# Debian 10 (buster), armv6 (Raspberry Pi Zero W)
echo "deb [trusted=yes] https://nfd-nightly-apt.ndn.today/raspberrypi buster main" \
  | sudo tee /etc/apt/sources.list.d/nfd-nightly.list
(read more)

Self-Hosted NDNts Nightly Build

Junxiao Shi
last updated

NDNts nightly build is a set of NPM-compatible tarballs compiled automatically from the development branch of NDNts, Named Data Networking (NDN) libraries for the modern web, distributed on https://ndnts-nightly.ndn.today website. Users can install NDNts nightly build following these instructions.

However, this website only stores the latest version of NDNts packages. This has been causing installation conflicts when NPM tries to look for previous versions. Moreover, as I have declared, I don't care much about backwards compatibility. With NPM, all published versions are stored indefinitely, so you can continue using an older version without being affected by breaking changes. On the other hand, once a new nightly build is uploaded, the previous version is overwritten and no longer available for downloads. You are then forced to cope with the breaking changes I introduce from time to time, possibly at higher frequency than you would like to.

Today, I'm introducing two methods for self-hosting NDNts nightly build. Both methods allow you to build a specific version of NDNts codebase from a checkout of the NDNts monorepo, and generate a set of tarballs that you can host locally on a server under your control. Afterwards, you can install NDNts packages from this server, without relying on my website and without being affected by my breaking changes.

Self-Hosted NDNts on an HTTP Server

This section was latest updated on 2024-03-06 to reflect latest changes.

(read more)

NDN-DPDK: NDN Forwarding at 100 Gbps on Commodity Hardware

Junxiao Shi, Davide Pesavento, Lotfi Benmohamed

Presented at: 7th ACM Conference on Information-Centric Networking (ICN 2020)

Since the Named Data Networking (NDN) data plane requires name-based lookup of potentially large tables using variable-length hierarchical names as well as per-packet state updates, achieving high-speed NDN forwarding remains a challenge. In order to address this gap, we developed a high-performance NDN router capable of reaching forwarding rates higher than 100 Gbps while running on commodity hardware. In this paper we present our design and discuss its tradeoffs. We achieved this performance through several optimization techniques that include adopting better algorithms and efficient data structures, as well as making use of the parallelism offered by modern multi-core CPUs and multiple hardware queues with user-space drivers for kernel bypass. Our open-source forwarder is the first software implementation of NDN to exceed 100 Gbps throughput while supporting the full protocol semantics. We also present the results of extensive benchmarking carried out to assess a number of performance dimensions and to diagnose the current bottlenecks in the packet processing pipeline for future scalability enhancements. Finally, we identify future work which includes hardware-assisted ingress traffic dispatching, dynamic load balancing across forwarding threads, and novel caching solutions to accommodate on-disk content stores.

Read full paper at ACM Digital Library: NDN-DPDK: NDN Forwarding at 100 Gbps on Commodity Hardware

NDN-DPDK logo

(read more)

NDNts Demo at NDN Community Meeting 2020

Junxiao Shi

NDN Community Meeting is an annual event that brings together a large community of researchers from academia, industry, and government, as well as users and other parties interested in the development of Named Data Networking (NDN) technology. Having no peer review process, it is a prime opportunity to showcase my personal projects to the community. I demo'ed my ndn-js home surveillance camera at NDNcomm 2018. This time, I decide to demo my flagship product, NDNts: NDN Libraries for the Modern Web.

The Demo Video

NDNts is a set of libraries with many different features, where do I start? I decide to select a subset of unique features that are not found in any other library:

  • The Endpoint API that enhances face by automatically handling repetitive tasks such as Interest retransmissions and packet signing/verification, so that app developers can focus on the application logic.
  • An implementation of trust schemas.
  • NDN Certificate Management protocol implementation, including a graphical user interface for the certificate authority component.

I also threw in two web applications:

(read more)

Getting Started with NDNts Web Application using webpack

Junxiao Shi
last updated

This article shows how to get started with NDNts, Named Data Networking (NDN) libraries for the modern web. In particular, it demonstrates how to write a consumer-only web application that connects to the NDN testbed, transmits a few Interests, and gets responses. This application uses JavaScript programming language and webpack module bundler.

Code samples in this article were last updated on 2024-03-06 to reflect latest changes.

Prepare the System

To use NDNts, you must have Node.js. As of this writing, NDNts works best with Node.js 20.x, and you should install that version. The easiest way to install Node.js is through Node Version Manager (nvm) or Node Version Manager (nvm) for Windows.

On Ubuntu 22.04, you can install nvm and Node.js with the following commands:

(read more)

NDNts Nightly Build

Junxiao Shi
last updated

NDNts nightly build is a set of NPM-compatible tarballs compiled automatically from the development branch of NDNts, Named Data Networking (NDN) libraries for the modern web. They are built by the Continuous Integration (CI) system and uploaded to the NDNts nightly build website: ndnts-nightly.ndn.today. Homepage of that website displays a list of URIs of available tarballs.

This article was latest updated on 2024-03-06 to reflect latest changes.

How to install NDNts nightly build

You can find available tarballs on NDNts nightly build website: ndnts-nightly.ndn.today.

To install a tarball as a local dependency within the current project, you can execute something like:

(read more)

NFD nightly packages

Junxiao Shi

NOTICE Instructions in this article are outdated. Please see latest information in NFD nightly usage.

NDN Forwarding Daemon (NFD) is the reference implementation of Named Data Networking (NDN) forwarding plane. The software is continuously developed, but binary releases happen rather infrequently. Recently, I made a workflow to build NFD and related software automatically.

Download page: nfd-nightly.ndn.today

Instructions

Which platform should I choose?

(read more)

Getting Started with NDNts in Node.js

Junxiao Shi
last updated

This article shows how to get started with NDNts, Named Data Networking (NDN) libraries for the modern web. In particular, it demonstrates how to write a producer and a consumer application in Node.js using JavaScript programming language, and transmit a few Interest and Data packets via NFD forwarder on the local machine.

Code samples in this article were last updated on 2024-03-06 to reflect latest changes.

Prepare the System

This guide is written for Ubuntu 22.04 operating system. If you have a Windows PC, you can enable Windows Subsystem for Linux and install Ubuntu 22.04 from the Microsoft Store. If you have a Macbook or a Linux machine other than Ubuntu 22.04, you can install Vagrant, and create a virtual machine from bento/ubuntu-22.04 template. All steps below should be executed inside Ubuntu 22.04 environment.

To use NDNts, you must have Node.js. As of this writing, NDNts works best with Node.js 20.x, and you should install that version. The easiest way to install Node.js is through Node Version Manager (nvm). To install nvm and then install Node.js, type the following commands in Ubuntu 22.04 terminal:

(read more)

Introducing NDNph and New Version of esp8266ndn

Junxiao Shi

NDNph is my latest Named Data Networking (NDN) client library. This article gives an overview of this library.

History and Motivation

In 2016, I started esp8266ndn. It contains a copy of ndn-cpp-lite, UCLA REMAP's C++ library that does not use dynamic memory allocations. I then added integrations with ESP8266's network stack and crypto functions, making esp8266ndn the first NDN library that works on the ESP8266 microcontroller. Using this library, I built several projects, including a wearable jewelry and a call button. University of Memphis also deployed several sensor nodes using esp8266ndn.

Over the years, esp8266ndn gained many new features, and widened platform support to include ESP32 and nRF52. However, using ndn-cpp-lite as the core is becoming problematic:

  • New protocol features show up slowly, because ndn-cpp-lite author would not add a feature until ndn-cxx has it, and design discussions for ndn-cxx sometimes take several years.
  • There is no generic TLV encoder/decoder, making it difficult to support TLV structures in application layer.
  • ndn-cpp-lite is bloated with obsolete features, due to their backwards compatibility guarantees. Consequently, binary code size is unnecessarily large.
  • Although I can add patches to ndn-cpp-lite during importing into esp8266ndn, it has been difficult to test these patches. This isn't ndn-cpp-lite's fault, but is still an issue.
(read more)