yoursunny blog
computer, Internet, and life

Deep Atlantic Storage: Sorting Bits

Junxiao Shi

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. Since Chia currency farming became popular in May, hard drive prices went up significantly. How can I afford to operate an unlimited free storage service?

"Advanced Sorting Technology"

One of the benefits listed on Deep Atlantic Storage webpage is:

  • Advanced sorting technology keeps your data neatly ordered.

What this means is that, content in the uploaded file would be sorted before being stored.

(read more)

Face and PacketHandler in NDNph

Junxiao Shi
last updated

Face is an overloaded term in Named Data Networking (NDN). My last article explained what is a face in NDN forwarders and NDN libraries, and then described the endpoint design in my NDNts library. This time, I'll introduce a unique face API design in my NDNph library.

NDNph is a C++ header-only library that enables low level application development. It supports multiple platforms, but is primarily designed for microcontrollers with limited hardware resources. In particular, RAM capacity is very limited, with typical values ranging from 50KB (ESP8266) to 320KB (ESP32). This necessitates a different API design for the face.

Overhead of a Traditional Face

Traditionally, a face in NDN libraries has the following features:

  • send and receive NDN network layer packets
  • match incoming Data against outgoing Interests
  • keep track of Interest timeouts
  • dispatch incoming Interests to producer callback functions
(read more)

Install Ubuntu from ISO on IPv6-only KVM Server in SolusIO

Junxiao Shi

I recently obtained a KVM virtual server on SolusIO platform, and I want to install Ubuntu 20.04 Server from the official ISO image. This is not as easy as I hoped, but I figured it out.

Note: if you are in a hurry, skip the "Background" and start from "Part 1" section.

Background: SolusIO cannot Mount ISO Image

SolusIO is a virtual infrastructure management solution published by Plesk International Gmbh, the same company behind the popular SolusVM software. They describe SolusIO to be the successor of SolusVM, with more focus on the self-service approach for end users.

SolusIO inherits the same clean user interface from SolusVM, and is easy to use. However, as a power user, I notice several features are missing in SolusIO. One of these features is the ability to install the operating system from an ISO image.

(read more)

Disabling VNC in Virtualizor ⇒ Lost Connectivity?

Junxiao Shi

The KVM server hosting my website went offline last month. Thinking the server might have crashed, I went to Virtualizor, the VPS control panel, to reboot the VPS. It did not solve the problem, so I proceeded with my disaster recovery plan.

The hosting provider, Spartan Host, explained that it was a router bug. They fixed the router after 4 hours, but my server did not come online.

Symptom

To investigate what went wrong with my VPS, I came back to Virtualizor to enable VNC access. Having VNC access is like attaching a monitor and a keyboard to the server. It would allow me to see any error messages printed on the screen and login to check whether there are configuration errors.

I didn't see any error through VNC connection. Thinking it might be a routing problem, I logged in with username and password, and ran a traceroute. To my surprise, the traceroute was able to reach Internet destination. Moreover, I can SSH into this server again.

(read more)

IPv6 Neighbor Discovery Responder for KVM VPS

Junxiao Shi

I Want IPv6 for Docker

I'm playing with Docker these days, and I want IPv6 in my Docker containers. The best guide for enabling IPv6 in Docker is how to enable IPv6 for Docker containers on Ubuntu 18.04. The first method in that article assigns private IPv6 addresses to containers, and uses IPv6 NAT similar to how Docker handles IPv4 NAT. I quickly got it working, but I noticed an undesirable behavior: Network Address Translation (NAT) changes the source port number of outgoing UDP datagrams, even if there's a port forwarding rule for inbound traffic; consequently, a UDP flow with the same source and destination ports is being recognized as two separate flows.

$ docker exec nfd nfdc face show 262
    faceid=262
    remote=udp6://[2001:db8:f440:2:eb26:f0a9:4dc3:1]:6363
     local=udp6://[fd00:2001:db8:4d55:0:242:ac11:4]:6363
congestion={base-marking-interval=100ms default-threshold=65536B}
       mtu=1337
  counters={in={25i 4603d 2n 1179907B} out={11921i 14d 0n 1506905B}}
     flags={non-local permanent point-to-point congestion-marking}
$ docker exec nfd nfdc face show 270
    faceid=270
    remote=udp6://[2001:db8:f440:2:eb26:f0a9:4dc3:1]:1024
     local=udp6://[fd00:2001:db8:4d55:0:242:ac11:4]:6363
   expires=0s
congestion={base-marking-interval=100ms default-threshold=65536B}
       mtu=1337
  counters={in={11880i 0d 0n 1498032B} out={0i 4594d 0n 1175786B}}
     flags={non-local on-demand point-to-point congestion-marking}

The second method in that article allows every container to have a public IPv6 address. It avoids NAT and the problems that come with it, but requires the host to have a routed IPv6 subnet. However, routed IPv6 is hard to come by on KVM servers, because virtualization platform such as Virtualizor does not support routed IPv6 subnets, but can only provide on-link IPv6.

On-Link IPv6 vs Routed IPv6

(read more)

yoursunny.com Disaster Recovery Plan: 104 Minutes Downtime, No Tears

Junxiao Shi

The OVH fire taught us the importance of having a disaster recovery plan for your website and online services. In 2017, I rebuilt yoursunny.com and moved everything from configuration to content into git repositories. One of the reasons was that, the git repository could serve as a backup of the website, so that I can recover the site from a data loss.

uptime last 24 hours, vps4 server, yoursunny.com website

Today, I was forced to execute (part of) my disaster recovery plan. The result was: website is successfully recovered after 1 hour and 44 minutes of downtime.

🟥 Down

When I waked up this morning, there were several alert emails from UptimeRobot telling me that my website was down, up, down, and up again. At the same time, I also received alerts that the VPS hosting the website was not responding to ping. I ignored those alerts, thinking that they would resolve itself in a few minutes.

(read more)

Ranked 10000 in Code Jam 2021 Qualification, Perl + Scala + Dart

Junxiao Shi

I spent the Saturday competing in Google Code Jam 2021 qualification round, and earned a well-rounded rank: exactly 10000.

yoursunny rank 10000

Code Jam is about Algorithms

My first time competing in Code Jam was 2012. I heard about Code Jam earlier that year, from a flyer posted by the ACM student chapter at University of Arizona. Since then, I participated in Code Jam every year, and 2021 is my 10th year competing in Code Jam.

Code Jam isn't about code or jam, but about mathematics and algorithms. Although I'm a good network programmer and write a lot of production quality code, I struggle at algorithms especially when I'm under a time pressure. Nevertheless, I continued competing in this annual event, so that I can remember the last bits of algorithms, and occasionally learn a few new tricks by reading through the post-round analyses.

(read more)

NDN Video Streaming over QUIC

Junxiao Shi

NDN over WebSockets == TCP over TCP

Named Data Networking (NDN) was first ported to web browser environment in 2012. At that time, a browser-based JavaScript application can communicate with the Internet via either XMLHTTPRequest or WebSocket. Feeling that WebSocket is a better match for the NDN implementation at the time, I wrote the initial code for a CCNx WebSocket proxy.

Web applications would connect to this proxy over TCP, negotiate a WebSocket connection, and send NDN packets in WebSocket frames. The proxy then decapsulates these frames, and delivers the NDN packets to ccnd forwarder over TCP.

NDN-over-WebSockets survived multiple protocol changes over the years, and made its way into the NDN Forwarding Daemon (NFD). It worked fine for simple NDN web applications, such as status pages and text chat, and even file retrievals.

Recently, with the rise of video streaming on the NDN testbed, congestion control functionality starts to show up in NDN libraries. Then, a question popped into my mind: WebSockets run over TCP, and NDN congestion control algorithms are largely borrowed from TCP, would this cause any problems?

(read more)

OVH Strasbourg: Halt and Catch Fire, Data Uploaded to the Cloud

Junxiao Shi

2021-03-10, an OVH Cloud data center in Strasbourg, France caught on fire. Thousands of servers have been destroyed by fire. Thousands more are currently unavailable due to power cut, and will remain offline for several more days.

Data stored in those servers have been uploaded to the cloud via black smoke.

a building is burning with black smoke rising to the sky

According to Hacker News, a Dev mistakenly invoked the Halt and Catch Fire instruction on an Uninterrupted Power Supply unit, causing this incident.

Chairman of OVH cloud advised clients to activate their disaster recovery plans, such as restoring off-site backups to a new cloud server. Some clients are crying because they did not have backups, or they stored their backups on another machine in the same data center. Other clients experienced no downtime because they designed their systems for datacenter scale redundancy.

(read more)

漫步在北京 (下) | Wandering in Beijing (2)

Junxiao Shi

北京历史悠久，景点众多，一篇文章写不下。 所以，接着上篇，我来继续回顾我在北京看到的风景名胜。

Beijing has a long history and more attractions than what I can fit in one article. Following the previous part, let me continue revisiting what I've experienced in Beijing.

北京的长城 | Great Wall

万里长城是中国古代建造的最为宏大的工程之一，也是中华民族精神的象征。 长城在春秋战国时期即已修筑，不过北京附近的长城大多数是明代建造的。 八达岭是北京最著名、交通比较方便的一段长城。 我在2013年6月16日游览了八达岭长城。

Great Wall of China is one of the most magnificent projects in ancient China. It has existed since more than 2500 years ago, but most remaining Great Wall sections near Beijing were built in the Ming Dynasty. Badaling is the most visited section of Great Wall because it's conveniently accessible by public transit. I visited Badaling on 16 June 2013.

(read more)