Today I Learned: openat()

fopen and open

In C programming language, the <stdio.h> header supplies functions for file input and output. To open a file, we usually use the fopen function. It is defined by the C language standard and works in every operating system.

Working at a lower level, there's also the open function. It is a system call provided by the Linux kernel and exposed through glibc.

Both fopen and open have an input parameter: the file pathname, as a NUL-terminated string. These two functions are declared like this:

FILE* fopen(const char* filename, const char* mode);

int open(const char* pathname, int flags);

Stardate 1302.8

Stardate 1302.8, San Francisco.

Just as I'm ready to shut the doors of yoursunny summer host sales office and spend the weekend with my snake collection, a customer rushed in. The customer shouted hurriedly: my storage server in Midwest Eurasia is involucrated, routing announcement is withdrawn, all my family memories are gone, can you help?

I opened my sleepy eyes, downed a bottle of Red Bull, and immediately started helping this customer. I sent a robot to the garden where her storage server was located, and provisioned a new storage server in our luna location. Within seconds, the robot found the original server and plugged into its USB port, and her precious data were being cloned to our infrastructure. At the same time, I was busy engraving a golden key card that contains a private key to decrypt the data.

15 minutes later, data restoration was complete, and I opened up the portal room. The customer swiped the card, a colorful cloud appeared on the portal. Inside, there's a faint wall of text:

GO
GO
GO
GO
GO

Deep Atlantic Storage: Streaming Bits

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. How does it work, and how can I afford to provide it?

This article is the third of a 3-part series that reveals the secrets behind Deep Atlantic Storage. The first part revealed that the uploaded file is sorted which drastically reduces its storage demand, and introduced the bit sorting algorithm. The second part covered how to process the upload in the browser using Web Workers. Now I'd continue from there, and explain where I store the files and how I offer downloads with reasonable costs.

Storage in the URL

Deep Atlantic Storage sorts the bits in every uploaded file. After sorting, each file can be represented by two numbers: the number of 0 bits, the number of 1 bits. Given these two numbers, the sorted file can be reconstructed.

I could make a database or use one of those fancy NoSQL thingy to store those numbers that represent the files, but I prefer my websites stateless so that I don't need to take backups. Therefore, I decided to encode those numbers in the URI.

在?吃过树上的知了么?

蝉,俗称知了,是一种昆虫。 每过17年,在美国东部地区,数十亿只【Brood X Cicadas】知了从地下钻出,开始交配繁殖。 只要有树木的地方,就可以听到知了大军日夜歌唱、没完没了,声强达到90分贝以上,令当地居民烦恼不已。

两只知了停在树枝上

知了虽吵,但对人畜无害,且具有丰富的蛋白质。 6月19日,阳光男孩前往附近小树林,捕捉知了,补充营养。 知了不会咬人,飞行速度很慢,只要被看见,就插翅难逃。 可是,这天已经接近【知了季】的尾声,所以阳光男孩在齐腰深的草丛里钻进钻出两个小时,也只捉到24只。

知了躲在草丛里

回来在水池里略作清洗,然后下油锅爆炒30秒,加少量盐调味。 透明的翅膀入口即化,金黄色的身体鲜嫩清爽,别有一番风味。

Deep Atlantic Storage: Reading File Upload in Web Workers

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. How does it work, and how can I afford to provide it?

This article is the second of a 3-part series that reveals the secrets behind Deep Atlantic Storage. The previous part introduced the algorithm I use to sort all the bits in a Uint8Array. Now I'd continue from there, and explain how the webpage accepts and processes file uploads.

File Upload

File upload has always been a part of HTML standard as long as I remembered:

<form action="upload.php" method="POST" enctype="multipart/form-data">
  <input type="file" name="file">
  <input type="submit" value="upload">
</form>

Deep Atlantic Storage: Sorting Bits

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. Since Chia currency farming became popular in May, hard drive prices went up significantly. How can I afford to operate an unlimited free storage service?

"Advanced Sorting Technology"

One of the benefits listed on Deep Atlantic Storage webpage is:

  • Advanced sorting technology keeps your data neatly ordered.

What this means is that, content in the uploaded file would be sorted before being stored.

Face and PacketHandler in NDNph

Face is an overloaded term in Named Data Networking (NDN). My last article explained what is a face in NDN forwarders and NDN libraries, and then described the endpoint design in my NDNts library. This time, I'll introduce a unique face API design in my NDNph library.

NDNph is a C++ header-only library that enables low level application development. It supports multiple platforms, but is primarily designed for microcontrollers with limited hardware resources. In particular, RAM capacity is very limited, with typical values ranging from 50KB (ESP8266) to 320KB (ESP32). This necessitates a different API design for the face.

Overhead of a Traditional Face

Traditionally, a face in NDN libraries has the following features:

  • send and receive NDN network layer packets
  • match incoming Data against outgoing Interests
  • keep track of Interest timeouts
  • dispatch incoming Interests to producer callback functions

Install Ubuntu from ISO on IPv6-only KVM Server in SolusIO

I recently obtained a KVM virtual server on SolusIO platform, and I want to install Ubuntu 20.04 Server from the official ISO image. This is not as easy as I hoped, but I figured it out.

Note: if you are in a hurry, skip the "Background" and start from "Part 1" section.

Background: SolusIO cannot Mount ISO Image

SolusIO is a virtual infrastructure management solution published by Plesk International Gmbh, the same company behind the popular SolusVM software. They describe SolusIO to be the successor of SolusVM, with more focus on the self-service approach for end users.

SolusIO inherits the same clean user interface from SolusVM, and is easy to use. However, as a power user, I notice several features are missing in SolusIO. One of these features is the ability to install the operating system from an ISO image.

Disabling VNC in Virtualizor ⇒ Lost Connectivity?

The KVM server hosting my website went offline last month. Thinking the server might have crashed, I went to Virtualizor, the VPS control panel, to reboot the VPS. It did not solve the problem, so I proceeded with my disaster recovery plan.

The hosting provider, Spartan Host, explained that it was a router bug. They fixed the router after 4 hours, but my server did not come online.

Symptom

To investigate what went wrong with my VPS, I came back to Virtualizor to enable VNC access. Having VNC access is like attaching a monitor and a keyboard to the server. It would allow me to see any error messages printed on the screen and login to check whether there are configuration errors.

I didn't see any error through VNC connection. Thinking it might be a routing problem, I logged in with username and password, and ran a traceroute. To my surprise, the traceroute was able to reach Internet destination. Moreover, I can SSH into this server again.