yoursunny blog
computer, Internet, and life

Deep Atlantic Storage: Streaming Bits

Junxiao Shi

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. How does it work, and how can I afford to provide it?

This article is the third of a 3-part series that reveals the secrets behind Deep Atlantic Storage. The first part revealed that the uploaded file is sorted which drastically reduces its storage demand, and introduced the bit sorting algorithm. The second part covered how to process the upload in the browser using Web Workers. Now I'd continue from there, and explain where I store the files and how I offer downloads with reasonable costs.

Storage in the URL

Deep Atlantic Storage sorts the bits in every uploaded file. After sorting, each file can be represented by two numbers: the number of 0 bits, the number of 1 bits. Given these two numbers, the sorted file can be reconstructed.

I could make a database or use one of those fancy NoSQL thingy to store those numbers that represent the files, but I prefer my websites stateless so that I don't need to take backups. Therefore, I decided to encode those numbers in the URI.

(read more)

Deep Atlantic Storage: Sorting Bits

Junxiao Shi

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. Since Chia currency farming became popular in May, hard drive prices went up significantly. How can I afford to operate an unlimited free storage service?

"Advanced Sorting Technology"

One of the benefits listed on Deep Atlantic Storage webpage is:

  • Advanced sorting technology keeps your data neatly ordered.

What this means is that, content in the uploaded file would be sorted before being stored.

(read more)

IPv6 Neighbor Discovery Responder for KVM VPS

Junxiao Shi

I Want IPv6 for Docker

I'm playing with Docker these days, and I want IPv6 in my Docker containers. The best guide for enabling IPv6 in Docker is how to enable IPv6 for Docker containers on Ubuntu 18.04. The first method in that article assigns private IPv6 addresses to containers, and uses IPv6 NAT similar to how Docker handles IPv4 NAT. I quickly got it working, but I noticed an undesirable behavior: Network Address Translation (NAT) changes the source port number of outgoing UDP datagrams, even if there's a port forwarding rule for inbound traffic; consequently, a UDP flow with the same source and destination ports is being recognized as two separate flows.

$ docker exec nfd nfdc face show 262
    faceid=262
    remote=udp6://[2001:db8:f440:2:eb26:f0a9:4dc3:1]:6363
     local=udp6://[fd00:2001:db8:4d55:0:242:ac11:4]:6363
congestion={base-marking-interval=100ms default-threshold=65536B}
       mtu=1337
  counters={in={25i 4603d 2n 1179907B} out={11921i 14d 0n 1506905B}}
     flags={non-local permanent point-to-point congestion-marking}
$ docker exec nfd nfdc face show 270
    faceid=270
    remote=udp6://[2001:db8:f440:2:eb26:f0a9:4dc3:1]:1024
     local=udp6://[fd00:2001:db8:4d55:0:242:ac11:4]:6363
   expires=0s
congestion={base-marking-interval=100ms default-threshold=65536B}
       mtu=1337
  counters={in={11880i 0d 0n 1498032B} out={0i 4594d 0n 1175786B}}
     flags={non-local on-demand point-to-point congestion-marking}

The second method in that article allows every container to have a public IPv6 address. It avoids NAT and the problems that come with it, but requires the host to have a routed IPv6 subnet. However, routed IPv6 is hard to come by on KVM servers, because virtualization platform such as Virtualizor does not support routed IPv6 subnets, but can only provide on-link IPv6.

On-Link IPv6 vs Routed IPv6

(read more)

Ranked 10000 in Code Jam 2021 Qualification, Perl + Scala + Dart

Junxiao Shi

I spent the Saturday competing in Google Code Jam 2021 qualification round, and earned a well-rounded rank: exactly 10000.

yoursunny rank 10000

Code Jam is about Algorithms

My first time competing in Code Jam was 2012. I heard about Code Jam earlier that year, from a flyer posted by the ACM student chapter at University of Arizona. Since then, I participated in Code Jam every year, and 2021 is my 10th year competing in Code Jam.

Code Jam isn't about code or jam, but about mathematics and algorithms. Although I'm a good network programmer and write a lot of production quality code, I struggle at algorithms especially when I'm under a time pressure. Nevertheless, I continued competing in this annual event, so that I can remember the last bits of algorithms, and occasionally learn a few new tricks by reading through the post-round analyses.

(read more)