Deep Atlantic Storage: Streaming Bits

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. How does it work, and how can I afford to provide it?

This article is the third of a 3-part series that reveals the secrets behind Deep Atlantic Storage. The first part revealed that the uploaded file is sorted which drastically reduces its storage demand, and introduced the bit sorting algorithm. The second part covered how to process the upload in the browser using Web Workers. Now I'd continue from there, and explain where I store the files and how I offer downloads with reasonable costs.

Storage in the URL

Deep Atlantic Storage sorts the bits in every uploaded file. After sorting, each file can be represented by two numbers: the number of 0 bits, the number of 1 bits. Given these two numbers, the sorted file can be reconstructed.

I could make a database or use one of those fancy NoSQL thingy to store those numbers that represent the files, but I prefer my websites stateless so that I don't need to take backups. Therefore, I decided to encode those numbers in the URI.

Deep Atlantic Storage: Sorting Bits

I'm bored on 4th of July holiday, so I made a wacky webpage: Deep Atlantic Storage. It is described as a free file storage service, where you can upload any file to be stored deep in the Atlantic Ocean, with no size limit and content restriction whatsoever. Since Chia currency farming became popular in May, hard drive prices went up significantly. How can I afford to operate an unlimited free storage service?

"Advanced Sorting Technology"

One of the benefits listed on Deep Atlantic Storage webpage is:

  • Advanced sorting technology keeps your data neatly ordered.

What this means is that, content in the uploaded file would be sorted before being stored.

Watch @EmojiTetra Live on ESP32 OLED Display

@EmojiTetra is an online game resembling Tetris, hosted on Twitter platform. Every 20 minutes, the @EmojiTetra account posts a tweet that displays the current game board, along with a four-option poll that allows visitors to vote for the game's next move: left, right, down, rotate, or plummet.

a tweet by @EmojiTetra

I find this game interesting. To watch or participate in @EmojiTetra, I need to unlock my tablet, open Twitter app, search for "EmojiTetra", and scroll past the pinned tweet in order to see the current game move. In total, this process needs 17 taps. Looking at the 0.96 inch OLED display on my Heltec WiFi_Kit_32 (paid link) board, I'm thinking: can I play @EmojiTetra on an ESP32?

Twitter API

Twitter offers an API that allows applications to retrieve and post tweets. GET statuses/user_timeline resource, for example, retrieves a collection of the most recent tweets posted by a specific user. To watch the game, "user timeline" is exactly what I need to retrieve the current game state.

GAppProxy架构及协议分析

GAppProxy是一款基于Google App Engine的HTTP代理服务器,GAppProxy的作者是dugang。

GAppProxy是用Python语言编写的,包含客户端、fetch服务器、负载平衡器三个模块。

精巧型代理服务器hProxyN的20090223版本加入了GAppProxy插件。 GAppProxy插件使hProxyN可以通过GAppProxy协议转发请求。

GAppProxy模块组成

客户端