A captive portal is a web page displayed to newly connected users before they are granted broader access to network resource. ESP8266, when configured as a WiFi access point, can serve a captive portal. On the other side of the spectrum, ESP8266 can be used as a WiFi client (aka STAtion), and it should be able to "click through" a captive portal as well.
Clearwave Solutions LLC provides WiFi service at my apartment. Their WiFi network shows a captive portal to each connected client once a week. The captive portal page has a giant Connect button, which instantly enables Internet access.
Having to press the Connect button every week is an annoyance on a computer or mobile phone. When it comes to little devices such as a temperature and humidity sensor, the existence of captive portal is a bigger problem because the ESP8266 does not have a web browser, so it is impossible for me press the Connect button. However, from network point of view, as long as the ESP8266 sends the correct packets, the WiFi gateway would think the button has been pressed.
To click through the captive portal without a web browser on the ESP8266, I just need to:
- Figure out what packets a client sends when pressing the Connect button.
- Write a program to send those packets from the ESP8266.
Fiddler: Capture the Packets from Captive Portal
Fiddler is a web debugging proxy. It runs on Windows, and can capture HTTP and HTTPS traffic from any web browser or other web clients. When my laptop is presented the captive portal, I opened Fiddler and captured an interaction.
Download: raw Fiddler session archive
From the traffic capture, I can see that when I attempted to access http://captive.apple.com, I'm redirected to a page on 104.236.150.115:8880, and a cookie is set under that domain.
Pressing the Connect button triggers a POST to http://104.236.150.115:8880/guest/s/c3o3zy94/login, after which I'm redirected two more times until reaching the original requested web page http://captive.apple.com.
Arduino Sketch to Click Through the Captive Portal
captiveLogin function uses ESP8266HTTPClient library to send HTTP requests to emulate clicking through a captive portal.
It performs the same steps as seen in the Fiddler traffic capture.
ESP8266HTTPClient by default discards HTTP response headers in order to save memory.
Since we need to obtain redirected URI and cookies, HTTPClient::collectHeaders is called so that HTTPClient class saves Location and Set-Cookie headers when parsing HTTP response headers.
Near the end of captiveLogin function, Internet connectivity is tested with both http://captive.apple.com/ and http://portquiz.net:8080/.
The reason of introducing the latter is that, the WiFi gateway sometimes still returns HTTP 302 status when port 80 is accessed immediately after "pressing" the Connect button, but access to other ports is not affected.
This code also includes randomizing the MAC address and DHCP hostname, so that I can test the procedure with a "new" client each time I reset the ESP8266. This requires copying a little ChangeMac library into the sketch.
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include "ChangeMac.hpp"
const char* ssid = "UniversityArms-WiFi";
const char* password = "<the-password>";
bool captiveLogin() {
static const char* LOCATION = "Location";
static const char* SET_COOKIE = "Set-Cookie";
static const char* HEADER_NAMES[] = {LOCATION, SET_COOKIE};
String uri;
{
HTTPClient http;
http.begin("http://captive.apple.com/");
http.collectHeaders(HEADER_NAMES, 2);
int httpCode = http.GET();
if (httpCode == 200) {
return true;
}
if (httpCode != 302 || !http.hasHeader(LOCATION)) {
return false;
}
uri = http.header(LOCATION);
Serial.print("portal=");
Serial.println(uri);
delay(2000);
}
String cookie;
{
HTTPClient http;
http.begin(uri);
http.collectHeaders(HEADER_NAMES, 2);
int httpCode = http.GET();
if (httpCode != 200 || !http.hasHeader(SET_COOKIE)) {
return false;
}
cookie = http.header(SET_COOKIE);
Serial.print("cookie=");
Serial.println(cookie);
delay(3000);
}
{
int pos = uri.lastIndexOf("/?");
if (pos < 0) {
return false;
}
HTTPClient http;
http.begin(uri.substring(0, pos) + "/login");
http.addHeader("Content-Type", "application/x-www-form-urlencoded");
http.addHeader("Cookie", cookie);
http.collectHeaders(HEADER_NAMES, 2);
int httpCode = http.POST("connect=Connect");
if (httpCode != 302 || !http.hasHeader(LOCATION)) {
return false;
}
uri = http.header(LOCATION);
cookie = http.header(SET_COOKIE);
Serial.print("redirect=");
Serial.println(uri);
delay(500);
}
{
HTTPClient http;
http.begin(uri);
int httpCode = http.GET();
if (httpCode != 302) {
return false;
}
delay(500);
}
{
HTTPClient http;
http.begin("http://captive.apple.com/");
int httpCode = http.GET();
if (httpCode == 200) {
return true;
}
}
{
HTTPClient http;
http.begin("http://portquiz.net:8080/");
int httpCode = http.GET();
return httpCode == 200;
}
}
void setup() {
Serial.begin(115200);
Serial.println();
Serial.println();
WiFi.mode(WIFI_STA);
WiFi.persistent(false);
uint8_t mac[6];
makeRandomMac(mac);
changeMac(mac);
Serial.print("MAC address is ");
Serial.println(WiFi.macAddress());
String hostname = "iPad-";
hostname += random(10);
hostname += random(10);
hostname += random(10);
hostname += random(10);
WiFi.hostname(hostname);
Serial.print("Hostname is ");
Serial.println(hostname);
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(WiFi.status());
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
if (!captiveLogin()) {
ESP.restart();
}
}
void loop() {
}Testing
To test this sketch, and have enough logs in case something goes wrong, it is a good idea to enable debug logs in Arduino IDE:
On the console log, I can see that the ESP8266 has successfully clicked through the captive portal and connected to the Internet.
