Saturday 12 August 2023

ESP8266Webserver and POST requests

Good examples of the Arduino ESP8266Webserver are difficult to find when it comes to writing a HTTP server which accepts POST.

There are different ways a POST request can commonly be made:

  • Using a form
  • Performing a simple fetch in Javascript
  • Performing a multipart fetch in Javascript

And there are different ways the server should respond depending on how the submission is made.

The Request

Using a form

<form action="/upload?o=option" method="POST" enctype="multipart/form-data">
    <p>File: <input type="file" name="name"></p>
    <input type="submit" value="Upload">
</form>

Performing a multipart Fetch in Javascript

Performing a multipart fetch in Javascript requires the use of a FormData() object, which is populated with the files.  For each entry that is appended to the form, a label, a binary object containing the file, and a filename are supplied.

The webserver is unable to access the Mime Type for the each supplied object (e.g. application/octet-stream, so the Mime Type must be inferred from the filename or the label).  Additional information can be passed with a query parameter - e.g. ?o=option if needed.

// Prepare binary and text data
const binarydata = new Uint8Array(new ArrayBuffer(1024)) ;
const textdata = "textmessage" ;
compressText(textdata, binarydata) ;

// Create formData object
const formData = new FormData() ;

// Store binary data
const b_blob = new Blob([binarydata], { type: "application/octet-stream"}) ;
formData.append("binarylabel", b_blob, "binary.dat") ;

// Store text data
const t_blob = new Blob([textdata], { type: "text/plain" }) ;
formData.append("textlabel", t_blob, "text.txt") ;

// Post the message
const request = new XMLHttpRequest() ;
request.open("POST", "/upload?o=option") ;
request.send(formData) ;

Performing a Simple Request in Javascript

Simple requests can be made, but only readable text can be supplied.

fetch("/upload?o=option", {
  method: "POST",
  body: JSON.stringify({
    "id": 7,
    "complete": true
  }),
  headers: {
    "Content-type": "application/json"
  }
});

ESP8266Webserver on

The ESP8266Webserver has an 'on' function, which can be used to register handlers for different uris.  There are several forms the 'on' function can take, and two of them are relevant.

The on function can be supplied with three callbacks, which the documentation calls 'fn' and 'ufn'.  'ufn' represents the upload function. so one may think that this is always used, however that is not the case.  In fact, 'ufn' is only used with multipart and form posts, and simple requests actually use the 'fn' callback.
  1. void on(const Uri &uri, THandlerFunction handler);
  2. void on(const Uri &uri, HTTPMethod method, THandlerFunction fn);
  3. void on(const Uri &uri, HTTPMethod method, THandlerFunction fn, THandlerFunction ufn);

void on(const Uri &uri, THandlerFunction handler)

The first callback simply maps to the second callback with the method fixed to HTTP_ANY, which allows any of the methods to be used


void on(const Uri &uri, HTTPMethod method, THandlerFunction fn)

The second callback registers a uri along with a method  and provides a callback function, for example:

Webserver.on("/get", HTTP_GET, myGetFunction) 

In the above example callback is made with no reference to any class object, and if declared within a class, it must be declared as a static function (see lambda function discussion below for an alternative).

class WebServer::ESP8266WebServer {

    WebServer() {
        on("/handle", HTTP_ANY, WebServer::doHandle) ;
    }

    static void doHandle() {
      Server.doProcess() ;
    }

    void doProcess() {
     switch (method()) {
       case HTTP_GET:
          send(200) ;
          break ;
       case HTTP_POST:
          // body of the post is treated specially
          // note that this is a simple mechanism
          // and can only handle text
          String postcontents = arg("plain") ;
          send(200) ;
          break ;
       default:
          send(500) ;
          break ; 
  }

}

void on(const Uri &uri, HTTPMethod method, THandlerFunction fn, THandlerFunction ufn)

The third callback is by far the most complex, and includes an upload function which is called to process each file within a multi-part upload (i.e. uploads which have a Content-Type of multipart/*).

If the upload is not multi-part, then the standard fn is called.

This function can be used with the HTTP_ANY method (unlike the ESP32 version of the webserver, the method is not a bitfield, and HTTP_GET | HTTP_POST is not allowed).

In this example, lambda functions [&]() are used to retain the context in the callback.

  on(
    "/config",
    HTTP_ANY,
    [&]() {
      switch (method()) {
        case HTTP_GET:
          serveConfigFile() ;
          break ;
        case HTTP_POST:
          send(400, "text/plain", "Requires multipart/* Content-Type");
          break ;
        default:
          send(405, "text/plain", "Method not supported");
          break ;
      }
    },
    [&]() {
      handleConfigUpload() ;
    }
  );


Disadvantages of the 'fn' callback:
  1. Binary uploads are not supported
  2. The text upload is stored in a string, and the user has no control over memory allocations

Upload Callback Function

The upload callback function is called several times for each file within a multi-part attachment.  For each part, the upload moves through several states:

UPLOAD_FILE_START:

In this state, the handler needs to prepare to receive the incoming file.
This could mean allocating memory, or opening a file handle ready to receive  the file data.
It is not clear as to whether there could be two concurrent uploads - one in the FILE_WRITE state and another in the FILE_START state.

Details of the transaction can be obtained through the upload function:

upload().filename - name of the file as supplied on the form / in the javascript
upload().name - label of the entry as supplied on the form / in the javascript
upload().type - would have expected this to be the mime type of the object, but it is currently random rubbish (12/08/2023)

UPLOAD_FILE_WRITE:

In this state, data is received from the upload (a separate call for each chunk of data, so a large file may have multiple calls).
Here, the handler needs to capture and store the received data.

UPLOAD_FILE_END:

In this state, the upload is complete.
Here, the handler needs to close, validate and store the uploaded file.