Standardized REST APIs, SOAP services and GraphQL APIs are only a part of the spectrum in HTTP requests and this module consists of the other types of commonly appearing HTTP requests which have some specific features that should be remembered. Also, sometimes some of these topics may be used in conjunction with the previously covered HTTP request practices.

Previously we covered transferring data over HTTP in JSON, XML or any other structured types, but sending or retrieving entire files or any payload as byte array is also possible. Therefore, this requires handling the files in the HTTP payload as byte arrays which is possible with three different Frends Tasks.

These Tasks have their own use cases and their names should indicate what to use them for. Only the payload, response or restricted collection of HTTP methods differ from HttpRequest or RestRequest Tasks. After all, HttpRequest and RestRequest Tasks eventually handle the payloads as byte arrays on the transmission protocol level but make the development easier as they provide the payloads as structured string formats for the human eyes. This means that any HTTP endpoint that can be consumed with previously mentioned Tasks, can be consumed with HttpRequestBytes, HttpSendBytes or HttpSendAndReceiveBytes Tasks as long as the payloads are converted from string to byte array or from byte array to string according to the use case needs.

HttpRequestBytes for getting bytes as the HTTP response.

HttpSendBytes for sending bytes as part of the HTTP request.

HttpSendAndReceiveBytes for sending bytes as well as receiving them as part of the HTTP request and response.

One can request bytes by using the HttpRequestBytes Task and configuring the HTTP request like any other while expecting the server to return the content as bytes. The Task returns also ContentType property which will indicate what type of file was received allowing properly handling the file later in the Process and the file as byte array itself would be referenced with #result.BodyBytes notation.

Sending bytes over HTTP requires the usage of HttpSendBytes Task which takes the payload as byte array. Also, it is a good practice to use required HTTP headers like Content-Type: octet-stream . Usually the documentation of the HTTP endpoint elaborates what HTTP headers should be used for indicating additional information, for instance the purpose or the name of the transmitted file.

To combine the sending and receiving of bytes, the HttpSendAndReceiveBytes Task must be used which will just combine the functionality of the presented Tasks above.

The following Process demonstrates the common use cases of transporting byte arrays over HTTP and how to write them to file or reading a file as byte array.

Sometimes an HTTP endpoint requires the transferred data to be in the URL of the request, meaning that the data needs to be ASCII format compatible without, for example, spaces and certain punctuations. This created a need for common encoding of these special characters so that the data could appear in URL, hence the name URL encoding.

For instance, in URL encoding space is '+', plus sign '%2B', equal sign '%3D', question mark '%3F' and so on. As an example, if we would URL encode string 'This + this = this! really?', we would get 'This+%2B+this+%3D+this!+really%3F'.

Initially, the URL encoding ensured that all kinds of characters could be transmitted in the URL of HTTP request but nowadays some endpoints require URL encoding in the payload as well. By definition, the query parameters in the URL of an HTTP request are URL encoded resulting in following format: ?key1=UrlEncoded(value1)&key2=UrlEncoded(value2). Additionally HTML forms utilize URL encoding for form submission but in the context of integrations, the main use case of URL encoding is to format the payload or data in URLs. The typical HTTP header for URL encoded data is Content-Type:application/x-www-form-urlencoded.

As an example, we could have an API for a service that sends SMS messages according the HTTP payloads which are key value pairs formatted with URL encoding and separated with &. The following image shows how that imaginary API call would look like, assuming the referenced data is URL encoded already.

Additionally, we could make sure that the referenced values from the trigger truly are URL encoded. For instance, we could URL encode the Message value with a C# expression in the handlebars: #{{WebUtility.UrlEncode(#trigger.data.Message)}}.

The need for URL encoding is usually well indicated in the HTTP endpoint or API documentation.

A multipart HTTP request allows the transmission of multiple files or datasets in a single HTTP request. This practice is utilized when the use case requires sending a file and additional metadata with it. The payload of the multipart HTTP request is split into parts separated by a boundary string defined in an HTTP header. Also the parts have their own descriptive headers. The following image shows an example of a multipart request.

In this example the URL would be www.api.example.com/upload and an HTTP header Content-Type: multipart/form-data; boundary=BOUNDARY_STRING. The body is formed of two parts starting with --BOUNDARY_STRING and the entire body ending with --BOUNDARY_STRING--. In the first part there is a file and in the second part there is a JSON formatted metadata.

Making this type of HTTP request can be achieved in two ways with Frends: using a specific SubmitForm SendForm Task or forming the HTTP requests with HttpRequest or RestRequest Tasks.

One common practice in HTTP requests is to use JSON Web Tokens (JWT) which allows more secure transmission of data between parties as the data is signed with open and standardized method. JWTs consist of header, payload and signature which are base64 URL encoded and joined with a dot to form strings like the following example.

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJwYXlsb2FkIjp7InByb3BlcnR5MSI6InByb3BlcnR5MWRhdGEiLCJwcm9wZXJ0eTIiOiJwcm9wZXJ0eTJkYXRhIn19.jxpj1D36Mruq9mgHgcjpHbyVVtfLAwJ5d8IE9UWhJH0

In this example the header eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9 has information about the hashing algorithm and the type of the token. When that header is converted back to JSON representation, we would have a following object:

{
"alg": "HS256",
"typ": "JWT"
}

The second part of the JWT, the payload eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJwYXlsb2FkIjp7InByb3BlcnR5MSI6InByb3BlcnR5MWRhdGEiLCJwcm9wZXJ0eTIiOiJwcm9wZXJ0eTJkYXRhIn19 contains the claims that are standardized statements about the consumer with additional information. Our example payload would convert to the following JSON object:

{
"sub": "1234567890",
"name": "John Doe",
"iat": 1516239022,
"payload": {
"property1": "property1data",
"property2": "property2data"
}
}

The last part of the JWT, the signature, is calculated with the algorithm stated in the header, in this case HS256. The calculation hashes the header and payload with a possible secret: HMACSHA256(base64UrlEncode(header)+"."+base64UrlEncode(payload),secret). This calculation would eventually produce the signature of jxpj1D36Mruq9mgHgcjpHbyVVtfLAwJ5d8IE9UWhJH0. You can check out this particular example at jwt.io.

There are multiple hashing algorithms available and even algorithms that utilize asymmetric public-private cryptography like RS256.

JWTs are commonly used as the Bearer tokens of OAuth flows as they can provide signed information like the claims about the resource consumer. Sometimes the HTTP payload itself can be a JWT which needs to be validated and parsed to be able to use the data in a integration use case.

Frends provides a set of Tasks for validating, parsing and creating JWTs as part of HTTP integration use cases.

These Tasks should be used in conjunction with the HTTP Tasks to transmit or handle JWTs.

HTTP payloads or authorization information of HTTP requests can be signed to ensure the transferred data itself has not been tampered with.

The next article is Introduction to Authentication in HTTP integrations

​