Wiznet makers

Hardwired TCP/IP Chapter 8: W55MH32 HTTP Client Example

In this article, we will provide a detailed explanation on how to implement the HTTP Client function on the W55MH32 chip. Through practical examples, we will also explain to you how to submit data to a specified website. For ease of explanation, in this example, we have chosen a website specifically for testing the HTTP protocol: httpbin.org. We have implemented both the commonly used GET and POST methods for submitting data via HTTP, for your reference.

Other network protocols used in this example, such as DHCP and DNS, as well as the initialization process of the W55MH32, please refer to the relevant chapters. We will not elaborate on them here.

1 Introduction to the HTTP Protocol

HTTP (HyperText Transfer Protocol) is an application layer protocol used in distributed, collaborative, and hypermedia information systems. It transmits data based on the TCP/IP communication protocol and forms the basis of data communication in the World Wide Web (WWW). The original purpose of designing HTTP was to provide a method for publishing and receiving HTML pages. The resources requested through the HTTP or HTTPS protocol are identified by Uniform Resource Identifiers (URI).

The above is a brief introduction to the HTTP protocol. If you want to have a deeper understanding of this protocol, please refer to the introduction on the Mozilla website:  HTTP Overview - HTTP | MDN

2 Characteristics of the HTTP Protocol

• Based on the request-response model: The client initiates a request, and the server processes it and returns a response. For example, when a user enters a website address in the browser, the browser sends an HTTP request to the corresponding server, and the server returns the web page content.
• Statelessness: HTTP itself does not store the state between requests; each request is independent. However, state can be maintained through mechanisms such as Cookies and Sessions.
• Connectionless: The meaning of connectionless is to limit each connection to handle only one request. After the server processes the client's request and receives the client's response, it immediately disconnects the connection.

3 HTTP application scenarios

Next, let's take a look at what operations and applications can be accomplished using the HTTP client mode on the W55MH32?

• Data collection and upload: The data collected by the sensors is uploaded to the server.
• Remote configuration and management: By requesting configuration files or management instructions from the server, remote management and control can be achieved. For example, industrial control devices can obtain the latest operation strategies or instructions.
• Firmware update (OTA): By requesting the download of the latest firmware package from the server, the function of remotely upgrading the firmware can be realized.
• Log and error report upload: System operation logs are uploaded at regular intervals to analyze the device status, or error reports are uploaded when abnormal situations occur, facilitating quick location and problem-solving.
• User authentication and authorization management: Through interaction with the server, the identity of users or devices is verified.

4 The basic workflow of the HTTP protocol

The request-response model of HTTP usually consists of the following steps:

1. Establish connection: A connection is established between the client and the server based on the TCP/IP protocol.
2. Send request: The client sends a request to the server, which includes the URL of the resource to be accessed, the request method (GET, POST, PUT, DELETE, etc.), request headers (such as Accept, User-Agent), and an optional request body (for POST or PUT requests).
3. Process request: After receiving the request, the server finds the corresponding resource based on the information in the request and performs the corresponding processing operation. This may involve retrieving data from a database, generating dynamic content, or simply returning a static file.
4. Send response: The server encapsulates the processed result in a response and sends it back to the client. The response includes a status code (used to indicate the success or failure of the request), response headers (such as Content-Type, Content-Length), and an optional response body (such as an HTML page, image data).
5. Close connection: After completing the request-response cycle, the connection between the client and the server will be closed, unless persistent connection (such as keep-alive in HTTP/1.1) is used.

5 HTTP request method

In the HTTP protocol, GET and POST are two commonly used request methods, which are used by the client to send data to the server and obtain resources.

GET Method

• The GET method is typically used to retrieve resources from the server. It has the following characteristics:
• Parameter passing: Request parameters are passed through the query string in the URL, in the form of ?key1=value1&key2=value2.
• Data size limit: Since the parameters are appended to the URL, their length may be limited by the URL length (depending on the settings of the browser and server).
• Security: The data is displayed in plain text in the URL, and is not suitable for transmitting sensitive information.

Request format:

GET <Request-URI> HTTP/<Version>
<Headers>
<Blank Line>

• Request-URI: Represents the path of the target resource, which may include parameters.
• Version: HTTP protocol version.
• Headers: Contains metadata, such as client attributes, supported formats, etc.
• Blank Line: Empty line.

POST method

The POST method is typically used to submit data to the server. It has the following characteristics:

• Parameter passing: Data is placed in the request body instead of in the URL.
• Data size limit: There is no obvious limit on the volume of POST requests, and a large amount of data can be transmitted.
• Security: Data is transmitted in the request body, which is relatively more secure.

Request format:

POST <Request-URI> HTTP/<Version>
<Headers>
<Blank Line>
<Body>

• Request-URI: The path of the target resource, usually the endpoint of the API.
• Headers: Metadata, such as content type and length.
• Blank Line: An empty line, used to separate headers from the body.
• Body: The main part of the data, including the length of the data sent by the client to the server.

6 HTTP protocol response content

The HTTP protocol response content consists of three parts: the status line, the response header, and the response body.

Status Line

The HTTP status line includes the HTTP protocol version, the status code, and the status description.

The status code is composed of three decimal numbers, and the first decimal number defines the type of the status code.

The status codes are divided into five categories:

• 1xx (Informational Status Code): Indicates that the received request is being processed.
• 2xx (Successful Status Code): Indicates that the request has been processed successfully.
• 3xx (Redirection Status Code): Requires subsequent actions to complete this request.
• 4xx (Client Error Status Code): Indicates that the request contains a syntax error or cannot be completed.
• 5xx (Server Error Status Code): The server encountered an error while processing the request.

Example:

HTTP/1.1 200 OK

Response headers

The response headers will contain information such as content type, length, and encoding.

Common response header fields include:

• Content-Type: The MIME type of the response content, such as text/html, application/json.
• Content-Length: The byte length of the response content.
• Server: Information about the server.
• Set-Cookie: Sets the Cookie for the client.

Example:

Content-Type: text/html; charset=UTF-8
Content-Length: 3495
Server: Apache/2.4.41 (Ubuntu)

Response body

The response body contains the actual data content, and its specific form depends on the type of response and the content of the request. For example: HTML page content, JSON data, binary data of a file, etc.

If the status code of the response is 204 No Content or 304 Not Modified, there is usually no body.

Note: An empty line is added between the response body and the response header to separate the content.

7 HTTP Request and Response Examples

An example of a GET request is as follows:

//Request

GET /get?username=admin&password=admin HTTP/1.1
Host:httpbin.org

//Response

HTTP/1.1 200 OK
Date: Tue, 10 Dec 2024 10:41:13 GMT
Content-Type: application/json
Content-Length: 278
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {
    "password": "admin", 
    "username": "admin"
  }, 
  "headers": {
    "Host": "httpbin.org", 
    "X-Amzn-Trace-Id": "Root=1-67581ac9-236349c67cb21dcc24c54215"
  }, 
  "origin": "118.99.2.9", 
  "url": "http://httpbin.org/get?username=admin&password=admin"
}

An example of a POST request is as follows:

//Request

POST /post HTTP/1.1
Host:httpbin.org
Content-Type:application/x-www-form-urlencode
Content-Length:29

username=admin&password=admin

//Response

HTTP/1.1 200 OK
Date: Tue, 10 Dec 2024 10:44:52 GMT
Content-Type: application/json
Content-Length: 374
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {}, 
  "data": "username=admin&password=admin", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Content-Length": "29",
    "Content-Type": "application/x-www-form-urlencode", 
    "Host": "httpbin.org", 
    "X-Amzn-Trace-Id": "Root=1-67581ba4-744fdecf1da1bcf3180f9fa3"
  }, 
  "json": null, 
  "origin": "118.99.2.9", 
  "url": "http://httpbin.org/post"
}

8 The implementation process

Next, let's take a look at how to implement an example of sending GET and POST requests in HTTP client mode on the W55MH32.

Note: Since this example requires access to the internet, please ensure that the network environment and configuration of the W55MH32 can access the internet normally.

The example uses the server at httpbin.org. The GET request path is httpbin.org/get, and the POST request path is httpbin.org/post.

After we send the request to httpbin.org, it will include the parameters submitted during our request in the response content.

Step 1: Resolve the domain name of the HTTP server using the DNS protocol

When submitting data to the httpbin.org server via HTTP, the first step is to establish a TCP connection with the server. It should be noted that establishing a TCP connection requires performing connection operations based on the IP address. The task undertaken by the do_dns() function here is to resolve the domain name using the DNS protocol, thereby obtaining the corresponding IP address.

1.     if (do_dns(ethernet_buf, org_server_name, org_server_ip))
2.     {
3.         printf("DNS request failed.\r\n");
4.         while (1)
5.         {
6.         }
7.     }

Step 2: Package HTTP GET and POST requests

Next, we need to package the data according to the specifications of the HTTP protocol as described earlier.

In this routine, we have defined two functions to generate the HTTP GET Header() and HTTP POST Header() respectively. The detailed code is as follows:

 1. /**
 2.  * @brief  HTTP GET :  Request package combination package.
 3.  * @param  pkt:    Array cache for grouping packages
 4.  * @return pkt:    Package length
 5.  */
 6. uint32_t http_get_pkt(uint8_t *pkt)
 7. {
 8.     *pkt = 0;
 9.     // request type URL HTTP protocol version
10.     strcat((char *)pkt, "GET /get?username=admin&password=admin HTTP/1.1\r\n");
11.  
12.     // Host address, which can be a domain name or a specific IP address.
13.     strcat((char *)pkt, "Host: httpbin.org\r\n");
14.     // end
15.     strcat((char *)pkt, "\r\n");
16.  
17.     return strlen((char *)pkt);
18. }
19.  
20. /**
21.  * @brief   HTTP POST :  Request package combination package.
22.  * @param   pkt:    Array cache for grouping packages
23.  * @return  pkt:    Package length
24.  */
25. uint32_t http_post_pkt(uint8_t *pkt)
26. {
27.     *pkt = 0;
28.     // request type URL HTTP protocol version
29.     strcat((char *)pkt, "POST /post HTTP/1.1\r\n");
30.  
31.     // Host address, which can be a domain name or a specific IP address.
32.     strcat((char *)pkt, "Host: httpbin.org\r\n");
34.     // Main content format
35.     strcat((char *)pkt, "Content-Type:application/x-www-form-urlencode\r\n");
36.  
37.     // Main content lenght
38.     strcat((char *)pkt, "Content-Length:29\r\n");
39.  
40.     // separator
41.     strcat((char *)pkt, "\r\n");
42.  
43.     // main content
44.     strcat((char *)pkt, "username=admin&password=admin");
45.     return strlen((char *)pkt);
46. }

The content of the http_get_pkt() packet is as follows:

GET /get?username=admin&password=admin HTTP/1.1
Host: httpbin.org

The content of the http_post_pkt() packet assembly is as follows:

POST /post HTTP/1.1
Host: httpbin.org
Content-Type:application/x-www-form-urlencode
Content-Length:29

username=admin&password=admin

Step 3: Send HTTP request, handle timeout and response content

1.     len = http_get_pkt(ethernet_buf);
2.     do_http_request(SOCKET_ID, ethernet_buf, len, org_server_ip, org_port);
3.  
4.     // Send a POST request.
5.     len = http_post_pkt(ethernet_buf);
6.     do_http_request(SOCKET_ID, ethernet_buf, len, org_server_ip, org_port);

The do_http_request() function is used to send an HTTP request and listen for the response. The detailed content is as follows:

  1. /**
  2.  * @brief  HTTP Client get data stream test.
  3.  * @param  sn:         socket number
  4.  * @param  buf:        request message content
  5.  * @param                                           len                                                                                                             request message length
  6.  * @param  destip:     destion ip
  7.  * @param  destport:   destion port
  8.  * @return 0:timeout,1:Received response..
  9.  */
 10.  
 11. uint8_t do_http_request(uint8_t sn, uint8_t *buf, uint16_t len, uint8_t *destip, uint16_t destport)
 12. {
 13.     uint16_t local_port   = 50000;
 14.     uint16_t recv_timeout = 0;
 15.     uint8_t  send_flag    = 0;
 16.     while (1)
 17.     {
 18.         switch (getSn_SR(sn))
 19.         {
 20.         case SOCK_INIT:
 21.             // Connect to http server.
 22.             connect(sn, destip, destport);
 23.             break;
 24.         case SOCK_ESTABLISHED:
 25.             if (send_flag == 0)
 26.             {
 27.                 // send request
 28.                 send(sn, buf, len);
 29.                 send_flag = 1;
 30.                 printf("send request:\r\n");
 31.                 for (uint16_t i = 0; i < len; i++)
 32.                 {
 33.                     printf("%c", *(buf + i));
 34.                 }
 35.                 printf("\r\n");
 36.             }
 37.             // Response content processing
 38.             len = getSn_RX_RSR(sn);
 39.             if (len > 0)
 40.             {
 41.                 printf("Receive response:\r\n");
 42.                 while (len > 0)
 43.                 {
 44.                     len = recv(sn, buf, len);
 45.                     for (uint16_t i = 0; i < len; i++)
 46.                     {
 47.                         printf("%c", *(buf + i));
 48.                     }
 49.                     len = getSn_RX_RSR(sn);
 50.                 }
 51.                 printf("\r\n");
 52.                 disconnect(sn);
 53.                 close(sn);
 54.                 return 1;
 55.             }
 56.             else
 57.             {
 58.                 recv_timeout++;
 59.                 delay_ms(1000);
 60.             }
 61.             // timeout handling
 62.             if (recv_timeout > 10)
 63.             {
 64.                 printf("request fail!\r\n");
 65.                 disconnect(sn);
 66.                 close(sn);
 67.                 return 0;
 68.             }
 69.             break;
 70.         case SOCK_CLOSE_WAIT:
 71.             // If there is a request error, the server will immediately send a close request,
 72.             // so the error response content needs to be processed here.
 73.             len = getSn_RX_RSR(sn);
 74.             if (len > 0)
 75.             {
 76.                 printf("Receive response:\r\n");
 77.                 while (len > 0)
 78.                 {
 79.                     len = recv(sn, buf, len);
 80.                     for (uint16_t i = 0; i < len; i++)
 81.                     {
 82.                         printf("%c", *(buf + i));
 83.                     }
 84.                     len = getSn_RX_RSR(sn);
 85.                 }
 86.                 printf("\r\n");
 87.                 disconnect(sn);
 88.                 close(sn);
 89.                 return 1;
 90.             }
 91.             close(sn);
 92.             break;
 93.         case SOCK_CLOSED:
 94.             // close socket
 95.             close(sn);
 96.             // open socket
 97.             socket(sn, Sn_MR_TCP, local_port, 0x00);
 98.             break;
 99.         default:
100.             break;
101.         }
102.     }
103. }

In this function, the program will execute a TCP Client mode state machine. For a detailed explanation, please refer to the TCP Client example section. When the program is in the SOCK_ESTABLISHED state, it will send one request content to the server. Then, it will listen for the server's response data and handle the timeout situation.

If the server returns an abnormal response, it will immediately close the connection. Therefore, we need to handle the content of the server's abnormal response in the SOCK_CLOSE_WAIT state.

9 Run results

After the burning routine is executed, the first thing you will see is that the PHY link detection and DHCP acquisition of network information have been printed. Then, the result of DNS resolving the domain name of the HTTP server is shown, as shown in the following figure:

Then we sent a GET request message, and the HTTP server returned a response message.

Both the request and response messages were printed out via the serial port, as shown in the following figure:

Finally, we sent a POST request message, and then the HTTP server returned a response message.

Both the request and response messages were printed out via the serial port, as shown in the following figure:

10 Summary

This article describes the method of implementing the HTTP Client function on the W55MH32 chip, and explains how to obtain data from the httpbin.org website. It elaborates on the concepts, characteristics, application scenarios, workflow, request methods, and response contents of the HTTP protocol, and provides request and response examples. It also demonstrates the implementation process on the W55MH32 chip.

The next article will explain how to implement the HTTP Server function on this chip, and introduce the principle and implementation steps of modifying the network address information of the W55MH32 through the browser. Stay tuned!

WIZnet is a non-chipfoundry semiconductor company founded in 1998. Its products include the Internet processor iMCU™, which adopts TOE (TCP/IP Offloading Engine) technology and is based on a unique patented fully hardwired TCP/IP. iMCU™ is designed for embedded Internet devices in various applications.

WIZnet has over 70 distributors worldwide, with offices in Hong Kong, South Korea, and the United States, providing technical support and product marketing.

The region managed by the Hong Kong office includes: Australia, India, Turkey, and Asia (excluding South Korea and Japan).