w5500 webserver with limited sockets

Question

Hello everyone, I’m implementing a webserver using the W5500. The website that I display has some resources (12 images). I do use all sockets so far and follow the cycle in the provided source code (https://wizwiki.net/wiki/doku.php/products:w5500:driver), which is basically open->listen->establish->receive->answer->close->open->listen->… This setting works fine, but I want to reduce the number of sockets, since I need them for other services. When I use less then 6 sockets (I tested with chrome), some of the resources are not displayed. (I know that chrome usually uses 6 sockets…) Question 1: Is there a way to configure the w5500 such that the browser retries its request, if all sockets are busy ? Question 2: Is there a way to tell the browser how many sockets it is allowed to use when requesting the resources (e.g. in the HTTP header field) ? I do not want to configure my browser settings Question 3: I also started playing with “persistent connection”, however the browser (chrome) does not send a second request, after it has send its first and received its answer. Do you have a code example, where you work with persisten connection ? I was already looking at the implementation of the w550web, but there it also uses the open->listen->establish->receive->answer->close cycle, although the http connection is set to keep-alive (might be a bug)

acpie360 · Answer

I only have three sockets reserved for an embedded web server. It has no problem with Chrome. However, I do use the non-blocking mode (SF_IO_NONBLOCK) for all three sockets.

Mxyxixptlick · Answer

Most browsers will open up to 6 concurrent connections and will not retry if a connection is rejected (e.g., because you don’t have one listening when it tries to connect). I revised an existing W5500 library to actively manage sockets while reserving some for other purposes. You’re welcome to use it, or to scavenge it for ideas or code snippets that serve your needs. Look in particular at sockets.cpp, in which I’ve implemented a function named manageSockets() that closes connections as needed to maintain the number of listeners you specify and leave a socket untouched if allocated to other purposes. Note that it depends on other functions recording the last time of use for each socket. You can find my version of the library at https://github.com/SapientHetero/Ethernet

Mxyxixptlick · Answer

I’ve found no way to run a webserver with less than 6 sockets using the current versions of Chrome, FireFox, Brave, Opera or Microsoft Edge browsers other than limiting the number of resources used by the pages you load. Browsers open multiple sockets to improve their own performance by loading resources in parallel at the expense of imposing a greater load on servers that they view as “not my problem”. There’s no way to limit the number of concurrent sockets used or to force retries of refused connections. This is what led me to write my socket management function. You could change your pages to reduce the opportunity for browsers to demand concurrent sockets. Embed javascripts and css in your header, and Base64-encode any jpg images and embed them in your HTML. You could also host resources that don’t strictly need to be on your system (e.g., .css & script files) on another server. If you eliminate all “linked” resources in your pages, the browser would need only one socket to load them. Another option is to specify long cache periods using the Cache-Control pragma for linked resources in your pages if they don’t change often. You’ll still get refused connections the first few times you load your pages, as the browser must load every resource once in order to cache it. But once all are cached you’ll be fine until the specified cache period expires. The downside is that if these resources change before the cache period expires, users will have to be trained to force a reload using the relevant browser feature. You haven’t said what your “other services” are, but clever socket sharing may allow you to do what you need despite browsers hogging 6 while loading pages. Is it possible to spread these services out in time, perhaps queuing them to run as sockets become available, or must they run on a strict schedule? If you use my code you’ll notice that I’ve included calls to a wdtClear() function in certain loops. This function resets a watchdog timer on the ATSAMD5x…

benjamin3er · Answer

Mxyxixptlick: I’ve found no way to run a webserver with less than 6 sockets using the current versions of Chrome, FireFox, Brave, Opera or Microsoft Edge browsers other than limiting the number of resources used by the pages you load. Browsers open multiple sockets to improve their own performance by loading resources in parallel at the expense of imposing a greater load on servers that they view as “not my problem”. There’s no way to limit the number of concurrent sockets used or to force retries of refused connections. This is what led me to write my socket management function. That’s how it feels, it would have been really nice though. Maybe the w5500 itself could have some “pseudo” sockets that have window size 0 and force the browser to retry until it gets a “real” socket. But this is a completely different story. I might start a new discussion on that… Mxyxixptlick: You could change your pages to reduce the opportunity for browsers to demand concurrent sockets. Embed javascripts and css in your header, and Base64-encode any jpg images and embed them in your HTML. You could also host resources that don’t strictly need to be on your system (e.g., .css & script files) on another server. If you eliminate all “linked” resources in your pages, the browser would need only one socket to load them. That looks like a hack that could work. I have only basic knowledge about web programming, so I was looking into http2 and pipelining (HTTP pipelining - Wikipedia), which looked very promising but did not work since most browser do not support pipelining Mxyxixptlick: You haven’t said what your “other services” are, but clever socket sharing may allow you to do what you need despite browsers hogging 6 while loading pages. Is it possible to spread these services out in time, perhaps queuing them to run as sockets become available, or must they run on a strict schedule? I have a TCP communication to poll status data from the uC. I know this takes only one socket (using a…

Mxyxixptlick · Answer

“what if the next update of chrome allows 8 or 10 parallel socket requests…” Actually, they used to use more sockets just a few years ago. Not sure why they dialed it back, perhaps in recognition of the coming IoT? Still, I feel your pain. It would be nice to have a device as inexpensive as the W5500 that supported more sockets.