W5500 UDP Packets bigger than 1257 lost

Question

HI There, i was searching the whole internet but couldn’t find a solution for my problem… I built a UDP broadcast receiver with an ESP32-Wroom and the W5500 using the SPI interface. So far so good, the Receiver works whilst receiving packages up to the size range of 1257 bytes. The langer ones fail to get recognized or are dumped by udp.parsePacket() using the Ethernet2.h library. I can see these packages in Wireshark but the receiver doesnt. The maximum size of the packages is 1464 which is below the threshold I believe. this package could be received via the ESP WLAN feature but that was way to slow for my puropses. Here’s the part of the code… let me know what else you need to find my mistake… #include #include #include #include "local_config.h" #define PIN_CS 5 #define UDP_PACKET_SIZE 1500 byte packetBuffer[UDP_PACKET_SIZE]; //The IP address that this arduino has requested to be assigned to. EthernetUDP Udp; uint16_t packetSize = 0; bool state = 0; const uint16_t localPort = 22815; /* * Wiz W5500 reset function. Change this for the specific reset * sequence required for your particular board or module. */ void WizReset() { Serial.print("Resetting Wiz W5500 Ethernet Board... "); pinMode(RESET_P, OUTPUT); digitalWrite(RESET_P, HIGH); delay(250); digitalWrite(RESET_P, LOW); delay(50); digitalWrite(RESET_P, HIGH); delay(350); Serial.println("Done."); } /* * This is a crock. It's here in an effort * to help people debug hardware problems with * their W5100 ~ W5500 board setups. It's * a copy of the Ethernet library enums and * should, at the very least, be regenerated * from Ethernet.h automatically before the * compile starts (that's a TODO item). * */ /* * Print the result of the hardware status enum * as a string. * Ethernet.h currently contains these values:- * * enum EthernetHardwareStatus { * EthernetNoHardware, * EthernetW5100, * EthernetW5200, * EthernetW5500 * }; * */ void prt_hwval(uint8_t refval) { switch (refval) { case 0: Serial.println("No hardware detected."); break; case 1: Serial.println("WizNet W5100 detected."); break; case 2: Serial.println("WizNet W5200 detected."); break; case 3: Serial.println("WizNet W5500 detected."); break; default: Serial.println ("UNKNOWN - Update espnow_gw.ino to match Ethernet.h"); } } /* * Print the result of the ethernet connection * status enum as a string. * Ethernet.h currently contains these values:- * * enum EthernetLinkStatus { * Unknown, * LinkON, * LinkOFF * }; * */ void prt_ethval(uint8_t refval) { switch (refval) { case 0: Serial.println("Uknown status."); break; case 1: Serial.println("Link flagged as UP."); break; case 2: Serial.println("Link flagged as DOWN. Check cable connection."); break; default: Serial.println ("UNKNOWN - Update espnow_gw.ino to match Ethernet.h"); } } void sendNTPpacket(const char *address) { // Set all bytes in the buffer to 0. memset(packetBuffer, 0, 1500); // Initialize values needed to form NTP request // (see…

Eugeny · Answer

Not familiar with the library, found this, I would start looking into the library and putting diagnostic output after each operation or register access to identify what is going on.

sisvpfis · Answer

WIreshark output from Microsoft UDP Tool Package with 1472byte of Data: 0000 ff ff ff ff ff ff 70 5a b6 91 9f 8b 08 00 45 00 ......pZ......E. 0010 05 d4 94 c4 00 00 80 11 00 00 c0 a8 b2 14 ff ff ................ 0020 ff ff fa bf 51 29 05 c0 a1 5e 61 62 63 64 65 66 ....Q)...^abcdef 0030 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 ghijabcdefghijab 0040 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 cdefghijabcdefgh 0050 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 ijabcdefghijabcd 0060 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a efghijabcdefghij 0070 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 abcdefghijabcdef 0080 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 ghijabcdefghijab 0090 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 cdefghijabcdefgh 00a0 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 ijabcdefghijabcd 00b0 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a efghijabcdefghij 00c0 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 abcdefghijabcdef 00d0 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 ghijabcdefghijab 00e0 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 cdefghijabcdefgh 00f0 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 ijabcdefghijabcd 0100 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a efghijabcdefghij 0110 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 abcdefghijabcdef 0120 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 ghijabcdefghijab 0130 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 cdefghijabcdefgh 0140 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 ijabcdefghijabcd 0150 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a efghijabcdefghij 0160 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 abcdefghijabcdef 0170 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 ghijabcdefghijab 0180 63 64 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 cdefghijabcdefgh 0190 69 6a 61 62 63 64 65 66 67 68 69 6a 61 62 63 64 ijabcdefghijabcd 01a0 65 66 67 68 69 6a 61 62 63 64 65 66 67 68 69 6a efghijabcdefghij 01b0 61 62 63 64 65 66 67 68 69 6a…

sisvpfis · Answer

WIreshark output from F1 22 Package with 1472byte of Data: 0000 ff ff ff ff ff ff d8 5e d3 a3 24 45 08 00 45 00 .......^..$E..E. 0010 05 d4 5d 97 00 00 80 11 64 7c c0 a8 b2 5d ff ff ..].....d|...].. 0020 ff ff d7 cf 51 29 05 c0 0d 86 e6 07 01 0b 01 00 ....Q).......... 0030 a9 7e 81 ea 9a b0 34 e2 10 08 28 44 a8 77 00 00 .~....4...(D.w.. 0040 00 ff ca 7a af c3 a7 88 bb 42 9b e8 0e c4 78 72 ...z.....B....xr 0050 89 3a 2a 0f b3 b8 a6 96 6a 3c 89 08 a7 00 4b 80 .:*.....j<....K. 0060 a6 7f fe fb 83 08 a6 ae 94 b9 d9 cd 8c bb d6 3b ...............; 0070 34 3a 30 ca 44 40 00 18 a7 bb cc 51 00 3d eb 74 4:0.D@.....Q.=.t 0080 c5 c3 b0 ec b5 42 ab 01 0a 43 00 00 00 00 00 00 .....B...C...... 0090 00 00 00 00 00 00 dc 05 4f 00 27 80 d3 7f 8e 02 ........O.'..... 00a0 de 05 00 00 00 00 00 00 00 00 00 00 00 00 12 21 ...............! 00b0 46 40 00 0a 1e bb 37 98 a3 bc 7b 1f bf c3 d4 1f F@....7...{..... 00c0 b6 42 01 48 b4 c2 00 00 00 00 00 00 00 00 00 00 .B.H............ 00d0 00 00 16 05 fc ff 1b 80 e1 7f f8 01 15 05 00 00 ................ 00e0 00 00 00 00 00 00 00 00 00 00 a4 84 46 40 00 40 ............F@.@ 00f0 08 39 20 2e 7c bc 2d ad c5 c3 05 e9 b5 42 46 c4 .9 .|.-......BF. 0100 09 43 00 00 00 00 00 00 00 00 00 00 00 00 d7 05 .C.............. 0110 30 00 27 80 d3 7f 8f 02 d7 05 00 00 00 00 00 00 0.'............. 0120 00 00 00 00 00 00 eb 23 46 40 00 e0 c3 ba 5a c4 .......#F@....Z. 0130 a3 bc 5a 71 bf c3 1b 24 b6 42 96 3d 94 c2 00 00 ..Zq...$.B.=.... 0140 00 00 00 00 00 00 00 00 00 00 b2 06 02 00 2e 80 ................ 0150 cd 7f 0d 02 b1 06 00 00 00 00 00 00 00 00 00 00 ................ 0160 00 00 65 b6 45 40 00 c0 8e b8 a1 54 83 bc 97 0d ..e.E@.....T.... 0170 bd c3 37 4b b6 42 d2 dc 83 c2 00 00 00 00 00 00 ..7K.B.......... 0180 00 00 00 00 00 00 c5 04 12 00 18 80 e4 7f f3 01 ................ 0190 c5 04 00 00 00 00 00 00 00 00 00 00 00 00 1d ad ................ 01a0 46 40 00 58 10 ba 01 86 79 bc 74 a6 bf c3 6e 24 F@.X....y.t...n$ 01b0 b6 42 0d 87 68 c2 00 00 00 00 00 00 00 00 00…

Eugeny · Answer

sisvpfis: 08 00 Can you please attach data in Wireshark format. What is the RX buffer size? If W5500 receives packet, and next packet arrives which has data larger than remaining RX buffer space, then this next packet will be discarded - you will not get it into the buffer because there’s simply no space. How does your code free the RX buffer? I do not see parsePacket() getting data out of there.

sisvpfis · Answer

Hi Eugeny, What do you mean by Wireshark format? *.pcapng? I didn’t change the size right now but already expermiented with that - no effect. As far as I know the basic size is 2kb per socket, 8 sockets, 16kb altogether. found this in the code: public: static const uint16_t SSIZE = 2048; // Max Tx buffer size private: static const uint16_t RSIZE = 2048; // Max Rx buffer size It seems tha even not all of the packages that i send manually are recognised by the controller, yet Wireshark sees them. What can I do to give you a better insight? Want the code?

sisvpfis · Answer

The Data itself is processed by udp.read() but only if there is something in it. But all of this is standard Ethernet2 library: int EthernetUDP::read(unsigned char* buffer, size_t len) { if (_remaining > 0) { int got; if (_remaining <= len) { // data should fit in the buffer got = recv(_sock, buffer, _remaining); } else { // too much data for the buffer, // grab as much as will fit got = recv(_sock, buffer, len); } if (got > 0) { _remaining -= got; return got; } } // If we get here, there's no data available or recv failed return -1; } int16_t recv(SOCKET s, uint8_t *buf, int16_t len) { // Check how much data is available int16_t ret = w5500.getRXReceivedSize(s); if ( ret == 0 ) { // No data available. uint8_t status = w5500.readSnSR(s); if ( status == SnSR::LISTEN || status == SnSR::CLOSED || status == SnSR::CLOSE_WAIT ) { // The remote end has closed its side of the connection, so this is the eof state ret = 0; } else { // The connection is still up, but there's no data waiting to be read ret = -1; } } else if (ret > len) { ret = len; } if ( ret > 0 ) { w5500.recv_data_processing(s, buf, ret); w5500.execCmdSn(s, Sock_RECV); } return ret; } uint16_t W5500Class::getRXReceivedSize(SOCKET s) { uint16_t val=0,val1=0; do { val1 = readSnRX_RSR(s); if (val1 != 0) val = readSnRX_RSR(s); } while (val != val1); return val; } void W5500Class::recv_data_processing(SOCKET s, uint8_t *data, uint16_t len, uint8_t peek) { uint16_t ptr; ptr = readSnRX_RD(s); read_data(s, ptr, data, len); if (!peek) { ptr += len; writeSnRX_RD(s, ptr); } } void W5500Class::read_data(SOCKET s, volatile uint16_t src, volatile uint8_t *dst, uint16_t len) { uint8_t cntl_byte = (0x18+(s<<5)); read((uint16_t)src , cntl_byte, (uint8_t *)dst, len); } void W5500Class::execCmdSn(SOCKET s, SockCMD _cmd) { // Send command to socket writeSnCR(s, _cmd); // Wait for command to complete while (readSnCR(s)) ; } uint16_t W5500Class::read(uint16_t _addr, uint8_t _cb, uint8_t *_buf, uint16_t _len) {…

Eugeny · Answer

sisvpfis: w5500.recv_data_processing(s, buf, ret); sisvpfis: void W5500Class::recv_data_processing(SOCKET s, uint8_t *data, uint16_t len, uint8_t peek) Does your compilator allow omitting arguments? What is the value of peek when recv_data_processing() is called?

sisvpfis · Answer

You are right, the nr. of parameters from call and function don’t add up. Yet there’s no warning concerning that. Should I worry? The peek output just before the if statement is and always stays “0”. Which means the pointer+length is calculated and written in the register, right? Btw. peek is initialized in the header file as “0”.

Eugeny · Answer

sisvpfis: Yet there’s no warning concerning that. Should I worry? If it is normal under programming language specification then someone changing default value will make application inoperable. sisvpfis: Which means the pointer+length is calculated and written in the register, right? This was my question - is it always calculated and target register updated? Can you please ensure everything is defined explicitly (e.g. value of this peek).

sisvpfis · Answer

Hey Eugeny, as far as I kept on reading I found out, that it may be possible that I can’t pickup the contents of the buffer fast enough to ensure to get every packet. Despite the ESP32 ist pretty fast compared to e.g. an AtMega32U4. But what i’m still wondering about is the fact that only these big packages seem to be the problem yet they are sent at the same rate!? So still struggling with this issue. The rest works really well for my purposes I think, if could sort out that size matter even If I didn’t catch every single packet it would still be great. therefore I was experimenting with the sending frequency of the data… (can be set to 10/20/30/60Hz). Sending only in 10Hz didn’t help either - the big ones are still gone. “peek” as i said is defined as “uint8_t peek = 0” in the header. I found no other operation that is altering peek whatsoever. Concering the parameters given for recv_data_processing function I investigated if there’s any other function to call with the right ammount of arguments… couldn’t find anything else but the already shown funtionality.

Eugeny · Answer

sisvpfis: But what i’m still wondering about is the fact that only these big packages seem to be the problem yet they are sent at the same rate!? If what you say is the case (must prove!) then the speed of buffer pointer advancement matters. And this speed may depend on many factors - your algorithm, time for ISR etc. How to prove? Speed up the communication by NOT reading data from the buffer. Just advance pointer, logging that you have got a packet. This way you will not spend time on reading packet data. sisvpfis: “peek” as i said is defined as “uint8_t peek = 0” in the header. I found no other operation that is altering peek whatsoever. This `peek is an argument to the function. Can you explicitly set to pass 0 every time you call function, or remove the condition to always advance the pointer? sisvpfis: therefore I was experimenting with the sending frequency of the data… Try increasing socket 0 RX buffer size to max 16KB.

sisvpfis · Answer

Hi Eugeny, I just solved my problem… But not by fixing the Ethernet2 lib. I did a fast test with the ethernet_generic lib an it worked instantly. At 30Hz send rate i receive 28.4Hz which ist far more than i expected! Thanks for all the effort you put Info this for me! @everyone else having Problems with receiving bigger packets… Switch to ethernet_generic lib! Regards! S.