Wiznet makers

The cover image of this article was generated by Gemini.

1. Background: Two Modes of wiznet5k and the Challenge of the lwIP Path

The WIZnet driver for MicroPython (wiznet5k) operates in two main modes depending on the use case.

[Table 1] Comparison of wiznet5k Operation Modes

While the lwIP (MACRAW) mode is highly attractive due to its compatibility, there has been a consistent need to improve its memory efficiency and optimize the processing path compared to the Hardware Stack mode. Recently, a significant update (PR) addressing these issues has been in progress, and I would like to share the details.

2. PR #17447 Analysis: Improving Data Path to be 'Near Zero-Copy'

The PR #17447, proposed by GitHub user @greezybacon, focuses on making the data transfer process between WIZnet (MACRAW) and lwIP (pbuf) more efficient.

💡 Technical Definition:

Here, "Zero-Copy" does not necessarily mean a complete "0-copy" (zero CPU copies), but rather accurately describes an improvement that removes the existing intermediate frame buffer to reduce copy operations and memory waste (i.e., near zero-copy optimization).

2-1) Shortening the Data Path: Removing the "Intermediate (Static) Frame Buffer"

In the existing lwIP (MACRAW) path, unnecessary intermediate buffers existed:

• Chip RX → (Driver Temporary Buffer) → lwIP pbuf
• lwIP pbuf → (Driver Temporary Buffer) → Chip TX

PR #17447 streamlines this structure by skipping the intermediate step where possible:

• Improvement: Chip RX → lwIP pbuf (Direct Copy)

2-2) Increasing Memory Efficiency: Saving ~1.5KB RAM

By removing the fixed Ethernet frame buffer (approx. 1514 bytes), we can secure an additional ~1.5KB of available RAM, depending on the board and environment. In resource-constrained MCU environments, this amount of memory saving is significant and highly perceptible.

2-3) Improving Communication Stability: "Preventing Packet Loss on OOM"

A practical advantage of this PR lies not just in performance, but in stability.

• Before: The driver would read the RX packet from the chip first, then attempt to allocate an lwIP pbuf. If memory was insufficient, the packet had to be dropped.
• After: "If no pbuf is available, do not read from the chip."

This allows the packet to remain safely in the WIZnet chip's internal buffer until system memory becomes available. Consequently, we can expect a mitigation of unnecessary packet loss (drops) during traffic bursts or memory pressure situations.

3. Technical Dive: Implementing Zero-Copy Mechanism

3-1) New Files Added

1.drivers/wiznet/macraw.c

• wiznet_macraw_send(): Sends packets directly from pbuf (LwIP's packet buffer) to WIZnet chip
• wiznet_macraw_recv(): Receives packets directly from WIZnet chip into pbuf
• Key improvement: If memory is insufficient, packets remain on the chip for later retrieval (previously they were discarded)

2.drivers/wizent/w5100.h & w5500.h

• Chip-specific optimized pbuf transmission logic for W5100/W5100S and W5500
• Handles wrap-around in the chip's internal buffer (loops back to start when reaching buffer end)

3-2) Modified Files

• extmod/network_wiznet5k.c

Key changes:

• Removed 1514-byte static buffer (deleted uint8_t eth_frame[1514])
  • Memory savings!
• wiznet5k_netif_output() function changed

// Before: Copy to buffer → Send
pbuf_copy_partial(p, self->eth_frame, p->tot_len, 0);
wiznet5k_send_ethernet(self, p->tot_len, self->eth_frame);

// After: Send pbuf directly
wiznet_macraw_send(p, 0);

• wiznet5k_poll() function improved

// Before: Receive to buffer → Allocate pbuf → Copy
wiznet5k_recv_ethernet(self);  // Stores in eth_frame
pbuf_alloc(... );
pbuf_take(p, self->eth_frame, len);

// After: Receive directly into pbuf
wiznet_macraw_recv(&packet, 0);  // Receives directly to pbuf
// If no memory, break and retry later

4. Status: Registered in release-1.28.0 Milestone (In Progress)

PR #17447 has been reviewed by MicroPython maintainers and is currently registered in the release-1.28.0 milestone.

This can be interpreted as a positive signal that it is likely to be included in the next release. (However, final merging and timing depend on the progress of the PR.)

5. Technical Summary

The changes in this PR can be summarized as follows:

1. Removal of Intermediate Frame Buffer: Reduces or eliminates dependency on the fixed frame buffer (~1514B).
2. Utilization of pbuf Chain: Simplifies TX/RX paths for better efficiency.
3. Backpressure Mechanism: Instead of reading and dropping RX packets when pbuf allocation fails, it leaves them in the chip, improving stability.
4. Chip-Specific Helpers: Optimizes the TX ring buffer write path for specific chips (W5100/W5500, etc.).

6. Note: Constraints and Trade-offs

When adopting this technology, certain constraints should be considered:

• Support Scope: Support for some older chips or specific configurations might be adjusted.
• Optimization Assumptions: The MACRAW path optimization may rely on simplified assumptions (e.g., Single Socket usage) due to its structural design.
• Testing Environment: Performance benefits or stability differences may vary depending on the MCU and WIZnet chip combination used.

In conclusion, this PR should be viewed not as an unconditional performance boost, but as a "directional improvement to make the lwIP mode more practical in resource-constrained environments."

7. WIZnet's Response: Alignment with New Chip Support (PR #18035)

In line with these ecosystem changes, I am also proceeding with PR #18035 to officially integrate WIZnet's latest product lineup (W6300, W6100, W55RP20, etc.) into MicroPython.

The key point here is the "Order of Application and Strategy."

1. New Chip Support Structure (#18035) should be merged first to establish a stable foundation.
2. Then, this lwIP Optimization (#17447) can be applied naturally on top of it.

This approach ensures that not only the existing W5500 but also the new chips can benefit from the same optimizations. I am closely communicating with the community to coordinate this roadmap.

9. Conclusion

This update is a great example of synergy between external developers' optimization contributions and WIZnet's efforts to support new chips.

If both PRs are successfully finalized, WIZnet products are expected to become more stable and efficient Ethernet solutions in the MicroPython environment, even for systems with memory constraints.

• Zero-Copy Optimization (PR #17447): https://github.com/micropython/micropython/pull/17447
• WIZnet New Chip Support (PR #18035): https://github.com/micropython/micropython/pull/18035

❓ FAQ

Q1. I am running out of memory when using the lwIP mode in MicroPython. Is there a solution

A. Yes, the updated driver (PR #17447) addresses this issue. It significantly reduces memory usage by eliminating the 1.5KB static buffer and implementing a "Zero-Copy" mechanism.

Q2. Does the Zero-Copy optimization work on chips other than the W5500?

A. Currently, the optimization has been implemented primarily for the W5500 and W5100 series.