Scatter‑Gather Data Aggregation Implemented With MicroPython DMA Chained Trigger
RP2040 scatter‑gather data aggregation implementation based on MicroPython DMA hardware chained‑trigger mechanism
In the code shown below, DMA‑chain triggering and trigger‑register‑write mechanisms implement concatenation of multiple strings into one memory buffer. This is a scatter‑gather data‑gather operation that assembles multiple fixed‑size memory blocks into a single memory space.
The source code can be found in the resource‑package path elegance‑devkit v1\Demo\69 DMA_ChainTrigger.
Sample code:
# Python env : MicroPython v1.23.0
# -*- coding: utf-8 -*-
# @Time : 2024/9/8 11:04 AM
# @Author : Li Qingshui
# @File : main.py
# @Description : DMA experiment, peripheral‑to‑memory transfer, DMA transfers ADC FIFO data to memory
# ======================================== Import related modules =========================================
# Import time‑related modules
import time
# Import addressof function for fetching object memory address
from uctypes import addressof
# Import DMA‑related modules
from rp2 import DMA
# Import array‑handling module
from array import array
# ======================================== Global variables ============================================
# Define list of input strings to be concatenated
input = ["FreakStudio", ":", "test", " of the scatter", " gather", " process"]
# Create 64‑byte destination bytearray buffer for concatenated result
output = bytearray(64)
# ======================================== Function definitions ============================================
# ======================================== Custom classes ============================================
def gather_strings(string_list: list, buf: bytearray) -> None:
"""
Concatenate string‑list content into destination buffer using DMA.
Implemented via cooperation of two DMA channels:
1. gather_dma channel: transfers string metadata (length + memory address)
2. buffer_dma channel: performs actual payload copy into destination buffer
Args:
string_list (list): List of MicroPython‑style input strings to concatenate
buf (bytearray): Pre‑allocated destination buffer with sufficient capacity
Returns:
None
Raises:
ValueError: Input argument type mismatch
RuntimeError: DMA transfer runtime error
"""
# Declare two DMA channel instances: gather_dma and buffer_dma
# gather_dma handles string length and address metadata
gather_dma = DMA()
# buffer_dma handles actual payload data copying
buffer_dma = DMA()
# Integer array storing string length and corresponding memory address
gather_list = array("I")
# Store sequentially: string length, string memory address
for s in string_list:
# Append string length
gather_list.append(len(s))
# Append memory address of string object
gather_list.append(addressof(s))
# Print gather_list content
# gather_list: array('I', [11, 536906720, 1, 536904296, 4, 536904298, 15, 536906752, 7, 536904303, 8, 536904311])
print("gather_list: ", gather_list)
# Append two zero values as list terminator, implementing Null Trigger (write all‑zero value to trigger register)
gather_list.append(0)
gather_list.append(0)
# Configure gather_dma control register, ring_size=3 for 8‑byte alignment, enable register ring‑buffer mode
gather_ctrl = gather_dma.pack_ctrl(ring_size=3, ring_sel=True)
# Configure gather_dma channel: write target points to TRANS_COUNT and READ_ADD_TRIG aliased registers (register index 14 and 15) of buffer_dma
gather_dma.config(
read=gather_list, # Read metadata from gather_list array
write=buffer_dma.registers[14:16], # Write metadata into buffer_dma aliased registers
count=2, # Transfer two 32‑bit items each iteration
ctrl=gather_ctrl # Apply DMA control‑register configuration
)
# Print unpacked gather_dma control‑register parameters:
# gather_ctrl: {'inc_read': 1, 'high_pri': 0, 'write_err': 0, 'ring_sel': 1,
# 'enable': 1, 'treq_sel': 63, 'sniff_en': 0, 'irq_quiet': 1, 'read_err': 0,
# 'chain_to': 0, 'busy': 0, 'inc_write': 1, 'ring_size': 3, 'bswap': 0, 'size': 2, 'ahb_err': 0}
print("gather_ctrl:",gather_dma.unpack_ctrl(gather_ctrl))
# Configure buffer_dma control register: 8‑byte single‑byte transfer, enable channel‑chaining
# After buffer_dma finishes transfer, it automatically triggers gather_dma for next metadata round
buffer_ctrl = buffer_dma.pack_ctrl(size=0, chain_to=gather_dma.channel)
# Set destination buffer buf as write target for buffer_dma, actual string payload will be written here
buffer_dma.config(write=buf, ctrl=buffer_ctrl)
# Start gather_dma DMA transfer
gather_dma.active(1)
# Calculate end physical address of gather_list
end_address = addressof(gather_list) + 4 * len(gather_list)
# Poll until gather_dma completes all metadata transfers
while gather_dma.read != end_address:
pass
# ======================================== Initialization ==========================================
# 3‑second power‑on delay
time.sleep(3)
# Print debug message indicating main program start
print("FreakStudio: DMA Chaining Triggers Test")
# ======================================== Main program ===========================================
# Print initial content of destination buffer
print(output)
# Use DMA to concatenate input string‑list into output buffer
gather_strings(input, output)
# Print final content of destination buffer
print(output)
The core function implementing scatter‑gather operation is gather_strings. It accepts two input arguments: string list for concatenation and pre‑allocated destination buffer.
Two DMA channel objects gather_dma and buffer_dma are created for separated tasks:
gather_dma: Delivers each string’s length and memory‑address metadata to buffer_dma.
buffer_dma: Executes actual payload copying and writes string content into destination buffer.
An integer array gather_list stores each string’s length and memory address; addressof(s) fetches memory address of string object s.
Then configure first DMA channel gather_dma. gather_dma.pack_ctrl builds DMA control block and enables ring‑buffer mode. Ring‑buffer configuration lets write‑address wrap‑around to starting offset after finishing one metadata pair transfer.
Parameter explanation:
ring_size=3: 8‑byte alignment (1<<3, 2^3 = 8). Each metadata pair (length + address) occupies exactly 8 bytes. ring_size=3 guarantees these two 32‑bit items are handled as one unit; every 8‑byte metadata block is written into two dedicated registers of buffer_dma.
ring_sel=True: Enable ring‑buffer mode applied only to write‑address. After gather_dma writes one metadata pair into buffer_dma registers, write‑address automatically wraps back to starting offset. This keeps writing to same pair of target registers without address increment.
Use this statement to inspect full control‑register configuration of gather_dma:
print("gather_ctrl:",gather_dma.unpack_ctrl(gather_ctrl))

inc_read=True makes read‑address advance to next metadata entry after each transfer, moving to next string’s metadata without rewinding. inc_write=True enables write‑address auto‑increment for payload transfer inside target buffer, so new string data appends instead of overwriting existing content. Meanwhile ring_size=3 forces write‑address wrap‑around for metadata writes beyond 8‑byte boundary.
gather_dma channel configuration items:
read=gather_list: DMA reads length‑and‑address metadata from gather_list.
write=buffer_dma.registers[14:16]: Write metadata into register index 14 and 15 aliased registers of buffer_dma (TRANS_COUNT and READ_ADD_TRIG trigger registers). Pass string length and source‑address into these registers; writing to READ_ADD_TRIG will immediately start buffer_dma channel transfer for current string payload.
count=2: Transfer two array items (length and address) per DMA iteration.
ctrl=gather_ctrl: Apply pre‑built ring‑mode DMA control‑block.
Configure second DMA channel buffer_dma via buffer_dma.pack_ctrl(size=0, chain_to=gather_dma.channel):
size=0: Set transfer granularity to single byte.
chain_to=gather_dma.channel: After buffer_dma finishes one payload transfer, automatically trigger gather_dma to supply next‑string metadata, forming hardware‑only chained workflow.
Configure buffer_dma write destination as target buffer buf and apply buffer_ctrl control‑block.
Detailed timing diagram for this DMA‑chaining workflow:
Runtime transfer sequence:
- Initial trigger: Activate
gather_dmachannel. It readslen1andaddr1metadata fromgather_list, writes intobuffer_dmaregisters. Writing to READ_ADDR_TRIG register firesbuffer_dma. - Metadata dispatch: After
gather_dmawrites metadata,buffer_dmais triggered. It copieslen1bytes from source addressaddr1into destination bufferbuf. - Chained trigger: On completion of
buffer_dmapayload transfer, it hardware‑triggersgather_dmaagain to fetch next‑string length‑and‑address metadata. Pollgather_dma.readvalue to detect end‑of‑metadata processing. - Iteration loop: Repeat above sequence until all entries inside
gather_listare fully processed.
Flash firmware and open serial terminal, output example:
As observed, after buffer_dma finishes one‑string payload transfer, it auto‑triggers gather_dma for next metadata entry. The whole workflow runs without CPU intervention on each individual transfer step and greatly improves throughput efficiency.
