Memory, cache, and why your ISR is slow

2.1 The layout

The Teensy 4.1 has 1024 KB of on-chip RAM, and the way it is divided catches people out.

The first 512 KB is called RAM1, and it is tightly coupled memory — TCM. Tightly coupled memory is connected to the Cortex-M7 through dedicated buses that bypass the normal system interconnect entirely. Access is single-cycle, deterministic, and never contends with DMA or with anything else on the bus. RAM1 is split into two halves at compile time, in 32 KB blocks: ITCM, which holds executable code, and DTCM, which holds your ordinary variables, your stack, and anything you have not explicitly placed elsewhere.

The second 512 KB is called RAM2, or OCRAM — on-chip RAM. This sits on the normal AXI bus. It is still fast, but access goes through the data cache, and it can contend with DMA transfers. In the Teensyduino world, RAM2 is where malloc allocates from, and where anything you mark with the DMAMEM attribute is placed.

Then there is the 8 MB of QSPI flash. Your program lives there when the board is powered off, but at boot the Teensy copies the code into ITCM and runs it from there. This is why a Teensy 4.1 takes a fraction of a second to start: it is not just jumping to a reset vector, it is performing a copy. It is also why your available fast RAM shrinks as your program grows — every kilobyte of code claims a kilobyte of ITCM.

Optionally, you can solder PSRAM to the underside pads. That gives you a large, slower external memory, reachable as EXTMEM. For a robot arm you would use it for something like a long recorded trajectory, or a big lookup table, not for anything in the real-time path.

2.2 Why this matters for real-time motion control

The practical rule that follows: anything in your step-generation interrupt should live in tightly coupled memory.

The reason is cache behaviour. Code and data in RAM2, or in flash, go through the M7's 32 KB instruction cache and 32 KB data cache. Caches help average-case throughput and hurt worst-case latency. When the data your interrupt needs happens to be in cache, the access takes a cycle. When it is not — a cache miss — you stall for many cycles while the memory system fetches a whole cache line. In ordinary code you never notice. In an interrupt that has to fire every 50 microseconds and produce a step pulse with tight timing, an unpredictable multi-cycle stall shows up as jitter, and jitter in step timing shows up as audible noise and, at high speeds, as lost steps.

On the Teensy the default is already the right one. Ordinary global and static variables land in DTCM, and code lands in ITCM, unless you have said otherwise. You mostly need to know this so that you don't do the wrong thing — for example, don't put your step-position array in DMAMEM because you read somewhere that DMAMEM is for buffers. Put DMA buffers there, and keep the motion state in DTCM.

The second half of the rule concerns cache coherency. If you set up a DMA transfer to move data from a buffer in RAM2 out to a peripheral, the CPU may have written that data into the data cache but not yet flushed it out to the actual RAM the DMA engine reads. The result is that the DMA sends stale data. The Teensyduino core provides arm_dcache_flush_delete() and related functions for exactly this, and any DMA-based approach must call them at the right moments. Missing them is the usual cause of "my DMA works sometimes" on the Teensy 4.x.

2.3 Practical guidance

Write your firmware normally to begin with. Do not preemptively scatter memory attributes around. When you get to the point of measuring step-pulse jitter with a scope — and you should get to that point — you will know exactly which buffers to move and why. The defaults on this chip are good, and moving things around before you have measured is guesswork.

Do, however, keep an eye on the compile-time report Teensyduino prints. It tells you how much of RAM1 has gone to code, how much is left for variables, and how much RAM2 is in use. If ITCM usage creeps up towards the point where your variables no longer fit in DTCM, you will want to know before the linker tells you in a confusing way.