In figure (a), with no prefetching, each cache miss reference suffers a memory latency delay L, thus disrupting the normal running of the program (R).
In figure (b) a prefetch instruction has been issued at a distance D (D > L) ahead of the reference being made and the latency delay is hidden.
In figure (c) D < L and the processor stalls for a time S = L - D
Clearly, the prefetch distance affects the amount of stall time experienced by the program.
Intuitively, one would expect that the larger the prefetch distance, the better the cache performance. However, studies by both Tullsen et al. [11] and Saavedra et al. [9] suggest that the ideal prefetch distance (for optimal processor performance) should be close to, or slightly less than the memory latency.
The reason is that when D > L, as in figure (b), larger values of (D-L) mean that the prefetched data sits in the cache for longer before it is used, and the more likely it is that it will be displaced by other data, or that it will itself replace other data that would otherwise have enjoyed a hit, thus generating another cache (replacement) miss.