Latency issues are common in embedded software. They are caused by complex, hidden parts
of the software stack. They usually become apparent late in the development process. By nature, they are costly and tricky to fix.
This was not an issue as long as embedded applications remained simple. Most applications were simply not affected by these issues as they were tolerant to latencies in the tens or hundreds of milliseconds. And when latencies mattered, the engineer in charge of the project knew the whole code base and could spend time tweaking timings with precise code optimizations.
This is no longer the case. Modern real-time applications (think machine vision, deep learning) are now entering life-critical embedded systems. They run on powerful multi-core processors and use deep software stacks totaling several millions of lines of code. The specifications require latencies below the millisecond. The complexity at play makes it imperative to use software designed for real-time systems.
Let's analyze what latency is, what are its causes and how we can minimize it.