Linux became a research vehicle very early on. Real-Time researchers set out to transform Linux to a Real-Time Operating system. These efforts were following two different approaches:

• Dual kernel approach: Utilizes a Nano- or a Micro-Kernel as RTOS kernel which runs the Linux Kernel in its idle task. The most prominent variants were: RT/Linux, RTAI, DROPS and later on Xenomai.

• In kernel real-time: Modifies the kernel code itself to gain Real-Time OS properties. The well known variants are KURT and Linux/RK

None of these projects was ever seriously attempting to be merged into the mainline kernel. RT/Linux, RTAI, Xenomai and Linux/RK were also deployed commercially with varying success and sustainability.

In autumn 2004 various patch series were posted from different interested parties, which provided basic technology to transform the Linux kernel into a Real-Time Operating System. None of these patches was providing a full solution.

These postings resulted in one of the largest mail discussions on LKML accompanied by flamewars and more or less pointless competitions to gain single digit microseconds in certain benchmarks.

Quite surprising for most kernel developers, Ingo Molnar, at the time maintainer of the kernel scheduler, picked up some of the patches or rewrote them mostly from scratch and combined them to a consistent set of patches aiming for in full kernel Real-Time support. Within no time a small team of core developers formed. Thomas Gleixner, who was working with the late Douglas Niehaus from Kansas University on a 2.6 based variant of the KURT realtime extension, brought in patches and technology from KURT. Steven Rostedt who was familiar with the Linux/RK work, brought in expertise and ideas from there. Various other developers joined the effort temporarily and helped to create the first halfways usable RT version in a relatively short time frame. The developers were aiming to integrate the RT work in the upstream kernel from the very beginning.

At the Linux kernel summit 2005 the general approach of integrating RT into the upstream kernel was discussed and most developers agreed with it under the premise that it is not disrupting the existing work and does not create road blocks for future development.

In the first years the RT work was mostly conducted by developers employed by RedHat and IBM due to a Raytheon/US Navy contract which was based on Linux Real-Time. After the contract was fulfilled, the interest in RT from IBM and RedHat subsided and the involved developers moved on to other areas of work.

During the first five years substantial parts of the Preempt-RT work found their way into the Linux Kernel:

• Generic Timekeeping
• High resolution timers
• Mutex infrastructure
• Generic interrupt handling infrastructure
• Priority inheritance for user space mutexes
• Preemptible and hierarchical RCU
• Threaded interrupt handlers
• Tracing infrastructure
• Lock dependency validator
• …

With some part-time funding from RedHat Preempt-RT was maintained and kept alive mostly by Thomas Gleixner and his team at Linutronix. The available funding for RT was not even sufficient to sustain maintenance. Despite substantial effort the expertise around Preempt-RT did not result in sustainable commercial success.

During this time Thomas Gleixner started to warn about the non sustainability of Preempt-RT at conferences and in personal conversations. Aside of hilarious proposals and whining nothing useful happened.

After RedHat cut the RT funding completely, Gleixner and his company linutronix decided to stop their engagement in Preempt-RT and Gleixner declared at ELCE 2014 in Dusseldorf that Preempt-RT is now a hobbyist project.

Shortly after the conference OSADL stepped up and provided full time funding for two engineers to keep Preempt-RT at least maintained and forwarded to newer kernel versions in 2015.

Preempt-RT maintenance work was resumed, but Gleixner made it entirely clear that if there would be no way forward towards proper mainline integration by the end of the year he'd be not interested to maintain RT out of tree forever.

In 2015, the Linux Foundation started an effort, led by Kate Stewart, to create a Real-Time Linux Collaborative Project under the umbrella of the Linux Foundation. At ELCE 2015 in Dublin, the project was officially announced and the kick-off meeting took place. The funding allows five engineers to work full time on the project with the clear target of merging the Preempt-RT patches into the mainline kernel.

In the first two years hundreds of patches emerging from the Preempt-RT team around Thomas Gleixner at Linutronix were integrated into the Linux kernel. Most of this work was preparatory and refactoring existing upstream code in a way to make the integration of Preempt-RT easier. The main achievements were:

• Rewrite of the CPU hotplug infrastructure
• Refactoring of the timer wheel
• Refactoring of high resolution timers
• Various other bits and pieces

While this work makes RT integration and maintenance easier it is not solely RT specific and it is valuable for the quality and performance of the Linux kernel on its own.

This effort reduced the size of the RT patch set and led to the more interesting and hard to solve RT specific problems. While the scope of these pending changes is not as huge as the already merged work, it's more difficult to integrate it cleanly and justify the merge into the upstream kernel.

The Meltdown/Spectre security disaster slowed development down a bit for about two months at the end of 2017 and early 2018 as Preempt-RT team lead Thomas Gleixner was heavily involved in the development of the mitigations for the Linux kernel. The RT team is working hard to make up for the time lost and come back on the estimated project time line.

As of March 2018 the project is in an interesting situation. The funding for 2018 is not clear yet, which means that eventually work force has to be reduced and the expected time lines have to be adapted. There is lots of effort going on to remedy that situation, so it hopefully will get back on track soon.

The RT team is really grateful for the support we received so far from the project members and from the Linux Foundation. We are aware that the project members take responsibility for the RT using industry as a whole and it would be certainly a good chance now for bystanders to join the effort and help along to secure the technology on which their products depend on. RT is one of the most ambitious Linux technology projects and to some extent the most disruptive technology which aimed for mainline integration. So far we succeeded to keep the impact on mainline at a minimum and we hope that with funding secured for the time after mainline integration this can be kept up to everyone's satisfaction: RT users, users of regular Linux and the upstream Linux maintainers and developers.

As most of you know 2018 has started with a big bang in the computer industry, which will keep us occupied for quite some time. The upstream kernel mitigation of meltdown variant 3 via https://lwn.net/Articles/742693/PTI kept me beyond busy for the last two month.

Despite that disruption, which definitely required all brains collaborating, I've tried to keep up with the ongoing work of merging the Preempt-RT patch set into the Linux kernel. We've made great progress on various topics, especially the jinxed dcache problem. Patches will be posted as soon as folks are back from their well earned vacation.

In 2017 the RTL developer team was making enormous progress and we've upstreamed an impressive 1.5 patches a day on average. The deviation from the time line which we predicted end of 2015 is still on a relatively small margin of less than 20 percent.

We'd like to express our gratitude to the RTL project Members who provided the funding for the past two years, so we were able to work with five engineers full time on interesting, challenging and sometimes mind boggling problems.

As things stand for 2018 the funding will be reduced, so we expect to reduce the number of developers accordingly to something between three and four. That will have significant impact on the time line and postpone the final merge of RT by a not yet determined amount of time. I can't tell how much at the moment as this would be pure speculation (pun intended).

Nevertheless, we're starting into 2018 with vigor and the vision of making as much progress towards mainlining RT as possible.

Thomas & RTL developer team