[Rtai] Question about isolating CPU's

[Felix Frey]

Hi

I have a dual core CPU and want to reserve one core for RTAI,
so I start Linux with "isolcpus=0"

cat /proc/rtai/hal shows:

** RTAI/x86:

CPU Frequency: 998053000 (Hz)
TIMER Frequency: 6238219 (Hz)
TIMER Latency: 0 (ns)
TIMER Setup: 160 (ns)

** Real-time IRQs used by RTAI:

#45 at ffffffffa0119050
#4352 at ffffffffa00b94a0
#4353 at ffffffffa00b8cc0
#4355 at ffffffff810dd8f0

** RTAI extension traps:

** RTAI SYSREQs in use: #1 #2

** MASK OF CPUs ISOLATED FOR RTAI: 0x1.

So far, everything is fine.

cat /proc/rtai/scheduler shows:

RTAI LXRT Real Time Task Scheduler.

Calibrated Time Base Frequency: 998053000 Hz
Calibrated user space latency: 2999 ns
Calibrated kernel space latency: 2999 ns
Calibrated oneshot timer setup_to_firing time: 160 ns

Number of RT CPUs in system: 2 (sized for 2)

Global heap: size = 2097152, used = 30592; <BSD>.
Kstack heap: size = 524288, used = 0; <BSD>.

Number of forced hard/soft/hard transitions: traps 0, syscalls 0

Priority Period(ns) FPU Sig State CPU Task HD/SF PID RT_TASK *
TIME
------------------------------------------------------------------------------
1000000000 0 Yes No 0x3 0:1 1 0 959
2 0 Yes No 0x3 0:1 2 1 1097
0 0 Yes No 0x1 0:1 3 1 1099
1000000050 0 Yes No 0x0 0:1 4 0 1105
TIMED

READY
1000000050 0 Yes No 0x0 1:2 5 0 1089
1 0 Yes No 0x3 1:2 6 1 1098
1000000050 0 Yes No 0x0 1:2 7 0 1103
TIMED


Now, I expect all rt-task to be on the same core.
So what means the CPU column?

Can anybody help with interpreting the table above?

Thanks,
Felix

[Paolo Mantegazza]

Isolating a CPU means just that no Linux task will be on that CPU.
Nonetheless even a standard Linux task can appear onto an isolated CPU
if sched_affinity is used to force it there.
Thus kernel worker threads can still be found on isolated CPUS.
Then, in the same way, it's up to you to force the tasks you want onto
any preferred CPU, whether isolated or not, using a single bit enabled
mask in rt_task_init_schmod (and its likes).
There have been a few RTAI versions in which it took the burden to force
all of Linux, irqs and task, off the isolated CPUs. I abandoned it
because if was unreliable on the Linux side. Sometime ending with RTAI
working impaired but with Linux stalled.
Therefore I've preferred to leave the burden to any user wanting to
force isolation even more.
Take into account that moving all Linux irqs off isolated CPUs can be
done, within plain Linux, by simply echoing appropriate irq affinity
masks, while for moving the tasks off a simple module should be written.
There is a README in RTAI explaining something more and Linux
Documentation has another document.

My way of working with what available is to: isolate CPUS, write a
simple script to move Linux itqs away my isolated CPUS, pu onto the
isolated CPUS the most demanding hard real time tasks (in term of
latency), place the other RTAI tasks on the non isolated CPUS.

Paolo


On 04/13/2016 03:14 PM, Felix Frey wrote:
> Hi
>
> I have a dual core CPU and want to reserve one core for RTAI,
> so I start Linux with "isolcpus=0"
>
> cat /proc/rtai/hal shows:
>
> ** RTAI/x86:
>
> CPU Frequency: 998053000 (Hz)
> TIMER Frequency: 6238219 (Hz)
> TIMER Latency: 0 (ns)
> TIMER Setup: 160 (ns)
>
> ** Real-time IRQs used by RTAI:
>
> #45 at ffffffffa0119050
> #4352 at ffffffffa00b94a0
> #4353 at ffffffffa00b8cc0
> #4355 at ffffffff810dd8f0
>
> ** RTAI extension traps:
>
> ** RTAI SYSREQs in use: #1 #2
>
> ** MASK OF CPUs ISOLATED FOR RTAI: 0x1.
>
> So far, everything is fine.
>
> cat /proc/rtai/scheduler shows:
>
> RTAI LXRT Real Time Task Scheduler.
>
> Calibrated Time Base Frequency: 998053000 Hz
> Calibrated user space latency: 2999 ns
> Calibrated kernel space latency: 2999 ns
> Calibrated oneshot timer setup_to_firing time: 160 ns
>
> Number of RT CPUs in system: 2 (sized for 2)
>
> Global heap: size = 2097152, used = 30592; <BSD>.
> Kstack heap: size = 524288, used = 0; <BSD>.
>
> Number of forced hard/soft/hard transitions: traps 0, syscalls 0
>
> Priority Period(ns) FPU Sig State CPU Task HD/SF PID RT_TASK *
> TIME
> ------------------------------------------------------------------------------
> 1000000000 0 Yes No 0x3 0:1 1 0 959
> 2 0 Yes No 0x3 0:1 2 1 1097
> 0 0 Yes No 0x1 0:1 3 1 1099
> 1000000050 0 Yes No 0x0 0:1 4 0 1105
> TIMED
>
> READY
> 1000000050 0 Yes No 0x0 1:2 5 0 1089
> 1 0 Yes No 0x3 1:2 6 1 1098
> 1000000050 0 Yes No 0x0 1:2 7 0 1103
> TIMED
>
>
> Now, I expect all rt-task to be on the same core.
> So what means the CPU column?
>
> Can anybody help with interpreting the table above?
>
> Thanks,
> Felix
> _______________________________________________
> Rtai mailing list
> ***@rtai.org
> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai

[Alec Ari]

Hi, what simple script are you referring to? Do you have an example?

Alec Ari

[Paolo Mantegazza]

________________________________________
From: Rtai [rtai-***@rtai.org] on behalf of Alec Ari [***@ymail.com]
Sent: Wednesday, April 13, 2016 7:47 PM
To: ***@rtai.org
Subject: Re: [Rtai] Question about isolating CPU's

Hi, what simple script are you referring to? Do you have an example?

Alec Ari

[Paolo Mantegazza]

Sorry, i clicked the wrong button.
I'm assuming you are referring to irq affinities.
At the moment I've none ready, I usually write one for a specific machine I'm working on.
It is simple and requires just these steps:
- "cat /proc/interrupts" to get the interrupts of interest
- writing for each of them "echo affinity_mask >/proc/irq/irq_number_of_interest/smp_affinity"
Usually my scripts are just a list of the above command and made executable, with the mask containing the bits of those cpus that are not isolated for RTAI.
That's all.

Paolo.

________________________________________
From: Rtai [rtai-***@rtai.org] on behalf of Alec Ari [***@ymail.com]
Sent: Wednesday, April 13, 2016 7:47 PM
To: ***@rtai.org
Subject: Re: [Rtai] Question about isolating CPU's

Hi, what simple script are you referring to? Do you have an example?

Alec Ari

[Emmanuel Pacaud]

Hi,

Le mer. 13 avril 2016 à 23:26, Paolo Mantegazza
<***@polimi.it> a écrit :
> Sorry, i clicked the wrong button.
> I'm assuming you are referring to irq affinities.
> At the moment I've none ready, I usually write one for a specific
> machine I'm working on.
> It is simple and requires just these steps:
> - "cat /proc/interrupts" to get the interrupts of interest
> - writing for each of them "echo affinity_mask
> >/proc/irq/irq_number_of_interest/smp_affinity"
> Usually my scripts are just a list of the above command and made
> executable, with the mask containing the bits of those cpus that are
> not isolated for RTAI.
> That's all.

On fedora/redhat, there is a irqbalance service that could help you
setup irq affinities.

The configuration file of this service is in /etc/sysconfig/irqbalance.
For example, if you want to isolate all the cpus except te first one,
it should have the following content:

# irqbalance is a daemon process that distributes interrupts across
# CPUS on SMP systems. The default is to rebalance once every 10
# seconds. This is the environment file that is specified to systemd
via the
# EnvironmentFile key in the service unit file (or via whatever method
the init
# system you're using has.
#
# ONESHOT=yes
# after starting, wait for a minute, then look at the interrupt
# load and balance it once; after balancing exit and do not change
# it again.
IRQBALANCE_ONESHOT=yes

#
# IRQBALANCE_BANNED_CPUS
# 64 bit bitmask which allows you to indicate which cpu's should
# be skipped when reblancing irqs. Cpu numbers which have their
# corresponding bits set to one in this mask will not have any
# irq's assigned to them on rebalance
#
IRQBALANCE_BANNED_CPUS=fe

#
# IRQBALANCE_ARGS
# append any args here to the irqbalance daemon as documented in the
man page
#
IRQBALANCE_ARGS=-d


Cheers,

Emmanuel.

[Doug Renton]

See... easy!
Too bad Fermat wasn't on a mailing list, it could have saved hundreds of
years!

On 04/13/2016 05:26 PM, Paolo Mantegazza wrote:
> Sorry, i clicked the wrong button.
> I'm assuming you are referring to irq affinities.
> At the moment I've none ready, I usually write one for a specific machine I'm working on.
> It is simple and requires just these steps:
> - "cat /proc/interrupts" to get the interrupts of interest
> - writing for each of them "echo affinity_mask >/proc/irq/irq_number_of_interest/smp_affinity"
> Usually my scripts are just a list of the above command and made executable, with the mask containing the bits of those cpus that are not isolated for RTAI.
> That's all.
>
> Paolo.
>
> ________________________________________
> From: Rtai [rtai-***@rtai.org] on behalf of Alec Ari [***@ymail.com]
> Sent: Wednesday, April 13, 2016 7:47 PM
> To: ***@rtai.org
> Subject: Re: [Rtai] Question about isolating CPU's
>
> Hi, what simple script are you referring to? Do you have an example?
>
> Alec Ari
> _______________________________________________
> Rtai mailing list
> ***@rtai.org
> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai
> _______________________________________________
> Rtai mailing list
> ***@rtai.org
> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai
>