Quantcast

QTractor: QThreads: Not realtime on Linux without root?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

QTractor: QThreads: Not realtime on Linux without root?

Tim-2
Hi list. Hope Rui gets this...
I have a puzzling technical question, involving QTractor.

I have been looking very closely at how Qt's class QThread works.

Actually, it doesn't. On Linux. As a normal user.

A long investigation into its source code and ultimately some test programs,
 shows that on Linux it ignores what priority you attempt to start it with,
 if running with normal user privileges.

In Qt's qtbase/src/corelib/thread/qthread_unix.cpp:

void QThread::start(Priority priority)
{
...
    pthread_attr_t attr;
    pthread_attr_init(&attr);
...
}

This code in Qt shows it initializes with default attributes which have
 scheduling 'other' and thus the range of priority levels is ZERO and thus
 QThread is lying about being at QThread::TimeCriticalPriority for example -
 it is NOT, it stored your requested level but did not actually set it.

I just finished examining the following programs using 'htop' plus a
 custom program to ask what each's scheduling policy is,
 using pthread_attr_getschedpolicy():

(With Jack running realtime etc. etc...)

a) MusE : Two threads running FIFO policy (policy = '1'). Good, expected.
    MusE directly uses pthreads, and set policies and levels.
    I assume since I'm in the audio group with good rt limits,
     some help is going on, 'cause I ain't runnin' as root.
b) QTractor : NO threads running FIFO (all policies = '0'). Bad? Unexpected?
    QTractor uses a few different QThreads with various priority settings    
     including one requesting TimeCriticalPriority. Seem's it's not happening?
c) Qt5Creator test program using QThread set to TimeCriticalPriority: Bad -
     thread has policy = '0'.

The Mixxx project found the exact same thing:
https://www.mixxx.org/wiki/doku.php/performance_improvements

"RJ discovered that Mixxx's requests for real-time priority on this thread are
 having no effect. Running as a regular user on Linux shows that the priority
 range is from 0 to 0, and as root from 1 to 99, but it is set to 1 (the
 lowest) by default. However, calling
 setPriority(QThread::TimeCriticalPriority) (while running as root)
 does result in priority escalation. "

A further look at the ticket shows they had to go with pthreads directly.

What are your thoughts on this? Am I doing something wrong?
Is QTractor really running realtime or not?

After seeing the source and fooling with this, I'm disliking QThread.

Before creating a QThread, I tried to elevate using seteuid, setegid
 and so on, and tried elevating to group 'audio'.
Permissions denied, of course.

Thanks.
Tim.
The MusE project.

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QTractor: QThreads: Not realtime on Linux without root?

Rui Nuno Capela
On 12/26/2016 12:58 AM, termtech wrote:

> Hi list. Hope Rui gets this...
> I have a puzzling technical question, involving QTractor.
>
> I have been looking very closely at how Qt's class QThread works.
>
> Actually, it doesn't. On Linux. As a normal user.
>
> A long investigation into its source code and ultimately some test programs,
>  shows that on Linux it ignores what priority you attempt to start it with,
>  if running with normal user privileges.
>
> In Qt's qtbase/src/corelib/thread/qthread_unix.cpp:
>
> void QThread::start(Priority priority)
> {
> ...
>     pthread_attr_t attr;
>     pthread_attr_init(&attr);
> ...
> }
>
> This code in Qt shows it initializes with default attributes which have
>  scheduling 'other' and thus the range of priority levels is ZERO and thus
>  QThread is lying about being at QThread::TimeCriticalPriority for example -
>  it is NOT, it stored your requested level but did not actually set it.
>
> I just finished examining the following programs using 'htop' plus a
>  custom program to ask what each's scheduling policy is,
>  using pthread_attr_getschedpolicy():
>
> (With Jack running realtime etc. etc...)
>
> a) MusE : Two threads running FIFO policy (policy = '1'). Good, expected.
>     MusE directly uses pthreads, and set policies and levels.
>     I assume since I'm in the audio group with good rt limits,
>      some help is going on, 'cause I ain't runnin' as root.
> b) QTractor : NO threads running FIFO (all policies = '0'). Bad? Unexpected?
>     QTractor uses a few different QThreads with various priority settings    
>      including one requesting TimeCriticalPriority. Seem's it's not happening?
> c) Qt5Creator test program using QThread set to TimeCriticalPriority: Bad -
>      thread has policy = '0'.
>
> The Mixxx project found the exact same thing:
> https://www.mixxx.org/wiki/doku.php/performance_improvements
>
> "RJ discovered that Mixxx's requests for real-time priority on this thread are
>  having no effect. Running as a regular user on Linux shows that the priority
>  range is from 0 to 0, and as root from 1 to 99, but it is set to 1 (the
>  lowest) by default. However, calling
>  setPriority(QThread::TimeCriticalPriority) (while running as root)
>  does result in priority escalation. "
>
> A further look at the ticket shows they had to go with pthreads directly.
>
> What are your thoughts on this? Am I doing something wrong?
> Is QTractor really running realtime or not?
>
> After seeing the source and fooling with this, I'm disliking QThread.
>
> Before creating a QThread, I tried to elevate using seteuid, setegid
>  and so on, and tried elevating to group 'audio'.
> Permissions denied, of course.
>
that is an interesting finding Tim. but maybe not one of big concern.

you see, none of the qtractor subclassed QThread's needs to be real-time
at all. the only instance on that premises is, of course, the jack
process one, which is a regular pthread for all the real, real-time
scheduling class (FF) and on started jackd -P rtprio minus 5 points as
usual for all jack clients.

and actually, the issue is clearly stated in Qt documention, see
http://doc.qt.io/qt-5/qthread.html#setPriority

byee
--
rncbc aka. Rui Nuno Capela


_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QTractor: QThreads: Not realtime on Linux without root?

Felipe Ferreri Tonello
In reply to this post by Tim-2
Hi Tim,

On 26/12/16 00:58, termtech wrote:

> Hi list. Hope Rui gets this...
> I have a puzzling technical question, involving QTractor.
>
> I have been looking very closely at how Qt's class QThread works.
>
> Actually, it doesn't. On Linux. As a normal user.
>
> A long investigation into its source code and ultimately some test programs,
>  shows that on Linux it ignores what priority you attempt to start it with,
>  if running with normal user privileges.
Yes, this is expected. See below.

>
> In Qt's qtbase/src/corelib/thread/qthread_unix.cpp:
>
> void QThread::start(Priority priority)
> {
> ...
>     pthread_attr_t attr;
>     pthread_attr_init(&attr);
> ...
> }
>
> This code in Qt shows it initializes with default attributes which have
>  scheduling 'other' and thus the range of priority levels is ZERO and thus
>  QThread is lying about being at QThread::TimeCriticalPriority for example -
>  it is NOT, it stored your requested level but did not actually set it.
>
> I just finished examining the following programs using 'htop' plus a
>  custom program to ask what each's scheduling policy is,
>  using pthread_attr_getschedpolicy():
>
> (With Jack running realtime etc. etc...)
>
> a) MusE : Two threads running FIFO policy (policy = '1'). Good, expected.
>     MusE directly uses pthreads, and set policies and levels.
>     I assume since I'm in the audio group with good rt limits,
>      some help is going on, 'cause I ain't runnin' as root.
> b) QTractor : NO threads running FIFO (all policies = '0'). Bad? Unexpected?
>     QTractor uses a few different QThreads with various priority settings    
>      including one requesting TimeCriticalPriority. Seem's it's not happening?
> c) Qt5Creator test program using QThread set to TimeCriticalPriority: Bad -
>      thread has policy = '0'.
>
> The Mixxx project found the exact same thing:
> https://www.mixxx.org/wiki/doku.php/performance_improvements
>
> "RJ discovered that Mixxx's requests for real-time priority on this thread are
>  having no effect. Running as a regular user on Linux shows that the priority
>  range is from 0 to 0, and as root from 1 to 99, but it is set to 1 (the
>  lowest) by default. However, calling
>  setPriority(QThread::TimeCriticalPriority) (while running as root)
>  does result in priority escalation. "
>
> A further look at the ticket shows they had to go with pthreads directly.
>
> What are your thoughts on this? Am I doing something wrong?
> Is QTractor really running realtime or not?
>
> After seeing the source and fooling with this, I'm disliking QThread.
>
> Before creating a QThread, I tried to elevate using seteuid, setegid
>  and so on, and tried elevating to group 'audio'.
> Permissions denied, of course.
>
This is not a Qt problem - it is a user permission problem.

If sched_setscheduler() returns -1, check if errno is set to EPERM. In
this case the user trying to perform this operation does not have
CAP_SYS_NICE[1] capability, which is *required*.

[1] http://man7.org/linux/man-pages/man7/capabilities.7.html

If you want this type of feature, set CAP_SYS_NICE to the group audio
that you are referring.

--
Felipe

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev

0x92698E6A.asc (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ?==?utf-8?q? ?==?utf-8?q? QTractor: QThreads: Not realtime on Linux without root?

R. Mattes
 
Am Dienstag, 03. Januar 2017 19:31 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
 

> If sched_setscheduler() returns -1, check if errno is set to EPERM. In
> this case the user trying to perform this operation does not have
> CAP_SYS_NICE[1] capability, which is *required*.
>
> [1] http://man7.org/linux/man-pages/man7/capabilities.7.html
>
> If you want this type of feature, set CAP_SYS_NICE to the group audio
> that you are referring.

??? How can I grant capabilities to a group? I thought capabilites where either given to
a user (via /etc/security/capability.conf) or to a binary (by means of setcap).

Cheers, Ralf Mattes


_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QTractor: QThreads: Not realtime on Linux without root?

Felipe Ferreri Tonello
Hi Ralf,

On 03/01/17 21:37, Ralf Mattes wrote:

>  
> Am Dienstag, 03. Januar 2017 19:31 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
>  
>
>> If sched_setscheduler() returns -1, check if errno is set to EPERM. In
>> this case the user trying to perform this operation does not have
>> CAP_SYS_NICE[1] capability, which is *required*.
>>
>> [1] http://man7.org/linux/man-pages/man7/capabilities.7.html
>>
>> If you want this type of feature, set CAP_SYS_NICE to the group audio
>> that you are referring.
>
> ??? How can I grant capabilities to a group? I thought capabilites where either given to
> a user (via /etc/security/capability.conf) or to a binary (by means of setcap).
AFAIK, pam_cap support users and groups.

--
Felipe

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev

0x92698E6A.asc (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QTractor: QThreads: Not realtime on Linux without root?

Felipe Ferreri Tonello
Hi Ralf,

On 11/01/17 12:52, Ralf Mattes wrote:

>  
> Am Mittwoch, 11. Januar 2017 13:21 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
>  
>> Hi Ralf,
>>
>> On 03/01/17 21:37, Ralf Mattes wrote:
>>>  
>>> Am Dienstag, 03. Januar 2017 19:31 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
>>>  
>>>
>>>> If sched_setscheduler() returns -1, check if errno is set to EPERM. In
>>>> this case the user trying to perform this operation does not have
>>>> CAP_SYS_NICE[1] capability, which is *required*.
>>>>
>>>> [1] http://man7.org/linux/man-pages/man7/capabilities.7.html
>>>>
>>>> If you want this type of feature, set CAP_SYS_NICE to the group audio
>>>> that you are referring.
>>>
>>> ??? How can I grant capabilities to a group? I thought capabilites where either given to
>>> a user (via /etc/security/capability.conf) or to a binary (by means of setcap).
>>
>> AFAIK, pam_cap support users and groups.
>
> Not according to my local manpages (pam_cap(8)  09/23/2011 and CAPABILITY.CONF(5) -- 09/23/2011).
> Do you have any y reference for your information?
I never tested, but try out based on this reference[1] paragraph 2.2.

Apparently there are two implementations for pam_cap. One supports the
other doesn't.

[1]
http://blog.sevagas.com/?Linux-security-using-a-limited-group-PAM-modules
[2] https://github.com/ekline/pamcap/blob/master/pam_capability.c
[3] https://github.com/pexip/os-libcap2/blob/master/pam_cap/pam_cap.c

Good luck.

--
Felipe

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev

0x92698E6A.asc (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ?==?utf-8?q? ?==?utf-8?q? QTractor: QThreads: Not realtime on Linux without root?

R. Mattes
 
Am Mittwoch, 11. Januar 2017 14:20 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
 

> Hi Ralf,
>
> On 11/01/17 12:52, Ralf Mattes wrote:
> >  
> > Am Mittwoch, 11. Januar 2017 13:21 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
> >  
> >> Hi Ralf,
> >>
> >> On 03/01/17 21:37, Ralf Mattes wrote:
> >>>  
> >>> Am Dienstag, 03. Januar 2017 19:31 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
> >>>  
> >>>
> >>>> If sched_setscheduler() returns -1, check if errno is set to EPERM. In
> >>>> this case the user trying to perform this operation does not have
> >>>> CAP_SYS_NICE[1] capability, which is *required*.
> >>>>
> >>>> [1] http://man7.org/linux/man-pages/man7/capabilities.7.html
> >>>>
> >>>> If you want this type of feature, set CAP_SYS_NICE to the group audio
> >>>> that you are referring.
> >>>
> >>> ??? How can I grant capabilities to a group? I thought capabilites where either given to
> >>> a user (via /etc/security/capability.conf) or to a binary (by means of setcap).
> >>
> >> AFAIK, pam_cap support users and groups.
> >
> > Not according to my local manpages (pam_cap(8)  09/23/2011 and CAPABILITY.CONF(5) -- 09/23/2011).
> > Do you have any y reference for your information?
>
> I never tested, but try out based on this reference[1] paragraph 2.2.

Yes, but that paragraph seems to be simply wrong. And the code you link to
in [3] clearly shows that.

> Apparently there are two implementations for pam_cap. One supports the
> other doesn't.

No. That's wrong. pam_cap doesn't support caps by group, your second link points
the pam_capability module. IIRC that was only ever available in OpenSuse. The git log
(single line ...) of that repository doesn't really make me want to integrate it into a seccurity
service.

Cheers, RalfD
 
 
 
 


_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QTractor: QThreads: Not realtime on Linux without root?

Felipe Ferreri Tonello
Hi Ralf,

On 11/01/17 14:25, Ralf Mattes wrote:

>  
> Am Mittwoch, 11. Januar 2017 14:20 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
>  
>> Hi Ralf,
>>
>> On 11/01/17 12:52, Ralf Mattes wrote:
>>>  
>>> Am Mittwoch, 11. Januar 2017 13:21 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
>>>  
>>>> Hi Ralf,
>>>>
>>>> On 03/01/17 21:37, Ralf Mattes wrote:
>>>>>  
>>>>> Am Dienstag, 03. Januar 2017 19:31 CET, Felipe Ferreri Tonello <[hidden email]> schrieb:
>>>>>  
>>>>>
>>>>>> If sched_setscheduler() returns -1, check if errno is set to EPERM. In
>>>>>> this case the user trying to perform this operation does not have
>>>>>> CAP_SYS_NICE[1] capability, which is *required*.
>>>>>>
>>>>>> [1] http://man7.org/linux/man-pages/man7/capabilities.7.html
>>>>>>
>>>>>> If you want this type of feature, set CAP_SYS_NICE to the group audio
>>>>>> that you are referring.
>>>>>
>>>>> ??? How can I grant capabilities to a group? I thought capabilites where either given to
>>>>> a user (via /etc/security/capability.conf) or to a binary (by means of setcap).
>>>>
>>>> AFAIK, pam_cap support users and groups.
>>>
>>> Not according to my local manpages (pam_cap(8)  09/23/2011 and CAPABILITY.CONF(5) -- 09/23/2011).
>>> Do you have any y reference for your information?
>>
>> I never tested, but try out based on this reference[1] paragraph 2.2.
>
> Yes, but that paragraph seems to be simply wrong. And the code you link to
> in [3] clearly shows that.
>
>> Apparently there are two implementations for pam_cap. One supports the
>> other doesn't.
>
> No. That's wrong. pam_cap doesn't support caps by group, your second link points
> the pam_capability module. IIRC that was only ever available in OpenSuse. The git log
> (single line ...) of that repository doesn't really make me want to integrate it into a seccurity
> service.
>
If that feature is really important for you, you can always patch
pam_cap.c from lipcap2.

It seems like a nice feature to have, IMO.

--
Felipe

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev

0x92698E6A.asc (7K) Download Attachment
Loading...