Audio/Midi system - RT prios..

classic Classic list List threaded Threaded
42 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Audio/Midi system - RT prios..

Florian Paul Schmidt-2

Hi,

i was wondering:

With the new shiny -rt kernels and realtime scheduling available to non
root users via the usual mechanisms, there's the possibility of really
finetuning an audio/midi system.

The main issue i am interested in is the interplay between midi and
audio in such a system. How to tune the audio side to get a very
reliable system is pretty easy these days, thanks to the great jack
audio connection kit, alsa and the new -rt kernels.

But now i wonder how midi software fits into this. I'm here interested
in the special case of a software sequencer (like i.e. Rosegarden)
driving a softsynth (like i.e. om-synth or supercollider3) or whatever.

Ok, on a normal audio tuned -rt equipped linux system the SCHED_FIFO
priorities which are used for the different components look something
like this:

99 - system timer
98 - RTC

81 - soundcard IRQ handler
80 - jack watchdog
70 - jack main loop
69 - jack clients' process loops

50 - the other IRQ handlers

Now, i wonder how midi threads would fit in best into this scheme. Let's
assume our midi sequencer uses either sleep() or RTC to get woken up at
regular intervals, and let's further assume that it properly deals with
these timing sources to get relatively jitter free midi output given
that it get's woken up often enough by the scheduler. I further assume
that the alsa seq event system is used and midi events are not queued
for future delivery but always delivered immediately.

All this implies that for midi delivery timing to not be influenced by
audio processing on the system (which gets a problem especially at large
buffer size, where quite a bit of work is done at a time), all the stuff
that handles midi should run with realtime priorities above the jack
stuff (i.e. around 90). I wonder whether it also needs to have a higher
priority than the soundcard irq handler, too. Does the jackd main loop
"inherit" the priority of the soundcard irq handler?

Anyways, one more thing to note is for this to work nicely, the
softsynth needs to have an extra midi handling thread that is also
running with a priority in the 90s range, so it can timestamp the event
properly when it arrives.

So i wonder now: Assuming our system is setup as described above and all
midi handling is done from threads with sufficiently high pririties not
to get disturbed by audio stuff, will the alsa event system play nice?

I ask this, because i have setup a system as above with a simple midi
generator (see code below) and some different softsynths (one of which i
have written which does have its midi thread at an appropriate priority.
you can get a tarball here.

http://affenbande.org/~tapas/ughsynth-0.0.3.tgz

Beware it eats unbelievable amounts of cpu and is in no way considered
being finished. it just lay around handy for this test ;)). But i still
get some regular jitter in my sound.

Here's recorded example output (running jackd at a periodsize of 1024
and the test notes are produced at a frequency of 8hz). First with
ughsynth then with jack-dssi-host hexter.so. The effect is less
prominent with hexter, i suppose because the jack load with it is only
at 2 or 3% as opposed to ughsynth that uses 50% here on my athlon 1.2
ghz box. In case you don't hear what i mean: The timing of every ca. 7th
or 8th note is a little bit off.

http://affenbande.org/~tapas/midi_timing.ogg

So i wonder: what's going wrong? Is the priorities setup described above
not correct? Is alsa seq handling somehow not done with RT priority?
What else could be wrong? Please enlighten me :)

And yeah, i do _not_ want to hear about jack midi. It's a good thing,
and i'm all for it as it will make at least some scenarios work great
(sequencer and softsynth both being jack midi clients), but not all.

Thanks in advance,
Flo

midi_timer.cc:

#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include <cstdlib>
#include <iomanip>

#include <pthread.h>
#include <linux/rtc.h>
#include <sys/ioctl.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
#include <poll.h>
#include <signal.h>
#include <time.h>

#include <alsa/asoundlib.h>

#define RTC_FREQ  2048.0
#define NOTE_FREQ    8.0
#define RT_PRIO     85

int main()
{
        int fd;

        fd = open("/dev/rtc", O_RDONLY);

        if (fd ==  -1) {
                perror("/dev/rtc");
                exit(errno);
        }

        int retval = ioctl(fd, RTC_IRQP_SET, (int)RTC_FREQ);

        if (retval == -1) {
                perror("ioctl");
                exit(errno);
        }

        std::cout << "locking memory" << std::endl;
        mlockall(MCL_CURRENT);

        // std::cout << "sleeping 1 sec" << std::endl;
        // sleep(1);

        snd_seq_t *seq_handle;

        int err, port_no;

        err = snd_seq_open(&seq_handle, "default", SND_SEQ_OPEN_OUTPUT, 0);

        if (err < 0) {
                std::cout << "error" << std::endl;
                exit(0);
        }

        std::string port_name = "midi_timer";
        // set the name to something reasonable..

        err = snd_seq_set_client_name(seq_handle, port_name.c_str());

        if (err < 0) {
                std::cout << "error" << std::endl;
                exit(0);
        }

        // this is the port others can connect to. we don't do autoconnect ourself
        err = snd_seq_create_simple_port(seq_handle, "midi_timer:output", SND_SEQ_PORT_CAP_READ|SND_SEQ_PORT_CAP_SUBS_READ, SND_SEQ_PORT_TYPE_MIDI_GENERIC);

        if (err < 0) {
                std::cout << "error" << std::endl;
                exit(0);
        }

        // on success we know our port no
        port_no = err;

        struct sched_param param;
        int policy;

        pthread_getschedparam(pthread_self(), &policy, &param);
        param.sched_priority = RT_PRIO;
        policy = SCHED_FIFO;
        pthread_setschedparam(pthread_self(), policy, &param);
       

        std::cout << "turning irq on" << std::endl;
        retval = ioctl(fd, RTC_PIE_ON, 0);

        if (retval == -1) {
                perror("ioctl");
                exit(errno);
        }

        snd_seq_event_t ev;

        unsigned long data;
       
        int ticks_passed = 0;

        while(1) {
                // then we read it
                retval = read(fd, &data, sizeof(unsigned long));
                if (retval == -1) {
                        perror("read");
                        exit(errno);
                }
                if ((float)ticks_passed >= (RTC_FREQ/NOTE_FREQ)) {
                        // std::cout << "play note" << std::endl;
                        ticks_passed -= (long int)(RTC_FREQ/NOTE_FREQ);

                        // play note
                        snd_seq_ev_clear(&ev);
                        snd_seq_ev_set_direct(&ev);
                        snd_seq_ev_set_subs(&ev);
                        snd_seq_ev_set_source(&ev, port_no);
                        ev.type = SND_SEQ_EVENT_NOTEON;
                        ev.data.note.note = 53;
                        ev.data.note.velocity = 100;
                       
                        snd_seq_event_output_direct(seq_handle, &ev);

                        snd_seq_drain_output(seq_handle);
                }
                data = (data >> 8);
                // std::cout << data << std::endl;
                ticks_passed += data;
        }
        return 0;
}


--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
On Fri, 30 Dec 2005 00:47:46 +0100
Florian Schmidt <[hidden email]> wrote:

[snip]

Hmm,

forget this post :) I need to do some more testing first..

Flo

--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
On Fri, 30 Dec 2005 01:52:10 +0100
Florian Schmidt <[hidden email]> wrote:

> On Fri, 30 Dec 2005 00:47:46 +0100
> Florian Schmidt <[hidden email]> wrote:
>
> [snip]
>
> Hmm,
>
> forget this post :) I need to do some more testing first..

Ok,

my synth was buggy (damn copy and paste). Now it works like a charm with
a setup as described in my original post. To illustrate the difference a
proper priority setup can make, here's debug output of my ughsynth
driven by the RTC based midi_timer note generator (links for both
programs at bottom of the mail):

note on, frame_time: 56249918
next event: 574
diff: 6071
note on, frame_time: 56255989
next event: 501
diff: 6144
note on, frame_time: 56262133
next event: 501
diff: 6144
note on, frame_time: 56268277
next event: 501
diff: 6143
note on, frame_time: 56274420
next event: 500
diff: 5500
note on, frame_time: 56279920
next event: 880
diff: 6000
note on, frame_time: 56285920
next event: 736
diff: 6000
note on, frame_time: 56291920
next event: 592
diff: 6054
note on, frame_time: 56297974
next event: 502
diff: 6145
note on, frame_time: 56304119
next event: 503
diff: 6142
note on, frame_time: 56310261
next event: 501
diff: 6144
note on, frame_time: 56316405
next event: 501
diff: 5517
note on, frame_time: 56321922
next event: 898
diff: 6001
note on, frame_time: 56327923
next event: 755
diff: 6000
note on, frame_time: 56333923
next event: 611
diff: 6035
note on, frame_time: 56339958
next event: 502
diff: 6144
note on, frame_time: 56346102
next event: 502
diff: 6143
note on, frame_time: 56352245
next event: 501
diff: 6144

The interesting number is the "diff" output as it tells us the
difference of the previous midi event timestamp to the current one.The
"next" field is the offset into the currently to-be-processed period. In
above output the midi handling thread of ughsynth ran with a priority of
59, which is below the jackd stuff in my system (-P 70).

Here's output with the midi handling in ughsynth running at a priority
of 95:

note on, frame_time: 71319937
next event: 385
diff: 6000
note on, frame_time: 71325937
next event: 241
diff: 6000
note on, frame_time: 71331937
next event: 97
diff: 6002
note on, frame_time: 71337939
next event: 979
diff: 6000
note on, frame_time: 71343939
next event: 835
diff: 6000
note on, frame_time: 71349939
next event: 691
diff: 6000
note on, frame_time: 71355939
next event: 547
diff: 6000
note on, frame_time: 71361939
next event: 403
diff: 6001
note on, frame_time: 71367940
next event: 260
diff: 6000
note on, frame_time: 71373940
next event: 116
diff: 6001
note on, frame_time: 71379941
next event: 997
diff: 6000
note on, frame_time: 71385941
next event: 853
diff: 6001
note on, frame_time: 71391942
next event: 710
diff: 6000
note on, frame_time: 71397942
next event: 566
diff: 6000
note on, frame_time: 71403942
next event: 422
diff: 6000
note on, frame_time: 71409942
next event: 278
diff: 6001
note on, frame_time: 71415943
next event: 135
diff: 6000
note on, frame_time: 71421943
next event: 1015

The difference is either 5999, 6000 or 6001 frames, which at a framerate of
48000hz is tightly around

1/(6000/48000) = 8 hz

which is exactly what the midi note generator is setup to do. The
variance is several orders of magnitude lower than in the previous
example output above with midi handling prio of 59, which does make an
audible difference:

http://affenbande.org/~tapas/stable_timing.ogg

as opposed to:

http://affenbande.org/~tapas/unstable_timing.ogg

To summarize here's how a well tuned -rt system for combined midi/audio
usage should look like (prioritywise):

99 System timer IRQ (you cannot change this anyways)
98 RTC IRQ
       
95
.
. Midi handling threads of softsynths/midi sequencers (preferably
.
85
       
82 Soundcard IRQ
       
80 Jackd watchdog thread
70 Jackd main loop
69 Jackd client (softsynths/midi sequencers) audio process callbacks
       
60
.
. Other IRQ handlers (disk, network, USB, GFX)
.
40

0 (SCHED_OTHER) All other software in the system

Sadly not many app authors are aware of this (as my unsuccessful quest
to get stable midi timing with available linux software showed), so i
hope this post does raise the awareness on the issue. It would be ideal,
if app authors allowed the user to finetune the realtime priorities of
each component of their software (well, especially the midi handling
part, as the audio processing priorities are determined by what priority
jack is given).

Here's the software i used for the test

http://affenbande.org/~tapas/midi_timer.tgz

http://affenbande.org/~tapas/ughsynth-0.0.3.tgz

Regards,
Flo

P.S.: i also summarized the results a little bit on this page:

http://tapas.affenbande.org/?page_id=40

Please let me know if there's any big errors on that page.

P.P.S.: Additionally to RTC or sleep() based mechanism (which relies
right now on the system timer frequency (which is a mere 250hz in
default kernels (easy to change to 1000hz though with a recompile))) the
future will also bring us POSIX high resolution timers. I don't know yet
how these will fit in exactly.

--
Palimm Palimm! http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Jens M Andreasen
Flo!

Is it important for the midi thread priority to be above the soundcard
IRQ, or is it enough to be above jackd?

How will having several sound/midi cards fit into this scheme? (I have a
builtin VIA chipset sound, a better quality Aureal PCI card and, for
good measure, a USB control surface.)

/jens


PS: Letting BIOS set up the IRQs (instead of Linux plug&pray) solved
some USB related underruns for me.

Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Chris Cannam-2
In reply to this post by Florian Paul Schmidt-2
Florian Schmidt writes:
> I further assume that the alsa seq event system
> is used

This is true of Rosegarden,

> and midi events are not queued
> for future delivery but always delivered immediately.

but this isn't -- Rosegarden always queues events
from a non-RT thread and lets the ALSA sequencer
kernel layer deliver them.  (Thru events are delivered
directly, with potential additional latency because of
the lower priority used for the MIDI thread.)  In
principle this should mean that only the priority of
the receiving synth's MIDI thread is significant for
the timing of sequenced events.  We also have a
mechanism to compensate for gradual drift between
the MIDI timing source (kernel timers or RTC) and
soundcard clock, when synchronising to audio, by
adjusting the sequencer skew factor.  (This happens
to be similar to the mechanism for slaving to MTC,
which is handy.)

In my experience this is all a long way from
foolproof.  The most common problems for users
seem to be:

 - ALSA sequencer uses kernel timers by default and
of course they only run at 100 or 250Hz in many
kernels.

 - ALSA sequencer can sync to RTC, but the
associated module (snd-rtctimer) appears to hang
some kernels solid when loaded or used.  I don't have
much information about that, but I can probably find
out some more.

 - ALSA sequencer can sync to a soundcard clock,
but this induces jitter when used with JACK and has
caused confusion for users who find themselves
inadvertently sync'd to an unused soundcard (the
classic "first note plays, then nothing" symptom).

The biggest advantage of course is not having to run
an RT MIDI timing thread.  My impression is that this
aspect of MusE (which does that, I think) causes
as many configuration problems for its users as using
ALSA sequencer queue timers does for Rosegarden's.  

Any more thoughts on this?


Chris
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
In reply to this post by Jens M Andreasen
On Fri, 30 Dec 2005 16:03:44 +0100
Jens M Andreasen <[hidden email]> wrote:

> Flo!
>
> Is it important for the midi thread priority to be above the soundcard
> IRQ, or is it enough to be above jackd?

This is not 100% clear to me. I'd figure it should be above soundcard
irq, too, just to be safe. I don't know enough about the internals of
how and if priority is inherited by threads waiting for IRQ's.

The previous posts were also all about midi routing from different apps
on the same machine. I do not know how sending/receiving MIDI to
physical ports is handled. Maybe the soundcard IRQ does play a role
here. But a higher prio thread waiting for a lower prio IRQ will not
really cause any troubles from my understanding (it will just work).

> How will having several sound/midi cards fit into this scheme?

Well, to get several audio cards working you will need to get them to
work in a single jack server first, so the part about the jack priority
setup still holds.

Then again just put all midi stuff again at the higher prios. I don't
see much changing here.

Flo

--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Paul Davis
In reply to this post by Chris Cannam-2
> The biggest advantage of course is not having to run
> an RT MIDI timing thread.  My impression is that this
> aspect of MusE (which does that, I think) causes
> as many configuration problems for its users as using
> ALSA sequencer queue timers does for Rosegarden's.  
>
> Any more thoughts on this?

several people have wanted JACK to export a thread create call that
would take care of the RT-ness. that way, if you can run JACK with RT
scheduling, you can run a MIDI thread too, with no extra steps. it would
also be useful for people doing FFT in JACK clients using a separate
thread.

i don't agree with florian that the MIDI thread should run with higher
priority than the JACK watchdog, btw. i think the watchdog should be
higher than anything else until such a time as the kernel guarantees
"watchdog" functionality itself.

--p


Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
In reply to this post by Chris Cannam-2
On Fri, 30 Dec 2005 15:17:04 +0000 GMT
"Chris Cannam" <[hidden email]> wrote:

Hi Chris,

> > and midi events are not queued
> > for future delivery but always delivered immediately.
>
> but this isn't -- Rosegarden always queues events
> from a non-RT thread and lets the ALSA sequencer
> kernel layer deliver them.  (Thru events are delivered
> directly, with potential additional latency because of
> the lower priority used for the MIDI thread.)  In
> principle this should mean that only the priority of
> the receiving synth's MIDI thread is significant for
> the timing of sequenced events.  

Hi,

i tested Rosegarden running with the system timer as timing source (RTC
is a bit broken atm on -rt kernels for me), and i do not get
satisfactory results. I used my ughsynth again (which is heavy on the
cpu which makes the problems just clearer) with its midi thread at
priority 95.

Here's example output with rosegarden producing a supposedly steady
stream of 16th notes at 120 bpm:

note on, frame_time: 205200106
next event: 744
next event: 746
diff: 5998
note on, frame_time: 205206104
next event: 599
next event: 600
diff: 6042
note on, frame_time: 205212146
next event: 497
next event: 498
diff: 6157
note on, frame_time: 205218303
next event: 510
next event: 511
diff: 6140
note on, frame_time: 205224443
next event: 506
next event: 507
diff: 6145
note on, frame_time: 205230588
next event: 507
next event: 508
diff: 5511
note on, frame_time: 205236099
next event: 898
next event: 899
diff: 6000
note on, frame_time: 205242099
next event: 754
next event: 755
diff: 5998
note on, frame_time: 205248097
next event: 608
next event: 609
diff: 6034
note on, frame_time: 205254131
next event: 498
next event: 499
diff: 6153
note on, frame_time: 205260284
next event: 507
next event: 508
diff: 6141
note on, frame_time: 205266425
next event: 504
next event: 505
diff: 6148
note on, frame_time: 205272573
next event: 507
next event: 509
diff: 5521
note on, frame_time: 205278094
next event: 908
next event: 910
next event: 510

which is again in the range as with my test program and ughsynth having
a low midi thread prio. This is clearly audible, too:

http://affenbande.org/~tapas/rosegarden_ughsynth.ogg

this is the rosegardenfile used for this:

http://affenbande.org/~tapas/test16th.rg

This would imply to me, that either the way the events are scheduled in
rosegarden is buggy (unlikely as it works fine when there's less audio
load on the system) or that the event queue delivery by ALSA is somehow
happening with only SCHED_OTHER priority as well.

I have not yet found an option for ALSA to configure this.

>  - ALSA sequencer uses kernel timers by default and
> of course they only run at 100 or 250Hz in many
> kernels.

In my case i have compiled the kernel to use a system timer frequency of
1000hz. It would be interesting to know though what priority the ALSA
event queue handling gets.

I also stumbled across some problems with sleep() and especially waking
up when the sleep time has expired in the course of writing my
rt_watchdog program. Sometimes the high prio SCHED_FIFO thread wasn't
woken up as long as a lower SCHED_FIFO prio thread hugged the cpu even
when the sleep time of the high prio thread was long expired.. Ingo told
me back then that there's extra kernel threads for the timing subsystem
which need to be setup to high prios too for this to work correctly.
Haven't really investigated further into this.

I need to write another small test app that uses sleep based timing and
a high prio, too, to drive ughsynth. Will report what results i get.

>  - ALSA sequencer can sync to RTC, but the
> associated module (snd-rtctimer) appears to hang
> some kernels solid when loaded or used.  I don't have
> much information about that, but I can probably find
> out some more.

I have never bothered to try this either.

>  - ALSA sequencer can sync to a soundcard clock,
> but this induces jitter when used with JACK and has
> caused confusion for users who find themselves
> inadvertently sync'd to an unused soundcard (the
> classic "first note plays, then nothing" symptom).

> The biggest advantage of course is not having to run
> an RT MIDI timing thread.  My impression is that this
> aspect of MusE (which does that, I think) causes
> as many configuration problems for its users as using
> ALSA sequencer queue timers does for Rosegarden's.  
>
> Any more thoughts on this?

>From my point of view just setting up a RT midi thread driven by RTC and
with a sufficiently high prio for dispatching midi events immediately is
the best way. As it seems to work well, at least for my small test case.

Further testing needs to be done though. I will report back.

I haven't really tried muse. Will do so if i find the time though..

Flo

--
Palimm Palimm!
http://tapas.affenbande.org

Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
In reply to this post by Paul Davis
On Fri, 30 Dec 2005 10:41:46 -0500
Paul Davis <[hidden email]> wrote:

> several people have wanted JACK to export a thread create call that
> would take care of the RT-ness. that way, if you can run JACK with RT
> scheduling, you can run a MIDI thread too, with no extra steps. it would
> also be useful for people doing FFT in JACK clients using a separate
> thread.

actually, with realtime-lsm, there's really no need for this, except for
some convenience. Every app can create its own RT threads these days.
The 2.4.x capabilities days are (thank gawd) over :)

> i don't agree with florian that the MIDI thread should run with higher
> priority than the JACK watchdog, btw. i think the watchdog should be
> higher than anything else until such a time as the kernel guarantees
> "watchdog" functionality itself.

Agreed. why not make it prio 98 by default then? (system timer should
still be higher i suppose). With a difference of only 10 between main
jack loop and the watchdog, it might get a little crowded :)

Flo

--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

wschweer
In reply to this post by Chris Cannam-2
On Friday 30 December 2005 16:17, Chris Cannam wrote:

> Florian Schmidt writes:
> > I further assume that the alsa seq event system
> > is used
>
> This is true of Rosegarden,
>
> > and midi events are not queued
> > for future delivery but always delivered immediately.
>
> but this isn't -- Rosegarden always queues events
> from a non-RT thread and lets the ALSA sequencer
> kernel layer deliver them.  (Thru events are delivered
> directly, with potential additional latency because of
> the lower priority used for the MIDI thread.)  In
> principle this should mean that only the priority of
> the receiving synth's MIDI thread is significant for
> the timing of sequenced events.  We also have a
> mechanism to compensate for gradual drift between
> the MIDI timing source (kernel timers or RTC) and
> soundcard clock, when synchronising to audio, by
> adjusting the sequencer skew factor.  (This happens
> to be similar to the mechanism for slaving to MTC,
> which is handy.)
>
> In my experience this is all a long way from
> foolproof.  The most common problems for users
> seem to be:
>
>  - ALSA sequencer uses kernel timers by default and
> of course they only run at 100 or 250Hz in many
> kernels.
>
>  - ALSA sequencer can sync to RTC, but the
> associated module (snd-rtctimer) appears to hang
> some kernels solid when loaded or used.  I don't have
> much information about that, but I can probably find
> out some more.
>
>  - ALSA sequencer can sync to a soundcard clock,
> but this induces jitter when used with JACK and has
> caused confusion for users who find themselves
> inadvertently sync'd to an unused soundcard (the
> classic "first note plays, then nothing" symptom).
>
> The biggest advantage of course is not having to run
> an RT MIDI timing thread.  My impression is that this
> aspect of MusE (which does that, I think) causes
> as many configuration problems for its users as using
> ALSA sequencer queue timers does for Rosegarden's.
>
> Any more thoughts on this?

its right that MusE uses a RT midi thread to schedule midi
events. ALSA is used only to deliver (route) midi events.
I think this is the easiest possible solution and gives the app
the best control over timing.
Using the ALSA seq api means that ALSA has to operate the RT thread which
only moves the problems to ALSA.
The ALSA seq api is from ancient time were no realtime threads were
available in linux. Only a kernel driver could provide usable
midi timing. But with the introduction of RT threads the
ALSA seq api is obsolete IMHO.

Midi is synced to audio in MusE by using JACK frame timing to
schedule midi events which is also easy and straightforward.

There is nothing for a user to configure except he changes the
priority of the JACK RT thread.
The priority of the MusE midi RT thread has to be at least one above the
JACK RT priority. The point is that this allows the midi thread
to interrupt the JACK audio process thread which is necessary
to provide acceptable midi timing.

Last note about RT-linux kernels: its not _that_ important. Its
only a micro optimization. A normal recent kernel works pretty well.
If your normal kernel does not operate with sufficient low latencies,
the RT-kernel will most likely also not work.

/Werner



Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Paul Davis
In reply to this post by Florian Paul Schmidt-2
On Fri, 2005-12-30 at 17:17 +0100, Florian Schmidt wrote:

> On Fri, 30 Dec 2005 10:41:46 -0500
> Paul Davis <[hidden email]> wrote:
>
> > several people have wanted JACK to export a thread create call that
> > would take care of the RT-ness. that way, if you can run JACK with RT
> > scheduling, you can run a MIDI thread too, with no extra steps. it would
> > also be useful for people doing FFT in JACK clients using a separate
> > thread.
>
> actually, with realtime-lsm, there's really no need for this, except for
> some convenience. Every app can create its own RT threads these days.
> The 2.4.x capabilities days are (thank gawd) over :)

you don't know the correct priority to use. i imagine an api along the
lines of:

        jack_create_thread (pthread_t*, void* (thread_function)(void*),
                            void* arg, int relative_to_jack);

the last argument would specify that the thread should run at, above or
below the jack RT thread(s) by a given amount. typical values would be
+1, 0, -1 etc.

> > i don't agree with florian that the MIDI thread should run with higher
> > priority than the JACK watchdog, btw. i think the watchdog should be
> > higher than anything else until such a time as the kernel guarantees
> > "watchdog" functionality itself.
>
> Agreed. why not make it prio 98 by default then? (system timer should
> still be higher i suppose). With a difference of only 10 between main
> jack loop and the watchdog, it might get a little crowded :)

good point.


Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
In reply to this post by wschweer
On Fri, 30 Dec 2005 17:37:13 +0100
Werner Schweer <[hidden email]> wrote:


> its right that MusE uses a RT midi thread to schedule midi
> events. ALSA is used only to deliver (route) midi events.
> I think this is the easiest possible solution and gives the app
> the best control over timing.
> Using the ALSA seq api means that ALSA has to operate the RT thread which
> only moves the problems to ALSA.

This is my understanding also.

> The ALSA seq api is from ancient time were no realtime threads were
> available in linux. Only a kernel driver could provide usable
> midi timing. But with the introduction of RT threads the
> ALSA seq api is obsolete IMHO.

I wouldn't say obsolete, but IMHO RT thread based midi dispatching is
more easy to get right.

> Midi is synced to audio in MusE by using JACK frame timing to
> schedule midi events which is also easy and straightforward.

> There is nothing for a user to configure except he changes the
> priority of the JACK RT thread.
> The priority of the MusE midi RT thread has to be at least one above the
> JACK RT priority. The point is that this allows the midi thread
> to interrupt the JACK audio process thread which is necessary
> to provide acceptable midi timing.

Yep. I agree.

Is this "one above the JACK RT priority" automated in muse? Your first
sentence seems to imply otherwise. It's probably a reasonable approach
to do it this way. Although manual user overide would be nice, too.

> Last note about RT-linux kernels: its not _that_ important. Its
> only a micro optimization. A normal recent kernel works pretty well.
> If your normal kernel does not operate with sufficient low latencies,
> the RT-kernel will most likely also not work.

I do not agree. While this is true for an otherwise unloaded system, it
is rather easy (on a vanilla kernel) to produce xruns by putting other
load on the system.

The IRQ priorization provided by -rt kernels is extremely useful to
avoid these. It is _vital_ to run jackd with a priority higher than
those IRQ handlers not doing audio/midi stuff (network, disk, etc). The
soundcard IRQ handler must run with a high prio for this to work, too.
I'm not all too sure about whether it matters which of the two (jack or
soundcard irq) is higher though, as long as both are higher than other
irq handlers.

It is very useful to be able to do other stuff while audio/midi is
working uninterrupted. I got used to be able to compile a kernel
alongside running jackd with a periodsize of 32 or 16 frames :) which
means, i can play o.e. guitar while waiting for the damn compile to
finish.

Flo

P.S.: softsynths need to have their midi thread higher prio than jackd,
too, for the flawless midi timing to work. So i suppose it's time for
some bug reports to softsynth authors :)

--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
In reply to this post by Paul Davis
On Fri, 30 Dec 2005 11:54:56 -0500
Paul Davis <[hidden email]> wrote:

> you don't know the correct priority to use. i imagine an api along the
> lines of:

true.

>
> jack_create_thread (pthread_t*, void* (thread_function)(void*),
>                             void* arg, int relative_to_jack);
>
> the last argument would specify that the thread should run at, above or
> below the jack RT thread(s) by a given amount. typical values would be
> +1, 0, -1 etc.

Why not simply

/*
 * returns the priority (1-99) of the jack main loop (which is already one
 * above the clients' process() threads or 0 if not realtime. Clients having
 * a midi handling thread should create it with a priority at least one
 * above the return value of this function.
 */
int jack_get_rt_priority();

Then the app can decide itself about how to create the thread.

> > Agreed. why not make it prio 98 by default then? (system timer should
> > still be higher i suppose). With a difference of only 10 between main
> > jack loop and the watchdog, it might get a little crowded :)
>
> good point.

OTOH, i'm not really sure if 9 priority levels isn't enough. It seems
one above jacks main thread should be good enough for most midi purposes.

Flo

--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Florian Paul Schmidt-2
In reply to this post by Chris Cannam-2
On Fri, 30 Dec 2005 15:17:04 +0000 GMT
"Chris Cannam" <[hidden email]> wrote:

>  - ALSA sequencer can sync to RTC, but the
> associated module (snd-rtctimer) appears to hang
> some kernels solid when loaded or used.  I don't have
> much information about that, but I can probably find
> out some more.

Yeah, i got a nice and juicy BUG in it (see below). So this is what
kills rosegarden regularly here when run with RTC timing source. I'm not
sure this is ALSA's fault, though, might be the -rt kernel, too. But
nonetheless after this happens rosegarden is hung and syslog tells me
"rtc: lost some interrupts at 1024hz" over and over until infinity (or
reboot):

Dec 30 17:30:27 mango kernel: BUG at include/linux/timer.h:83!
Dec 30 17:30:27 mango kernel: ------------[ cut here ]------------
Dec 30 17:30:27 mango kernel: kernel BUG at include/linux/timer.h:83!
Dec 30 17:30:27 mango kernel: invalid operand: 0000 [#1]
Dec 30 17:30:27 mango kernel: PREEMPT
Dec 30 17:30:27 mango kernel: Modules linked in: snd_rtctimer snd_seq_dummy snd_seq_oss snd_seq_midi snd_seq_midi_event snd_seq realtime iptable_nat ipt_addrtype ipt_state iptable_filter agpgart snd_intel8x0 usb_storage scsi_mod ohci_hcd usbcore ipt_MASQUERADE ip_nat ip_tables ip_conntrack snd_ice1712 snd_ice17xx_ak4xxx snd_ak4xxx_adda snd_cs8427 snd_i2c snd_mpu401_uart bsd_comp ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt sis900 mii crc32 snd_cs46xx gameport snd_rawmidi snd_seq_device snd_ac97_codec snd_ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc
Dec 30 17:30:27 mango kernel: CPU:    0
Dec 30 17:30:27 mango kernel: EIP:    0060:[<c0238311>]    Not tainted VLI
Dec 30 17:30:27 mango kernel: EFLAGS: 00210296   (2.6.15-rc7-rt1)
Dec 30 17:30:27 mango kernel: EIP is at rtc_do_ioctl+0x9c1/0xa00
Dec 30 17:30:27 mango kernel: eax: 00000024   ebx: 00000001   ecx: 00200246   edx: 00000001
Dec 30 17:30:27 mango kernel: esi: 00200202   edi: 00821192   ebp: d680ea40   esp: e45e1d94
Dec 30 17:30:27 mango kernel: ds: 007b   es: 007b   ss: 0068   preempt: 00000001
Dec 30 17:30:27 mango kernel: Process rosegardenseque (pid: 6134, threadinfo=e45e0000 task=dcad58c0 stack_left=7520 worst_left=-1)
Dec 30 17:30:27 mango kernel: Stack: c02e8de1 c02ed6bf 00000053 22222222 22222222 22222222 22222222 22222222
Dec 30 17:30:27 mango kernel:        22222222 22222222 c0331b20 e45e0000 e45e0000 c013953a e45e0000 00000001
Dec 30 17:30:27 mango kernel:        00200246 00200246 00000000 d680ea40 c02dda53 c0331b20 00007005 f09f7e44
Dec 30 17:30:27 mango kernel: Call Trace:
Dec 30 17:30:27 mango kernel:  [<c013953a>] sub_preempt_count+0x1a/0x20 (56)
Dec 30 17:30:27 mango kernel:  [<c02dda53>] _spin_lock_irqsave+0x23/0x60 (28)
Dec 30 17:30:27 mango kernel:  [<f09f70a3>] rtctimer_start+0x43/0x70 [snd_rtctimer] (40)
Dec 30 17:30:27 mango kernel:  [<f081da09>] snd_timer_start1+0x89/0xa0 [snd_timer] (20)
Dec 30 17:30:27 mango kernel:  [<f081db2f>] snd_timer_start+0xaf/0xe0 [snd_timer] (16)
Dec 30 17:30:27 mango kernel:  [<f09e9f61>] snd_seq_timer_continue+0x41/0x70 [snd_seq] (36)
Dec 30 17:30:27 mango kernel:  [<f09e8664>] snd_seq_queue_process_event+0x144/0x160 [snd_seq] (16)
Dec 30 17:30:27 mango kernel:  [<f09e86d7>] snd_seq_control_queue+0x57/0xb0 [snd_seq] (32)
Dec 30 17:30:27 mango kernel:  [<f09e3cf1>] snd_seq_deliver_single_event+0x181/0x190 [snd_seq] (28)
Dec 30 17:30:27 mango kernel:  [<f09e3f02>] snd_seq_deliver_event+0x42/0xa0 [snd_seq] (52)
Dec 30 17:30:27 mango kernel:  [<f09e41a1>] snd_seq_client_enqueue_event+0x91/0x160 [snd_seq] (28)
Dec 30 17:30:27 mango kernel:  [<f09e444b>] snd_seq_write+0x16b/0x200 [snd_seq] (44)
Dec 30 17:30:27 mango kernel:  [<c0164eb5>] vfs_write+0xd5/0x1b0 (80)
Dec 30 17:30:27 mango kernel:  [<c016505b>] sys_write+0x4b/0x80 (36)
Dec 30 17:30:27 mango kernel:  [<c0102e71>] syscall_call+0x7/0xb (40)
Dec 30 17:30:27 mango kernel: ---------------------------
Dec 30 17:30:27 mango kernel: | preempt count: 00000001 ]
Dec 30 17:30:27 mango kernel: | 1-level deep critical section nesting:
Dec 30 17:30:27 mango kernel: ----------------------------------------
Dec 30 17:30:27 mango kernel: .. [<c013948a>] .... add_preempt_count+0x1a/0x20
Dec 30 17:30:27 mango kernel: .....[<00000000>] ..   ( <= _stext+0x3feffde0/0x60)
Dec 30 17:30:27 mango kernel:
Dec 30 17:30:27 mango kernel: ------------------------------
Dec 30 17:30:27 mango kernel: | showing all locks held by: |  (rosegardenseque/6134 [dcad58c0,   1]):
Dec 30 17:30:27 mango kernel: ------------------------------
Dec 30 17:30:27 mango kernel:
Dec 30 17:30:27 mango kernel: #001:             [e1d2d4e8] {&timer->lock}
Dec 30 17:30:27 mango kernel: ... acquired at:               snd_timer_start+0x8e/0xe0 [snd_timer]
Dec 30 17:30:27 mango kernel:


--
Palimm Palimm!
http://tapas.affenbande.org
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Chris Cannam-2
In reply to this post by Florian Paul Schmidt-2
Florian Schmidt writes:
> Here's example output with rosegarden producing a
> supposedly steady stream of 16th notes at 120 bpm:
> [...]

Those results certainly are pretty poor.  We do have
a very similar test in the Rosegarden tree (the
complainer test) but it doesn't stress the system
quite the way it seems your program does.

I'll have to review the sequencer API and look at
adding a separate RT MIDI thread as an alternative
(which should be straightforward enough).  The
rationale for using queued events is simple -- ALSA
provides the service, why duplicate it? -- but it's
probably true that we've already spent far more time
working around problems with it than we saved by not
duplicating it.  (Does anyone else use queued
sequencer events in earnest?)


Chris
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Chris Cannam-2
In reply to this post by Florian Paul Schmidt-2
Florian Schmidt:
> Yeah, i got a nice and juicy BUG in it (see below). So
> this is what kills rosegarden regularly here when
> run with RTC timing source.

That'll be the chap.  Mind you, I never saw the RTC-
based timer measure significantly better than the
system timer at 1000Hz.  Although your measurements
may vary, and it seems, probably would.


Chris
Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Fons Adriaensen
In reply to this post by Paul Davis
On Fri, Dec 30, 2005 at 11:54:56AM -0500, Paul Davis wrote:

> you don't know the correct priority to use. i imagine an api along the
> lines of:
>
> jack_create_thread (pthread_t*, void* (thread_function)(void*),
>                             void* arg, int relative_to_jack);
>
> the last argument would specify that the thread should run at, above or
> below the jack RT thread(s) by a given amount. typical values would be
> +1, 0, -1 etc.

- It's fairly easy to find out a JACK client's thread priority - some
  of my apps, e.g. Aeolus do this to set their thread priorities relative
  to it.

- It could be wise to express *all* (including JACK's) priorities relative
  to the maximum. That way things will still work when the kernel developers
  decide to revise their numbering scheme.

- Some apps may want to run as well without linking to libjack, so I'm not
  so sure that this is the right place for a RT-thread creation routine.
  Anyway, unless you want to use capabilities or work around completely
  broken things such as NPTL 0.60, creating a RT thread *is* quite simple.
  If you are using an application framework or toolkit, and it provides
  safe communication between RT and e.g. GUI threads, then it probably
  will have a call to do it anyway.


Best wishes to all !

--
FA

 


Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

wschweer
In reply to this post by Florian Paul Schmidt-2
On Friday 30 December 2005 18:06, Florian Schmidt wrote:
> On Fri, 30 Dec 2005 17:37:13 +0100
>
> Werner Schweer <[hidden email]> wrote:
...

>
> > Last note about RT-linux kernels: its not _that_ important. Its
> > only a micro optimization. A normal recent kernel works pretty well.
> > If your normal kernel does not operate with sufficient low latencies,
> > the RT-kernel will most likely also not work.
>
> I do not agree. While this is true for an otherwise unloaded system, it
> is rather easy (on a vanilla kernel) to produce xruns by putting other
> load on the system.
>
> The IRQ priorization provided by -rt kernels is extremely useful to
> avoid these. It is _vital_ to run jackd with a priority higher than
> those IRQ handlers not doing audio/midi stuff (network, disk, etc). The
> soundcard IRQ handler must run with a high prio for this to work, too.
> I'm not all too sure about whether it matters which of the two (jack or
> soundcard irq) is higher though, as long as both are higher than other
> irq handlers.

higher priority thread can interrupt lower priority threads. What do
you gain if the soundcard can interrupt the jack thread? I believe
it does not matter.
Interrupt routines on a well behaved system are using only some
microseconds so it should not matter at all for audio purposes.
Or do i miss something here?

>
> It is very useful to be able to do other stuff while audio/midi is
> working uninterrupted. I got used to be able to compile a kernel
> alongside running jackd with a periodsize of 32 or 16 frames :) which
> means, i can play o.e. guitar while waiting for the damn compile to
> finish.

32 or 16 frames is IMHO insanely low. Lets assume your keyboard is
only 3.5m away from the drummer, you are about 10msec out of sync, which
translates to about 256 frames. This works reliable on a vanilla kernel
whatever you are doing in the background.
An interesting question is what max. latencies are accepted for real live
situations?
I can comfortably play keyboard at 20msec latency.
Something really bad is a timing _jitter_ of midi events. For
some drumloops you can hear a jitter of 2ms or lower. Latencies are
not so important for me but low jitter is.

>
> Flo
>
> P.S.: softsynths need to have their midi thread higher prio than jackd,
> too, for the flawless midi timing to work. So i suppose it's time for
> some bug reports to softsynth authors :)

Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Lee Revell
In reply to this post by Florian Paul Schmidt-2
On Fri, 2005-12-30 at 18:25 +0100, Florian Schmidt wrote:

> On Fri, 30 Dec 2005 15:17:04 +0000 GMT
> "Chris Cannam" <[hidden email]> wrote:
>
> >  - ALSA sequencer can sync to RTC, but the
> > associated module (snd-rtctimer) appears to hang
> > some kernels solid when loaded or used.  I don't have
> > much information about that, but I can probably find
> > out some more.
>
> Yeah, i got a nice and juicy BUG in it (see below).

You had better report this to LKML like yesterday so 2.6.15 does not get
released with this bug!

Lee

Reply | Threaded
Open this post in threaded view
|

Re: Audio/Midi system - RT prios..

Lee Revell
In reply to this post by wschweer
On Fri, 2005-12-30 at 19:01 +0100, Werner Schweer wrote:
> higher priority thread can interrupt lower priority threads. What do
> you gain if the soundcard can interrupt the jack thread? I believe
> it does not matter.
> Interrupt routines on a well behaved system are using only some
> microseconds so it should not matter at all for audio purposes.
> Or do i miss something here?

I think you are right.  It's not the relative priority of IRQ threads
that matter (most IRQs run for far less than 1ms anyway and the
soundcard IRQ thread and JACK should never contend) but the
preemptibility of softirqs (and a few rude hardirqs like IDE) that makes
the biggest difference.  Long running softirqs can delay preemption by
~10ms with 2.6.14 OOTB.

Lee

123