Quantcast

need help mixing on the fly

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

need help mixing on the fly

dave

For a couple years I've been trying, without success, to reliably mix two
audio streams on the fly in C.  I'm using libao, libmodplug,
libsamplerate, libsndfile, and libvorbis.  The program is Frotz
(https://github.com/DavidGriffith/frotz), a Z-machine emulator for playing
old Infocom games as well as newer ones.  Could I please get some help
with this?


--
David Griffith
[hidden email]
_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: need help mixing on the fly

Fons Adriaensen-3
On Sat, Feb 18, 2017 at 01:44:32AM +0000, David Griffith wrote:
 
> For a couple years I've been trying, without success, to reliably
> mix two audio streams on the fly in C.  I'm using libao, libmodplug,
> libsamplerate, libsndfile, and libvorbis.  The program is Frotz
> (https://github.com/DavidGriffith/frotz), a Z-machine emulator for
> playing old Infocom games as well as newer ones.  Could I please get
> some help with this?

Mixing audio from two audio streams just requires adding the streams
sample by sample. I'm pretty sure that the real problem here is not
mixing but something else. But with the limited info you provide we
can only guess.

Where are the streams coming from ? What are the formats ? Where is
the result supposed to go ? etc. etc. etc.


Ciao,

--
FA

A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: need help mixing on the fly

Julius Smith
In reply to this post by dave
Hi David,

Attached is the output from

  faust -a sndfile.cpp tmix2.dsp > tmix2.cpp

for the Faust program (tmix2.dsp):

  // two-channel audio mixer:
  gain1 = hslider("Gain1",0.5,0,1,0.001);
  gain2 = hslider("Gain2",0.5,0,1,0.001);
  process = _, _ : *(gain1), *(gain2) :> _;

(See tmix2.cpp attached)

Cheers,
- Julius


On Fri, Feb 17, 2017 at 5:44 PM, David Griffith <[hidden email]> wrote:

For a couple years I've been trying, without success, to reliably mix two audio streams on the fly in C.  I'm using libao, libmodplug, libsamplerate, libsndfile, and libvorbis.  The program is Frotz (https://github.com/DavidGriffith/frotz), a Z-machine emulator for playing old Infocom games as well as newer ones.  Could I please get some help with this?


--
David Griffith
[hidden email]
_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev



--

Julius O. Smith III <[hidden email]>
Professor of Music and, by courtesy, Electrical Engineering
CCRMA, Stanford University
http://ccrma.stanford.edu/~jos/ 


_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev

tmix2.cpp (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: need help mixing on the fly

dave
In reply to this post by Fons Adriaensen-3
On Sat, 18 Feb 2017, Fons Adriaensen wrote:

> On Sat, Feb 18, 2017 at 01:44:32AM +0000, David Griffith wrote:
>
>> For a couple years I've been trying, without success, to reliably
>> mix two audio streams on the fly in C.  I'm using libao, libmodplug,
>> libsamplerate, libsndfile, and libvorbis.  The program is Frotz
>> (https://github.com/DavidGriffith/frotz), a Z-machine emulator for
>> playing old Infocom games as well as newer ones.  Could I please get
>> some help with this?
>
> Mixing audio from two audio streams just requires adding the streams
> sample by sample. I'm pretty sure that the real problem here is not
> mixing but something else. But with the limited info you provide we
> can only guess.
>
> Where are the streams coming from ? What are the formats ? Where is
> the result supposed to go ? etc. etc. etc.

The audio comes from AIFF, OGGV, and MOD files which are embedded in an
IFF container file which also contains game data.  Using libmodplug,
libsamplerate, libsndfile, and libvorbis; these audio chunks are turned
into stereo streams of floats.  There are only two streams.  If two
streams are active, mixing is done.  The resulting single stream of floats
is converted to pcm16 and fed into libao for output.

The two streams are either "music" or "bleep".  The music stream can be
fed with audio data from an OGGV chunk or MOD chunk, but not both.  The
bleep stream is fed from an AIFF chunk.  If a new stream of the same type
is started, the new one immediately takes over the old one.  This works.
If both types are played at once, they're supposed to be mixed.  That
process doesn't go right and I don't know where or why, though I suspect
trouble with threads, mutexes, and/or semaphores.  The problem manifests
in distorted sound and usually a segfault.

More detail...

A mixer thread is spawned when Frotz starts up.  It waits for one or both
float buffers to fill whereupon it mixes their contents, converts the
result to pcm16 and calls libao to play it.  playaiff() or playmusic() are
spawned by the main thread as separate threads to read audio data from the
container IFF and fill up the float buffers for the mixer to read.

The defective code is in
https://github.com/DavidGriffith/frotz/blob/ao-curses/src/curses/ux_audio.c

I have created a test game to be loaded by Frotz for testing audio
development.  It's at http://661.org/soundtest2.blb.  The game is a text
adventure that simulates a single room which contains a Commodore 64, a
Commodore Amiga, and a small box with buttons on it.  Turning on the C64
causes an OGGV file to play.  The Amiga plays a MOD file.  The
buttons on the small box cause AIFFs to play.  To tickle the bug, turn on
one of the computers and then press a button on the box.

I'm not interested in using SDL or Pulseaudio for this project.


--
David Griffith
[hidden email]

A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: need help mixing on the fly

dave
On Sun, 19 Feb 2017, David Griffith wrote:

> I have created a test game to be loaded by Frotz for testing audio
> development.

Oops.  I forgot to add an URL to the test game:
http://661.org/soundtest2.blb


--
David Griffith
[hidden email]

A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: need help mixing on the fly

Fons Adriaensen-3
In reply to this post by dave
On Sun, Feb 19, 2017 at 01:47:39PM +0000, David Griffith wrote:

> A mixer thread is spawned when Frotz starts up.  It waits for one or
> both float buffers to fill whereupon it mixes their contents,
> converts the result to pcm16 and calls libao to play it.  playaiff()
> or playmusic() are spawned by the main thread as separate threads to
> read audio data from the container IFF and fill up the float buffers
> for the mixer to read.

/me looks at code...

This is all so horribly wrong that I don't know where to be
begin. Honestly, I'm surprised it works at all (assuming it
does, with one source).

But the main error seems to be the way the audio_full and
audio_empty semas are used. If there are two producers,
audio_full is posted twice. And then the two producers
are waiting on the same sema, audio_empty. This can't
work ever. It's not some small bug -- the entire logic
and structure of this code is just wrong.

Apart from that, if this code is meant to be Linux only,
consider using ALSA or Jack. If not, consider something
like Portaudio. OSS and everything built on it have been
deprecated for at least 12 years now.

Ciao,

--
FA

A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)

_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: need help mixing on the fly

dave
On Sun, 19 Feb 2017, Fons Adriaensen wrote:

> On Sun, Feb 19, 2017 at 01:47:39PM +0000, David Griffith wrote:
>
>> A mixer thread is spawned when Frotz starts up.  It waits for one or
>> both float buffers to fill whereupon it mixes their contents,
>> converts the result to pcm16 and calls libao to play it.  playaiff()
>> or playmusic() are spawned by the main thread as separate threads to
>> read audio data from the container IFF and fill up the float buffers
>> for the mixer to read.
>
> /me looks at code...
>
> This is all so horribly wrong that I don't know where to be
> begin. Honestly, I'm surprised it works at all (assuming it
> does, with one source).
>
> But the main error seems to be the way the audio_full and
> audio_empty semas are used. If there are two producers,
> audio_full is posted twice. And then the two producers
> are waiting on the same sema, audio_empty. This can't
> work ever. It's not some small bug -- the entire logic
> and structure of this code is just wrong.
>
> Apart from that, if this code is meant to be Linux only,
> consider using ALSA or Jack. If not, consider something
> like Portaudio. OSS and everything built on it have been
> deprecated for at least 12 years now.

I admit that I don't have a good handle on the way the producer/consumer
setup works.  I got it with one producer and one consumer.  With two
producers, I got lost.

The code is meant to be portable across at least Linux and BSD.  That's
why I picked libao for output.

--
David Griffith
[hidden email]

A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
_______________________________________________
Linux-audio-dev mailing list
[hidden email]
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Loading...