How to combine microphone audio and music from application to pipe to video conference

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

How to combine microphone audio and music from application to pipe to video conference

Samir Parikh
Hello,

I am trying to combine the audio from my microphone (either built in
microphone from my laptop or bluetooth headset) with music playing
from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
the audio input to video conference services such as Jitsi.

I don't know much about Linux audio internals and wasn't sure how to
do this.  Can I do this completely via the operating system using
Pulse Audio?  Do I need to do something with Jack?  Or do I need
specialized software such as OBS, Ardour or Reaper?

Any help or suggestions on pointing me in the right direction or how
to get started would be greatly appreciated.  I'm hoping that I can do
this completely through software without external hardware such as a
mixer or something.

Thanks in advance.

Samir
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

David W. Jones
Hi, Samir!

On 5/15/20 5:40 PM, Samir Parikh wrote:
> Hello,
>
> I am trying to combine the audio from my microphone (either built in
> microphone from my laptop or bluetooth headset) with music playing
> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
> the audio input to video conference services such as Jitsi.

My experience with Jitsi is that its audio input is limited to two
sources: Whatever microphone you have connected to your computer, and
your chosen YouTube URL (if you use Jitsi's option to stream a YouTube
URL). It lets you stream a YouTube URL, but that seems to take over the
Jitsi screen. I think at that point maybe you could talk in your
microphone and have it audible over the YouTube stream... but I don't
know, I haven't tried it.

I tried to find a way to stream just audio from my music library here
(mplayer playing through JACK) but couldn't find a way to get Jitsi to
take JACK as an input. The only options I saw were hardware ones.

I don't use Pulseaudio, maybe it could be set up to get around Jitsi's
idea of "hardware"?

> I don't know much about Linux audio internals and wasn't sure how to
> do this.  Can I do this completely via the operating system using
> Pulse Audio?  Do I need to do something with Jack?  Or do I need
> specialized software such as OBS, Ardour or Reaper?
>
> Any help or suggestions on pointing me in the right direction or how
> to get started would be greatly appreciated.  I'm hoping that I can do
> this completely through software without external hardware such as a
> mixer or something.
>
Maybe play your music to a channel connected to an external mixer,
connect your regular microphone to a different channel on the mixer,
then connect that output to a microphone in on your compute?

I don't know, just guessing. Hope it helps!

--
David W. Jones
[hidden email]
authenticity, honesty, community
http://dancingtreefrog.com
"My password is the last 8 digits of π."
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Len Ovens
In reply to this post by Samir Parikh
On Fri, 15 May 2020, Samir Parikh wrote:

> I am trying to combine the audio from my microphone (either built in
> microphone from my laptop or bluetooth headset) with music playing
> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
> the audio input to video conference services such as Jitsi.
>
> I don't know much about Linux audio internals and wasn't sure how to
> do this.  Can I do this completely via the operating system using
> Pulse Audio?  Do I need to do something with Jack?  Or do I need
> specialized software such as OBS, Ardour or Reaper?

OBS is probably the easiest way. You can add your local video if needed as
well. OBS will work fine with just pulse or jack. I don't know much about
jitsi it self but if it uses pulse as the default device there should be
no problem.


--
Len Ovens
www.ovenwerks.net
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

David W. Jones
On 5/15/20 7:25 PM, Len Ovens wrote:

> On Fri, 15 May 2020, Samir Parikh wrote:
>
>> I am trying to combine the audio from my microphone (either built in
>> microphone from my laptop or bluetooth headset) with music playing
>> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
>> the audio input to video conference services such as Jitsi.
>>
>> I don't know much about Linux audio internals and wasn't sure how to
>> do this.  Can I do this completely via the operating system using
>> Pulse Audio?  Do I need to do something with Jack?  Or do I need
>> specialized software such as OBS, Ardour or Reaper?
>
> OBS is probably the easiest way. You can add your local video if
> needed as well. OBS will work fine with just pulse or jack. I don't
> know much about jitsi it self but if it uses pulse as the default
> device there should be no problem.

I just checked the Jitsi Meet using Chromium (they recommend
Chrome/Chromium, apparently Jitsi uses some newer streaming standards
that Firefox either doesn't fully support or is just simply too slow).
For microphone, it offers my USB headset, USB audio, Builtin and
Pulseaudo JACK source. Which I remember I do have Pulseaudio set up here
feeding to JACK sink. Output include the preceding plus PA Jack sink.

I think that means you could have either your streaming audio going into
it, or your microphone, but not both.

--
David W. Jones
[hidden email]
authenticity, honesty, community
http://dancingtreefrog.com
"My password is the last 8 digits of π."
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

David W. Jones
On 5/15/20 7:56 PM, david wrote:
On 5/15/20 7:25 PM, Len Ovens wrote:
On Fri, 15 May 2020, Samir Parikh wrote:

I am trying to combine the audio from my microphone (either built in
microphone from my laptop or bluetooth headset) with music playing
from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
the audio input to video conference services such as Jitsi.

I don't know much about Linux audio internals and wasn't sure how to
do this.  Can I do this completely via the operating system using
Pulse Audio?  Do I need to do something with Jack?  Or do I need
specialized software such as OBS, Ardour or Reaper?

OBS is probably the easiest way. You can add your local video if needed as well. OBS will work fine with just pulse or jack. I don't know much about jitsi it self but if it uses pulse as the default device there should be no problem.

I just checked the Jitsi Meet using Chromium (they recommend Chrome/Chromium, apparently Jitsi uses some newer streaming standards that Firefox either doesn't fully support or is just simply too slow). For microphone, it offers my USB headset, USB audio, Builtin and Pulseaudo JACK source. Which I remember I do have Pulseaudio set up here feeding to JACK sink. Output include the preceding plus PA Jack sink.

I think that means you could have either your streaming audio going into it, or your microphone, but not both.

FWIW, I just tried using the PA Jack Source as my "microphone" input. Not a sound, even with music playing into it. So, good luck. If you can do live streaming on YouTube, I suppose you could live stream your audio that way and play the YT URL in Jitsi.

-- 
David W. Jones
[hidden email]
authenticity, honesty, community
http://dancingtreefrog.com
"My password is the last 8 digits of π."

_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Len Ovens
In reply to this post by David W. Jones
On Fri, 15 May 2020, david wrote:

> On 5/15/20 7:25 PM, Len Ovens wrote:
>> On Fri, 15 May 2020, Samir Parikh wrote:
>>
>>> I am trying to combine the audio from my microphone (either built in
>>> microphone from my laptop or bluetooth headset) with music playing
>>> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
>>> the audio input to video conference services such as Jitsi.
>>>
>>> I don't know much about Linux audio internals and wasn't sure how to
>>> do this.  Can I do this completely via the operating system using
>>> Pulse Audio?  Do I need to do something with Jack?  Or do I need
>>> specialized software such as OBS, Ardour or Reaper?
>>
>> OBS is probably the easiest way. You can add your local video if
>> needed as well. OBS will work fine with just pulse or jack. I don't
>> know much about jitsi it self but if it uses pulse as the default
>> device there should be no problem.
>
> I just checked the Jitsi Meet using Chromium (they recommend
> Chrome/Chromium, apparently Jitsi uses some newer streaming standards
> that Firefox either doesn't fully support or is just simply too slow).
> For microphone, it offers my USB headset, USB audio, Builtin and
> Pulseaudo JACK source. Which I remember I do have Pulseaudio set up here
> feeding to JACK sink. Output include the preceding plus PA Jack sink.
You are probably using the module-jackdbus-detect in pulse. that by loads
the two modules jack source and jack sink. However you can add as many of
those as you want. Also, it seems pulse can see other devices but using
zita-ajbridge they can become jack devices and mixed there. Then all
become part of jack source.



--
Len Ovens
www.ovenwerks.net

_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Stuart Longland
In reply to this post by Samir Parikh
On 16/5/20 1:40 pm, Samir Parikh wrote:
> Hello,
>
> I am trying to combine the audio from my microphone (either built in
> microphone from my laptop or bluetooth headset) with music playing
> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
> the audio input to video conference services such as Jitsi.

I actually had a similar requirement to pipe audio between Zoom and a
SIP telephone client.  I have no idea how to link Bluetooth into it, my
headset is a USB one (Logitech G930).

I'll be doing the exact same thing tomorrow night as my radio club
(Brisbane Area WICEN Group) will be meeting online via Zoom, and one of
the members in our club does not have a computer or "smartphone" at his
home.

The approach I did was this:

1. Installed JACK2 and qjackctl… pointed those at hw:Headset operating
at 16kHz sample rate 16-bit audio (microphone on the headset is limited
to this)

2. Configured ~/.asoundrc with the ALSA JACK plug-in:

> pcm.!default {
>         type plug
>         slave { pcm "jack" }
> }
>
> pcm.jack {
>         type jack
>         playback_ports {
>                 0 system:playback_1
>                 1 system:playback_2
>         }
>         capture_ports {
>                 0 system:capture_1
>                 1 system:capture_1
>         }
> }

3. Twinkle (the SIP client) was pointed directly ALSA to the "jack"
interface

4. Via PulseAudio, Zoom was pointed at JACK for audio

In qjackctl, I then patched Twinkle and Zoom together after establishing
the call… that effectively bridged the person on the telephone to the
conference.

I haven't tried with RhythmBox, but did get Clementine to successfully
connect to JACK and pipe audio to both the SIP client and to the
conference.  I'd look in RhythmBox for a way to direct it to JACK
directly instead of going through PulseAudio.

Jitsi also works with the above set-up: prior to the first test Zoom
"meeting" (which was more of a social get-together) I did try a test
with Jitsi, and was able to link a SIP desk phone (which was "standing
in" for the PSTN user) to Jitsi through the same technique.

I didn't try a telephone link-up with Slack, however I also use the same
headset + JACK set-up with a work teleconference every weekday without
issues.  I see no reason why it wouldn't work there too.

For Bluetooth headsets, you'd need some way of exposing the headset to
ALSA directly so JACK can access it.  Normally the BlueZ Bluetooth stack
links up to PulseAudio.  Also bear in mind, unless there's some
successor to the HSP profile, you'll be limited to 8kHz 8-bit mono
audio, which will sound pretty bloody terrible!

Yes, there's A2DP, but from what I've seen, most headsets only do A2DP
in one direction: usually for listening to music, when a call comes in
they switch to HSP for the telephone call.  I'd be very happy to be
proven wrong on this, but Bluetooth has brought nothing but
disappointment for me.

Based on this, you might be better served investing in a wireless USB
headset.

Regards,
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Samir Parikh
In reply to this post by David W. Jones
david wrote on 5/16/20 2:07 AM:

> On 5/15/20 7:56 PM, david wrote:
>> On 5/15/20 7:25 PM, Len Ovens wrote:
>>> On Fri, 15 May 2020, Samir Parikh wrote:
>>>
>>>> I am trying to combine the audio from my microphone (either built in
>>>> microphone from my laptop or bluetooth headset) with music playing
>>>> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
>>>> the audio input to video conference services such as Jitsi.
>>>>
>>>> I don't know much about Linux audio internals and wasn't sure how to
>>>> do this.  Can I do this completely via the operating system using
>>>> Pulse Audio?  Do I need to do something with Jack?  Or do I need
>>>> specialized software such as OBS, Ardour or Reaper?
>>>
>>> OBS is probably the easiest way. You can add your local video if
>>> needed as well. OBS will work fine with just pulse or jack. I don't
>>> know much about jitsi it self but if it uses pulse as the default
>>> device there should be no problem.
>>
>> I just checked the Jitsi Meet using Chromium (they recommend
>> Chrome/Chromium, apparently Jitsi uses some newer streaming standards
>> that Firefox either doesn't fully support or is just simply too slow).
>> For microphone, it offers my USB headset, USB audio, Builtin and
>> Pulseaudo JACK source. Which I remember I do have Pulseaudio set up
>> here feeding to JACK sink. Output include the preceding plus PA Jack
>> sink.
>>
>> I think that means you could have either your streaming audio going
>> into it, or your microphone, but not both.
>>
> FWIW, I just tried using the PA Jack Source as my "microphone" input.
> Not a sound, even with music playing into it. So, good luck. If you can
> do live streaming on YouTube, I suppose you could live stream your audio
> that way and play the YT URL in Jitsi.
>

Hi David,

First off, thank you so much for your replies to my post and apologies
for the delay in getting back to you!

I know your tried a few things using Jitsi Meet that don't look like
they will do what I am looking for but out of curiosity, how did you get
all of your audio sources to appear as the "microphone" input?  I'm not
familiar with PulseAudio Jack Source so that may be part of my issue.
For me, when using Jitsi Meet in Chrome, the only audio sources I see
are these:  https://imgur.com/a/wkAS0Al

Thanks again for your time on this.

Samir
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Samir Parikh
In reply to this post by Stuart Longland
On 5/16/20 11:05 PM, Stuart Longland wrote:

> On 16/5/20 1:40 pm, Samir Parikh wrote:
>> Hello,
>>
>> I am trying to combine the audio from my microphone (either built in
>> microphone from my laptop or bluetooth headset) with music playing
>> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
>> the audio input to video conference services such as Jitsi.
>
> I actually had a similar requirement to pipe audio between Zoom and a
> SIP telephone client.  I have no idea how to link Bluetooth into it, my
> headset is a USB one (Logitech G930).
>
> I'll be doing the exact same thing tomorrow night as my radio club
> (Brisbane Area WICEN Group) will be meeting online via Zoom, and one of
> the members in our club does not have a computer or "smartphone" at his
> home.
>
> The approach I did was this:
>
> 1. Installed JACK2 and qjackctl… pointed those at hw:Headset operating
> at 16kHz sample rate 16-bit audio (microphone on the headset is limited
> to this)
>
> 2. Configured ~/.asoundrc with the ALSA JACK plug-in:
>> pcm.!default {
>>          type plug
>>          slave { pcm "jack" }
>> }
>>
>> pcm.jack {
>>          type jack
>>          playback_ports {
>>                  0 system:playback_1
>>                  1 system:playback_2
>>          }
>>          capture_ports {
>>                  0 system:capture_1
>>                  1 system:capture_1
>>          }
>> }
>
> 3. Twinkle (the SIP client) was pointed directly ALSA to the "jack"
> interface
>
> 4. Via PulseAudio, Zoom was pointed at JACK for audio
>
> In qjackctl, I then patched Twinkle and Zoom together after establishing
> the call… that effectively bridged the person on the telephone to the
> conference.
>
> I haven't tried with RhythmBox, but did get Clementine to successfully
> connect to JACK and pipe audio to both the SIP client and to the
> conference.  I'd look in RhythmBox for a way to direct it to JACK
> directly instead of going through PulseAudio.
>
> Jitsi also works with the above set-up: prior to the first test Zoom
> "meeting" (which was more of a social get-together) I did try a test
> with Jitsi, and was able to link a SIP desk phone (which was "standing
> in" for the PSTN user) to Jitsi through the same technique.
>
> I didn't try a telephone link-up with Slack, however I also use the same
> headset + JACK set-up with a work teleconference every weekday without
> issues.  I see no reason why it wouldn't work there too.
>
> For Bluetooth headsets, you'd need some way of exposing the headset to
> ALSA directly so JACK can access it.  Normally the BlueZ Bluetooth stack
> links up to PulseAudio.  Also bear in mind, unless there's some
> successor to the HSP profile, you'll be limited to 8kHz 8-bit mono
> audio, which will sound pretty bloody terrible!
>
> Yes, there's A2DP, but from what I've seen, most headsets only do A2DP
> in one direction: usually for listening to music, when a call comes in
> they switch to HSP for the telephone call.  I'd be very happy to be
> proven wrong on this, but Bluetooth has brought nothing but
> disappointment for me.
>
> Based on this, you might be better served investing in a wireless USB
> headset.
>
> Regards,
>
Hi Stuart,

Thanks for the detailed response!  I have to admit that I was a bit
intimidated by your reply as while I'm very comfortable with using the
command line, I have no experience tinkering within the bowels of ALSA,
PulseAudio and JACK.

Before I start to unpack your steps, do you have any suggestions for
resources on how to get started with JACK?  The wiki is a bit
impenetrable for a complete beginner like me. Seems like I need to do a
bit more reading before I can tackle your suggestions.

Thanks again.
Samir
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Martin Desrumaux
In reply to this post by Len Ovens
On 16/05/2020 07:25, Len Ovens wrote:

> On Fri, 15 May 2020, Samir Parikh wrote:
>
>> I am trying to combine the audio from my microphone (either built in
>> microphone from my laptop or bluetooth headset) with music playing
>> from Rhythmbox Music Player running on Ubuntu 16.04 and pipe that as
>> the audio input to video conference services such as Jitsi.
>>
>> I don't know much about Linux audio internals and wasn't sure how to
>> do this.  Can I do this completely via the operating system using
>> Pulse Audio?  Do I need to do something with Jack?  Or do I need
>> specialized software such as OBS, Ardour or Reaper?
>
> OBS is probably the easiest way. You can add your local video if
> needed as well. OBS will work fine with just pulse or jack. I don't
> know much about jitsi it self but if it uses pulse as the default
> device there should be no problem.

I'd also look towards OBS solution.

In the previous week, I was looking for streaming both video and audio
from OBS Studio to Jitsi.
I found this Github thread:
https://github.com/CatxFish/obs-virtual-cam/issues/17

For the video, you can use the following OBS plugin:
https://github.com/CatxFish/obs-v4l2sink
This plugin uses the v4l2loopback kernel module.

For audio, some user suggested to create a virtual sound card with `sudo
modprobe snd-aloop`.
See this comment:
https://github.com/CatxFish/obs-virtual-cam/issues/17#issuecomment-514565571

With this solution, you should be able to configure volumes of "Desktop
Audio" and "Mic/Aux" in OBS's Audio Mixer and stream it to ffmpeg which
forward the stream to the virtual audio card.
Then, you can use the virtual audio card as your microphone in Jitsi or
any other software / web app.

However, this solution may have latency. If you need low latency, I
guess that you'd need to ditch ffmpeg and use Jack instead of pulseaudio
but I'm not sure which way to go.

Hope it helps :)

>
>
> --
> Len Ovens
> www.ovenwerks.net
> _______________________________________________
> Linux-audio-user mailing list
> [hidden email]
> https://lists.linuxaudio.org/listinfo/linux-audio-user
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user
Reply | Threaded
Open this post in threaded view
|

Re: How to combine microphone audio and music from application to pipe to video conference

Stuart Longland
In reply to this post by Samir Parikh
On 17/5/20 11:08 pm, Samir wrote:
> Thanks for the detailed response!  I have to admit that I was a bit
> intimidated by your reply as while I'm very comfortable with using the
> command line, I have no experience tinkering within the bowels of ALSA,
> PulseAudio and JACK.

No problems there… I'm learning things as I go as well.  Right now, I'm
contemplating an ALSA plug-in and Android app that would let me use the
headset plugged into my phone as a "wireless" (by WiFi) headset
(streaming signed linear audio over RTP), as I'm not happy with the
available USB wireless headsets (notably, that most restrict capture
sample rates to 16kHz).

> Before I start to unpack your steps, do you have any suggestions for
> resources on how to get started with JACK?  The wiki is a bit
> impenetrable for a complete beginner like me. Seems like I need to do a
> bit more reading before I can tackle your suggestions.

The most important thing would be /etc/security/limits.conf settings so
that your user can run stuff with real-time priority.  If PulseAudio
works, then chances are this is already taken care of.

`qjackctl` isn't too intimidating to get started with.

https://stuartl.longlandclan.id.au/blog/2008/10/31/wireless-h/ shows one
set-up I had years ago for streaming between two sound cards so that I
could use a HF radio via a wireless headset.  A side-benefit of that was
being able to use FreqTweak to tune out un-modulated carriers and other
annoyances of radio communications.

The parameters I mainly play with are:
- input and output devices
- sample rate
- number of channels
- frames/period (powers of two, 1024 or greater normally)
- periods/buffer (usually 3)
- force 16-bit if the underlying hardware doesn't support 32-bit PCM
(often the case in this scenario)

frames/period and periods/buffer will affect latency, so more periods or
more frames, less break-up and XRuns, but more latency.

A lot of people here would be trying to tune that as low as possible as
they'd be doing live performances and such with their rigs.  For someone
like myself who just wants some flexibility in audio routing and isn't
so concerned about perfection, I can be more lenient on latency.

Really, something in between PulseAudio's ease of use and JACK's
flexibility is what I'm after.  If I could make PulseAudio feed signals
between applications the way I can with JACK, I'd use that.  Conversely
if I could make JACK handle hot-plug and multiple audio devices the way
PulseAudio does, I'd be there.

JACK gets close though, so that's where I am right now.

Regards,
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

Help fund COVID-19 research:
https://stuartl.longlandclan.id.au/blog/2020/04/20/who-covid19/
_______________________________________________
Linux-audio-user mailing list
[hidden email]
https://lists.linuxaudio.org/listinfo/linux-audio-user