What is required to make Android use a new Audio HAL

13

I am writing a new Android Audio HAL, to let my App to feed audio to other apps, to allow my handheld remote microphone input to reach Google app. Essentially, a Virtual Audio Cable.

Its a work in progress. I'll probably be overriding AUDIO_DEVICE_IN_BACK_MIC, but this is open for suggestions.

I have doubts on how to make sure Android uses this HAL for input.

Do I need to replace audio.primary.default.so or should I leave it as audio.vcable.default.so?

More specifically: If I am not replacing primary, how will Android know to use my HAL instead of primary?


Update:

I could really use any help in this work. Any pointers are helpful.

I have written an Audio HAL module. I have added following (bold items) to audio_policy.conf:

global:

global_configuration {
  attached_output_devices AUDIO_DEVICE_OUT_SPEAKER|**AUDIO_DEVICE_OUT_LINE**
  default_output_device AUDIO_DEVICE_OUT_SPEAKER
  attached_input_devices AUDIO_DEVICE_IN_BUILTIN_MIC|AUDIO_DEVICE_IN_BACK_MIC|AUDIO_DEVICE_IN_REMOTE_SUBMIX|**AUDIO_DEVICE_IN_LINE**
}

and under audio_hw_modules

  vloop {
    inputs {
      vloop {
        sampling_rates 16000
        channel_masks AUDIO_CHANNEL_IN_MONO
        formats AUDIO_FORMAT_PCM_16_BIT
        devices AUDIO_DEVICE_IN_LINE
      }
    }
    outputs {
      vloop {
        sampling_rates 16000
        channel_masks AUDIO_CHANNEL_OUT_STEREO
        formats AUDIO_FORMAT_PCM_16_BIT
        devices AUDIO_DEVICE_OUT_LINE
        flags AUDIO_OUTPUT_FLAG_DIRECT
      }
    }
  }

I also added following (bold) to AudioFlinger.cpp

static const char * const audio_interfaces[] = {
    AUDIO_HARDWARE_MODULE_ID_PRIMARY,
    AUDIO_HARDWARE_MODULE_ID_A2DP,
    AUDIO_HARDWARE_MODULE_ID_USB,
    **AUDIO_HARDWARE_MODULE_ID_VLOOP**
};

I can see that during boot, my HAL gets loaded, and I get these logs:

10-06 06:14:40.365 194-194/? I/AudioFlinger: Using default 3000 mSec as standby time.
10-06 06:14:46.664 194-194/? I/AudioPolicyService: AudioPolicyService CSTOR in new mode
10-06 06:14:46.673 194-194/? I/APM::ConfigParsingUtils: loadAudioPolicyConfig() loaded /system/etc/audio_policy.conf
10-06 06:14:46.681 194-194/? D/audio_hw_primary: adev_open: enter
10-06 06:14:46.797 194-194/? I/AudioFlinger: loadHwModule() Loaded primary audio interface from QCOM Audio HAL (audio) handle 1
10-06 06:14:46.797 194-194/? I/AudioFlinger: openOutput(), module 1 Device 2, SamplingRate 48000, Format 0x000001, Channels 3, flags 2
10-06 06:14:46.797 194-194/? I/AudioFlinger: AudioStreamOut::open(), mHalFormatIsLinearPcm = 1
10-06 06:14:46.798 194-194/? I/AudioFlinger: HAL output buffer size 240 frames, normal sink buffer size 960 frames
10-06 06:14:46.813 194-194/? I/AudioFlinger: Using module 1 has the primary audio interface
10-06 06:14:46.816 194-607/? I/AudioFlinger: AudioFlinger's thread 0xb4140000 ready to run
10-06 06:14:46.816 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2
10-06 06:14:46.818 194-194/? I/AudioFlinger: openOutput(), module 1 Device 2, SamplingRate 48000, Format 0x000001, Channels 3, flags 8
10-06 06:14:46.818 194-194/? I/AudioFlinger: AudioStreamOut::open(), mHalFormatIsLinearPcm = 1
10-06 06:14:46.818 194-194/? I/AudioFlinger: HAL output buffer size 960 frames, normal sink buffer size 960 frames
10-06 06:14:46.818 194-608/? I/AudioFlinger: AudioFlinger's thread 0xb3dc0000 ready to run
10-06 06:14:46.818 194-607/? E/AudioFlinger: no wake lock to update!
10-06 06:14:46.818 194-608/? D/audio_hw_primary: out_set_parameters: enter: usecase(0: deep-buffer-playback) kvpairs: routing=2
10-06 06:14:46.818 194-608/? E/AudioFlinger: no wake lock to update!
10-06 06:14:46.820 194-609/? I/AudioFlinger: AudioFlinger's thread 0xb3c40000 ready to run
10-06 06:14:46.823 194-194/? I/AudioFlinger: loadHwModule() Loaded a2dp audio interface from A2DP Audio HW HAL (audio) handle 7
10-06 06:14:46.828 194-194/? I/AudioFlinger: loadHwModule() Loaded usb audio interface from USB audio HW HAL (audio) handle 8
10-06 06:14:46.832 194-194/? I/r_submix: adev_open(name=audio_hw_if)
10-06 06:14:46.832 194-194/? I/AudioFlinger: loadHwModule() Loaded r_submix audio interface from Wifi Display audio HAL (audio) handle 9
10-06 06:14:46.832 194-194/? D/r_submix: submix_audio_device_create_pipe_l(addr=0, idx=9)
10-06 06:14:46.833 194-610/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run
10-06 06:14:46.833 194-194/? D/r_submix: submix_audio_device_release_pipe_l(idx=9) addr=0
10-06 06:14:46.833 194-194/? D/r_submix: submix_audio_device_destroy_pipe_l(): pipe destroyed
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open: audio_hw_if
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1678
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1685
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1688
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open(): 1722
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_init_check(): 1252
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_set_master_volume: 1.000000
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_set_master_mute: 0
10-06 06:14:46.835 194-194/? I/AudioFlinger: loadHwModule() Loaded vloop audio interface from UI_audio_HW_HAL (audio) handle 11
10-06 06:14:46.835 194-194/? D/audio_vloop: adev_open_input_stream(): 1490
10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_sample_rate(): 979
10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_channels(): 1017
10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_channels: 0x00000001
10-06 06:14:46.835 194-194/? D/audio_vloop: in_get_format(): 1029
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format: 0x00000001
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format(): 1029
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format: 0x00000001
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels(): 1017
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels: 0x00000001
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size(): 1005
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size: 1600
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size(): 1005
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_buffer_size: 1600
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format(): 1029
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_format: 0x00000001
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_sample_rate(): 979
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels(): 1017
10-06 06:14:46.836 194-194/? D/audio_vloop: in_get_channels: 0x00000001
10-06 06:14:46.838 194-613/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run
10-06 06:14:46.838 194-194/? D/audio_vloop: adev_close_input_stream(): 1570
10-06 06:14:46.839 194-194/? W/APM::AudioPolicyManager: Input device 00020000 unreachable
10-06 06:14:46.839 194-611/? D/audio_vloop: looper_thread(): 216: Entered
10-06 06:14:46.839 194-611/? D/audio_vloop: looper_thread(): 366: Exiting
10-06 06:15:07.137 616-616/? I/InputManager: Initializing input manager, mUseDevInputEventForAudioJack=false
10-06 06:15:10.155 616-616/? I/SystemServer: Audio Service
10-06 06:15:10.222 194-607/? E/AudioFlinger: no wake lock to update!
10-06 06:15:10.222 194-608/? E/AudioFlinger: no wake lock to update!
10-06 06:15:10.224 194-614/? D/audio_hw_primary: adev_set_mic_mute: state 0
10-06 06:15:10.224 194-614/? D/audio_vloop: adev_set_mic_mute: 0
10-06 06:15:14.061 194-614/? D/audio_hw_primary: adev_set_parameters: enter: A2dpSuspended=false
10-06 06:15:14.061 194-614/? D/audio_vloop: adev_set_parameters(): [A2dpSuspended=false]
10-06 06:15:14.084 194-194/? I/AudioFlinger: systemReady
10-06 06:15:16.308 194-194/? D/audio_hw_primary: adev_set_mic_mute: state 0
10-06 06:15:16.308 194-194/? D/audio_vloop: adev_set_mic_mute: 0
10-06 06:15:17.072 194-194/? D/audio_hw_primary: adev_set_parameters: enter: A2dpSuspended=false
10-06 06:15:17.072 194-194/? D/audio_vloop: adev_set_parameters(): [A2dpSuspended=false]
10-06 06:15:25.023 733-733/? W/AudioTrack: AUDIO_OUTPUT_FLAG_FAST denied by client; transfer 4, track 44100 Hz, output 48000 Hz
10-06 06:15:25.032 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2
10-06 06:15:25.043 194-607/? D/audio_hw_primary: select_devices: out_snd_device(2: speaker) in_snd_device(0: none)
10-06 06:15:25.043 194-607/? D/audio_hw_primary: enable_snd_device: snd_device(2: speaker)
10-06 06:15:25.050 194-607/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: low-latency-playback
10-06 06:15:26.431 1150-1298/? I/MicrophoneInputStream: mic_starting com.google.android.apps.gsa.speech.audio.ag@c6eb0e1
10-06 06:15:26.443 194-1585/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run
10-06 06:15:26.447 1150-1298/? I/MicrophoneInputStream: mic_started com.google.android.apps.gsa.speech.audio.ag@c6eb0e1
10-06 06:15:26.457 194-1585/? D/audio_hw_primary: select_devices: out_snd_device(0: none) in_snd_device(38: voice-rec-mic)
10-06 06:15:26.457 194-1585/? D/audio_hw_primary: enable_snd_device: snd_device(38: voice-rec-mic)
10-06 06:15:26.460 194-1585/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: audio-record
10-06 06:15:26.942 1150-1271/? I/AudioController: internalShutdown
10-06 06:15:26.943 1150-1271/? I/MicrophoneInputStream: mic_close com.google.android.apps.gsa.speech.audio.ag@c6eb0e1
10-06 06:15:26.943 1150-1298/? E/AudioRecord-JNI: Error -4 during AudioRecord native read
10-06 06:15:26.986 194-1585/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: audio-record
10-06 06:15:26.987 194-1585/? D/audio_hw_primary: disable_snd_device: snd_device(38: voice-rec-mic)
10-06 06:15:27.066 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2
10-06 06:15:27.100 194-607/? D/AudioFlinger: mixer(0xb4140000) throttle end: throttle time(7)
10-06 06:15:30.257 194-607/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: low-latency-playback
10-06 06:15:30.257 194-607/? D/audio_hw_primary: disable_snd_device: snd_device(2: speaker)
10-06 06:15:30.262 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2
10-06 06:15:30.272 194-607/? D/audio_hw_primary: select_devices: out_snd_device(2: speaker) in_snd_device(0: none)
10-06 06:15:30.273 194-607/? D/audio_hw_primary: enable_snd_device: snd_device(2: speaker)
10-06 06:15:30.280 194-607/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: low-latency-playback
10-06 06:15:30.347 194-607/? D/AudioFlinger: mixer(0xb4140000) throttle end: throttle time(10)
10-06 06:15:31.517 194-607/? D/audio_hw_primary: out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2
10-06 06:15:31.751 1150-1298/? I/MicrophoneInputStream: mic_starting com.google.android.apps.gsa.speech.audio.ag@dd13203
10-06 06:15:31.762 194-1826/? I/AudioFlinger: AudioFlinger's thread 0xb3bc0000 ready to run
10-06 06:15:31.771 1150-1298/? I/MicrophoneInputStream: mic_started com.google.android.apps.gsa.speech.audio.ag@dd13203
10-06 06:15:31.780 194-1826/? D/audio_hw_primary: select_devices: out_snd_device(0: none) in_snd_device(38: voice-rec-mic)
10-06 06:15:31.780 194-1826/? D/audio_hw_primary: enable_snd_device: snd_device(38: voice-rec-mic)
10-06 06:15:31.783 194-1826/? D/audio_hw_primary: enable_audio_route: apply and update mixer path: audio-record
10-06 06:15:34.695 194-607/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: low-latency-playback
10-06 06:15:34.695 194-607/? D/audio_hw_primary: disable_snd_device: snd_device(2: speaker)
10-06 06:15:34.850 1150-1271/? I/AudioController: internalShutdown
10-06 06:15:34.851 1150-1271/? I/MicrophoneInputStream: mic_close com.google.android.apps.gsa.speech.audio.ag@dd13203
10-06 06:15:34.851 1150-1298/? E/AudioRecord-JNI: Error -4 during AudioRecord native read
10-06 06:15:34.885 194-1826/? D/audio_hw_primary: disable_audio_route: reset and update mixer path: audio-record
10-06 06:15:34.885 194-1826/? D/audio_hw_primary: disable_snd_device: snd_device(38: voice-rec-mic)

Mine is audio_vloop. I can see that Android opens my device, then opens input stream, and then closes the input stream. It never attempts to open output stream. audio_vloop implements both input and output streams. After this, nothing in audio_vloop is ever invoked by Android.

I made a small app that plays audio (from a pcm file for now). I want to redirect this output to my HAL. To achieve this, I believe I need to do a AudioTrack.setPreferredDevice() on my audio track. I found that Audio Manager should have a list of all audio devices.

so I call :

AudioDeviceInfo aDevInfo[] = am.getDevices(AudioManager.GET_DEVICES_OUTPUTS);

It only finds 1 device, more info on this device:

10-06 06:37:01.962 3295-3663/? D/AudioPlayer: Have [1] devices
10-06 06:37:01.964 3295-3663/? D/AudioPlayer: devInfo[0]: [Landroid.media.AudioDeviceInfo;@90bd9da
10-06 06:37:01.965 3295-3663/? D/AudioPlayer: getProductName()AOSP on Flo
10-06 06:37:01.965 3295-3663/? D/AudioPlayer: getType()2
10-06 06:37:01.966 3295-3663/? D/AudioPlayer: isSink()true
10-06 06:37:01.966 3295-3663/? D/AudioPlayer: isSource()false

This seemingly is from audioPort which I have not implemented. So its not from my HAL.

I've obviously missed one or more steps before Android will allow my app to talk to my device.

I need to be able to send audio from my app into my HAL. Later, I also need to be able to receive audio from my HAL (through AudioRecord etc).

What did I missed in integrating my HAL into Android? Do I need to implement Audio Ports? Is something else required?


Update 2

I found there is a typo in AOSP, in AudioPolicyManager.cpp@2922

Instead of Output, it prints Input

I had this log AudioPolicyManager: Input device 00020000 unreachable which I disregarded assuming it is talking about BT/A2DP input device.

I fixed the log for my device, and it turns out to be Line out device which we want to use. I am debugging this direction now.

android
audio
hal
asked on Stack Overflow Sep 28, 2016 by GPS • edited Oct 7, 2016 by GPS

1 Answer

6

Found answers.

For out stream:

  1. Outputs that don't support AUDIO_CHANNEL_OUT_STEREO are not being opened by Android
  2. Outputs that mention flag AUDIO_OUTPUT_FLAG_DIRECT are not opened by Android automatically. This is evident in following code in AudioPolicyManager.cpp

        if ((outProfile->mFlags & AUDIO_OUTPUT_FLAG_DIRECT) != 0) {
            continue;
        }
    

    from here

There may be a way to open them in programatically, but I have not found answer to that.

These two, once fixed were sufficient for Android to start using my out stream.

For in stream:

In stream was already a part of results of AudioManager.getDevices()

So it was possible to read from vloop in stream after AudioTrack.setPreferredDevice().

To ensure that other apps will read mic input from vloop, I had to declare it to implement AUDIO_DEVICE_IN_BUILTIN_MIC. For this to work, I also removed AUDIO_DEVICE_IN_BUILTIN_MIC from primary HAL in audio_policy.conf.

Additionally, I made in stream stereo only to maintain compatibility with out stream buffer format.

After these changes, I see that there are continuous read and write calls coming to vloop.

UPDATE:

I later found that above mentioned behavior is dependent on Audio Policy Manager implementation. Most of them behave same way, (e.g. most open INBUILT_MIC for VOICE_RECOGNITION input) but some may not (Nexus Player) For these outliers, either implemnent what their APMs open, or modify APMs to open what your HAL implements.

answered on Stack Overflow Oct 9, 2016 by GPS • edited Jan 5, 2017 by GPS

User contributions licensed under CC BY-SA 3.0