can PyAudio handle non-interleaved buffers?

2

I'm having trouble passing non-interleaved stereo data to PyAudio.

update (rewording question substantially)

The examples given in the PyAudio docs show how to pass interleaved data with a callback. A buffer of interleaved stereo data has a shape of (nframes, 2), and looks like this:

array([[f0l, f0r],
       [f1l, f1r],
       [f2l, f2r],
       ...], dtype=float32)

Partly for didactic reasons, I'd like to use non-interleaved data in my callback. Reading the docs, I think a buffer of non-interleaved stereo data has a shape of (2, nframes) and looks like this:

array([[f0l, f1l, f2l, ...],
       [f0r, f1r, f2r, ...]], dtype=float32)

And AFAICT, the call to set up the stream in non-interleaved mode should OR in the non-interleaved bit in the format argument like this:

PA_NONINTERLEAVED = 0x80000000
stream = p.open(format=pyaudio.paFloat32 | PA_NONINTERLEAVED,
                channels=DEFAULT_CHANNEL_COUNT,
                rate=SRATE,
                output=True,
                stream_callback=callback)

But when I try to run this, it segfaults or blasts my ears with digital noise or doesn't play anything -- all hallmarks of a memory smasher somewhere.

Has anyone succeeded in passing non-interleaved data to PyAudio?

original question (and full code example)

The following code is supposed to write a 440Hz sinewave to the left channel and a 442Hz sinewave to the right channel.

Instead it segfaults.

I have successfully run a similar test using interleaved numpy data, but was hoping I could keep the channel data separate. Any pointers on what I'm doing wrong?

# python sketches/s05.py 
"""
PyAudio / NumPy example: synthesize and play stereo sine waves
using non-interleaved buffers
"""

import pyaudio
import numpy as np
import time

p = pyaudio.PyAudio()

SRATE = 44100
DEFAULT_CHANNEL_COUNT = 2
DEFAULT_FRAME_COUNT = 1024
# from http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html
PA_NONINTERLEAVED = 0x80000000

class Sinner:
    def __init__(self, freq, channel):
        self._dtheta = freq * 2.0 * np.pi / SRATE
        self._theta = 0.0
        self._channel = channel

    def render(self, buffer, frame_count):
        thetas = np.arange(frame_count) * self._dtheta + self._theta
        np.sin(thetas, buffer[self._channel])
        self._theta += frame_count * self._dtheta

sin_l = Sinner(440.0, 0)
sin_r = Sinner(442.0, 1)
out_data = np.zeros((DEFAULT_CHANNEL_COUNT, DEFAULT_FRAME_COUNT),
                    dtype=np.float32)

def callback(in_data, frame_count, time_info, status):
    global sin_l, sin_r, out_data

    if (out_data.shape[1] != frame_count):
        # resize numpy array if needed
        out_data = np.zeros((DEFAULT_CHANNEL_COUNT, frame_count),
                            dtype=np.float32)
    sin_l.render(out_data, frame_count)
    sin_r.render(out_data, frame_count)

    print(out_data[0])
    print(out_data[1])
    return (out_data * 0.4, pyaudio.paContinue)

stream = p.open(format=pyaudio.paFloat32 | PA_NONINTERLEAVED,
                channels=DEFAULT_CHANNEL_COUNT,
                rate=SRATE,
                output=True,
                stream_callback=callback)

stream.start_stream()
while stream.is_active(): time.sleep(0.1)
stream.stop_stream()
stream.close()
p.terminate()
python
numpy
pyaudio
asked on Stack Overflow Apr 28, 2016 by fearless_fool • edited May 1, 2016 by fearless_fool

2 Answers

0

I very much doubt the didactic superiority of non-interleaved data, it would be interesting to hear the other part of your reasons.

Are you aware that matplotlib plots the columns if you pass it a 2-dimensional array?

Also, it looks like you're overlooking the order setting of your array. What you're saying in your question only makes sense if you are assuming order='C'. You should be able to get the behavior you want by running your array through numpy.asfortranarray() and then passing the buffer to PyAudio (but without using the PA_NONINTERLEAVED flag!). Note that this will make an unnecessary copy of your data in many cases.

Given your example code, you seem to misunderstand how PortAudio's "non-interleaved" mode works. In this mode, PortAudio assumes that input and output data are C-arrays of pointers to a bunch of other arrays each holding the data for one channel. To be able to use this, you would need to get pointers to each row of your NumPy arrays. I don't know if that's possible in pure Python ...

Finally, some advertisement: You should consider using the sounddevice module instead of PyAudio. It still uses interleaved data and the channels are stored in columns, but it directly supports NumPy arrays and should be much easier to install and use than PyAudio.

answered on Stack Overflow May 2, 2016 by Matthias
0

No, at least still in version 0.2.11. When recording something this callback code allows to project only on one channel, in the example the first channel. The trick is to use NumPy and arrange the data into a array (cube) with dimensions frame x channel x data (= SAMPLE_WIDTH) and then project onto the first channel (0):


# 16 bits sample width
SAMPLE_WIDTH = 2


def __audio_callback__(in_data, frame_count, time_info, status_flags):
    channels = int(len(in_data) / (SAMPLE_WIDTH * frame_count))
    buffer = np.frombuffer(in_data, dtype=uint8)
    first_channel = buffer.reshape(frame_count, channels, SAMPLE_WIDTH)[:, 0, :].tobytes()
    audio_samples.append(first_channel)
    # when not recording, return here the processed data as first tuple part
    return None, pyaudio.paContinue
answered on Stack Overflow Dec 28, 2020 by k_o_

User contributions licensed under CC BY-SA 3.0