Rhasspy satellite with Raspberry Pi

I have several Raspberry Pies around the house, serving as voice interfaces (Rhasspy “satellites”), some of them also as media players using Kodi. They all have a ReSpeaker 2-Mics Pi HAT sound card, which contains two microphones and an audio output, connected to a small speaker.

For a speaker, I typically use the passive half of an Amazon Basics PC Speakers pair, available from amazon.de in Germany, or amazon.com in the US.

This is part of my Rhasspy-and-OpenHAB home automation setup. For an overview, see this blog post.

Operating system and kernel version

A limitation of the installation I describe below is that it will currently only work with a v5 Linux kernel, because the sound card drivers are not available for the v6 kernel, as I write this in November 2023.

My Raspberry Pi based satellites run on Raspbian 10 (buster) and Linux kernel 5.10.103-v7+. In preparing for this blog post, I tried to re-create the installation with the latest version, Raspberry Pi OS 12 (bookworm) with Linux kernel 6.1, but I ran into various difficulties: the sound card driver is not available for v6 kernels yet, and I couldn’t install Rhasspy, because of some package incompatibilites. I’ll try again in a year, but for now, if you want to re-create my setup, you have to find an “old” Raspbian 10 image to put on your SD card, e.g. here.

Install sound card driver

I didn’t like the drivers provided by the manufacturer of the ReSpeaker sound card, too many changes to my setup, too invasive, so I am using the simpler driver for the WM8960 sound chip, which is at the heart of this sound card.

sudo apt-get install raspberrypi-kernel-headers
git clone https://github.com/pguyot/wm8960.git
cd wm8960
make
sudo make install

Next, edit /boot/config and include

dtparam=i2s=on
dtoverlay=i2s-mmap
dtoverlay=wm8960

Configure sound card settings

The WM8960 sound card has a lot of configuration parameters, I’m not claiming I understand all of them. The following is the output from amixer --card WM8960 contents for a configuration that works for me.

contents Download

Use alsamixer -c wm8960 to adjust your sound card accordingly. Once you are satisfied with your settings, store them with alsactl store.

In my settings, microphone gain is cranked up fairly high, so I can speak to it from 2-3 meters away without raising my voice. Also, while the sound card has 2 microphones, I only feed one of them to the voice detection engine (parameter ‘ADC Data Output Select’ is set to 1, meaning ‘Left Data = Left ADC; Right Data = Left ADC’. Theoretically, one could use the signals from both microphones for some sophisticated digital signal processing like beamforming, but that is not supported by Rhasspy, as far as I know.

Test the driver

Let’s test the driver. First, see which sound cards are available. On my Raspberry Pi, which also runs Kodi with audio output through HDMI, this is the response I get when I enter aplay -l:

card 0: b1 [bcm2835 HDMI 1], device 0: bcm2835 HDMI 1 [bcm2835 HDMI 1] this is used by Kodi
 Subdevices: 3/4
 Subdevice #0: subdevice #0
 Subdevice #1: subdevice #1
 Subdevice #2: subdevice #2
 Subdevice #3: subdevice #3
card 1: Headphones [bcm2835 Headphones], device 0: bcm2835 Headphones [bcm2835 Headphones]
 Subdevices: 4/4
 Subdevice #0: subdevice #0
 Subdevice #1: subdevice #1
 Subdevice #2: subdevice #2
 Subdevice #3: subdevice #3
card 2: wm8960 [wm8960], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
 Subdevices: 1/1
 Subdevice #0: subdevice #0

When I enter arecord -l to see the list of available sound card inputs, I get

card 2: wm8960 [wm8960], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
 Subdevices: 1/1
 Subdevice #0: subdevice #0

To test sound capture, I enter

arecord -D "plughw:wm8960,0" -f dat -d 10 -t wav voicetest.wav

… and then I make some sounds while the recording goes on for 10 seconds. Afterwards, I can play back the recording and verify that it sounds ok

aplay -D "plughw:CARD=wm8960,DEV=0" voicetest.wav

Install Rhasspy

This was tested with Rhasspy 2.5.11. To install the package, type

wget https://github.com/rhasspy/rhasspy/releases/latest/download/rhasspy_armhf.deb
sudo apt install ./rhasspy_armhf.deb

Configure the satellite

Start Rhasspy, browse to the Web UI at http://machinename:12101/advanced, and copy & paste these settings, then click Save Profile:

{
    "intent": {
        "system": "hermes"
    },
    "microphone": {
        "arecord": {
            "device": "plughw:CARD=wm8960,DEV=0",
            "udp_audio_host": "localhost",
            "udp_audio_port": "12202"
        },
        "system": "arecord"
    },
    "mqtt": {
        "enabled": "true",
        "host": "your-mqtt-server-name-goes-here",
        "site_id": "your-site-id-goes-here"
    },
    "sounds": {
        "aplay": {
            "device": "plughw:CARD=wm8960,DEV=0"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "system": "hermes"
    },
    "text_to_speech": {
        "system": "hermes"
    },
    "wake": {
        "porcupine": {
            "keyword_path": "porcupine_raspberry-pi.ppn",
            "udp_audio": "localhost:12202"
        },
        "system": "porcupine"
    }
}

You may get some error messages on the Web UI saying that certain modules are not installed. If that happens, just go ahead and install through the Web UI.

Now, when I switch to the Settings page, it looks like this (I have obviously chosen a site id of “raspi7“). Basically I have selected

audio recording is done with arecord, using the device plughw:CARD=wm8960,DEV=0, streaming audio content to localhost:12202, until teh wake word is detected.
wake word detection listens for the word “porcupine” on audio stream localhost:12202 .
speech to text is done on the the server, once the wake word has been detected
intent recognition is also done on the server, not on the Raspberry Pi satellite
text to speech is also done on the server, and streamed to the satellite via MQTT
No dialog management (listen – beep after wake word – listen to command etc) is done on the satellite

Start Rhasspy automatically

Create a systemd service file /etc/systemd/system/rhasspy.service with this content

[Unit]
Description=Rhasspy Service
After=network-online.target syslog.target

[Service]
Type=simple
User=admin
WorkingDirectory=/home/admin  # <-- adjust this to your setup
ExecStart=/bin/bash -c '/usr/bin/rhasspy --profile en 2>&1 | cat'
StandardOutput=syslog
StandardError=syslog
RestartSec=10
Restart=on-failure
SyslogIdentifier=rhasspy

[Install]
WantedBy=multi-user.target

then enter (as root)

systemctl daemon-reload
systemctl enable rhasspy
systemctl start rhasspy

Let the master know about the satellite

Finally, you need to make the satellite known to the master Rhasspy installation. Browse to the Web UI on the server where your Rhasspy master is running, and add the site id for our new satellite to the “Satellite siteIds” fields in the sections “Speech to Text”, “Intent Recognition”, “Text to Speech” and “Dialogue Management”. Here’s what this looks like on my installation, with multiple satellites:

Location and orientation

The optimal orientation would be with the sound card vertical, I suppose, with the microphones pointing towards the person speaking a command. In my setup, orientation is far from ideal, but it still works sufficiently well, maybe due to my particular settings (see above). In one room, the Raspberry Pi + ReSpeaker HAT sandwich is on top of a bookshelf, i.e. with the microphones facing upwards, i.e. the voice source (me) at -30° elevation, both microphones at an equal distance from me (let’s call that 0° azimuth), about 3m away. In another room, the sandwich is on a low shelf, again with the microphones facing upwards, the voice source typically at 0° elevation, about 1m away from where I typically sit, but I am way off to the side, almost 90° azimuth.

The Raspberry Pies are not in a closed case though, I only use a Simple Case made of plexiglas sheets, with the top shelf over the microphones replaced with a piece of perf board.

Conclusion

Everything works reasonably well, I hardly ever have to repeat myself, and I don’t have to raise my voice. Sometimes, there is a “false start”, i.e. the system detects the wakeword “Porcupine” when I didn’t say anything, but the TV or radio was playing … although I have never heard them use the word “porcupine” on TV …

Addendum: making it work with a v6 kernel

Ok, I had to go back and try to install this on a machine with a v6 kernel, in my case a Raspberry Pi OS Lite “bullseye” 32-bit, Linux 6.1.21-v7+ on armv7l.

Rhasspy can be installed as described above, without errors. However, I needed to sudo apt-get install libffi6 to install a missing library. I still can’t figure out how to install it on Raspberry Pi OS based on Debian 12 “bookworm”.

The sound card driver can be made to compile, install and work with a few small changes to the source code. Download as described above, then edit file wm8960.c according to these instructions :

in line 1385, change .non_legacy_dai_naming = 1 to .legacy_dai_naming = 0
in line 1476, change static int wm8960_i2c_remove( to static void wm8960_i2c_remove(
in line 1478, change return 0; to return;