Segmentation Fault

[S1MB10T3]

After running ./run.sh -c -H20 -m0 -M mind/boot on both my Debian laptop and my raspberry pi I get a segmentation fault.

Console Log:

pi@raspberrypi:~/openassistant$ ./run.sh -c -H20 -m0 -M mind/boot                                                                                                 
Current configuration:
[NAME]                  [DEFLT]         [VALUE]
-agc                    none            none
-agcthresh              2.0             2.000000e+00
-allphone
-allphone_ci            no              no
-alpha                  0.97            9.700000e-01
-ascale                 20.0            2.000000e+01
-aw                     1               1
-backtrace              no              no
-beam                   1e-48           1.000000e-48
-bestpath               yes             yes
-bestpathlw             9.5             9.500000e+00
-ceplen                 13              13
-cmn                    current         current
-cmninit                8.0             8.0
-compallsen             no              no
-debug                                  0
-dict
-dictcase               no              no
-dither                 no              no
-doublebw               no              no
-ds                     1               1
-fdict
-feat                   1s_c_d_dd       1s_c_d_dd
-featparams
-fillprob               1e-8            1.000000e-08
-frate                  100             100
-fsg
-fsgusealtpron          yes             yes
-fsgusefiller           yes             yes
-fwdflat                yes             yes
-fwdflatbeam            1e-64           1.000000e-64
-fwdflatefwid           4               4
-fwdflatlw              8.5             8.500000e+00
-fwdflatsfwin           25              25
-fwdflatwbeam           7e-29           7.000000e-29
-fwdtree                yes             yes
-hmm
-input_endian           little          little
-jsgf
-keyphrase
-kws
-kws_delay              10              10
-kws_plp                1e-1            1.000000e-01
-kws_threshold          1               1.000000e+00
-latsize                5000            5000
-lda
-ldadim                 0               0
-lifter                 0               0
-lm
-lmctl
-lmname
-logbase                1.0001          1.000100e+00
-logfn
-logspec                no              no
-lowerf                 133.33334       1.333333e+02
-lpbeam                 1e-40           1.000000e-40
-lponlybeam             7e-29           7.000000e-29
-lw                     6.5             6.500000e+00
-maxhmmpf               30000           30000
-maxwpf                 -1              -1
-mdef
-mean
-mfclogdir
-min_endfr              0               0
-mixw
-mixwfloor              0.0000001       1.000000e-07
-mllr
-mmap                   yes             yes
-ncep                   13              13
-nfft                   512             512
-nfilt                  40              40
-nwpen                  1.0             1.000000e+00
-pbeam                  1e-48           1.000000e-48
-pip                    1.0             1.000000e+00
-pl_beam                1e-10           1.000000e-10
-pl_pbeam               1e-10           1.000000e-10
-pl_pip                 1.0             1.000000e+00
-pl_weight              3.0             3.000000e+00
-pl_window              5               5
-rawlogdir
-remove_dc              no              no
-remove_noise           yes             yes
-remove_silence         yes             yes
-round_filters          yes             yes
-samprate               16000           1.600000e+04
-seed                   -1              -1
-sendump
-senlogdir
-senmgau
-silprob                0.005           5.000000e-03
-smoothspec             no              no
-svspec
-tmat
-tmatfloor              0.0001          1.000000e-04
-topn                   4               4
-topn_beam              0               0
-toprule
-transform              legacy          legacy
-unit_area              yes             yes
-upperf                 6855.4976       6.855498e+03
-uw                     1.0             1.000000e+00
-vad_postspeech         50              50
-vad_prespeech          20              20
-vad_startspeech        10              10
-vad_threshold          2.0             2.000000e+00
-var
-varfloor               0.0001          1.000000e-04
-varnorm                no              no
-verbose                no              no
-warp_params
-warp_type              inverse_linear  inverse_linear
-wbeam                  7e-29           7.000000e-29
-wip                    0.65            6.500000e-01
-wlen                   0.025625        2.562500e-02

Segmentation fault

[joshuashort]

Try running with a different logging level (e.g. logging.DEBUG) https://github.com/vavrek/openassistant/blob/6a51f51c0599d4843310334adffe3a4a6f3404e8/run.py#L6 I wish there was a switch to do that.

And I haven't poked around in the code recently enough to know what phase of running you're getting to. That output is from Sphinx.. is -m0 the correct audio for your system? Try leaving that parameter out of the command when you run it (that should make it try to figure out the system default device).

[Dennsen]

I have the same problem. Here's the output with DEBUG enabled. It's a Ubuntu 16.04 system.

$./run.sh -c -H20 -M mind/boot
DEBUG:modules.speech_recognition.gst:Loading
DEBUG:main:Arguments: Namespace(continuous=True, history=20, invalid_sentence_command=None, microphone=None, mind_dir='mind/boot', pass_words=False, valid_sentence_command=None)
INFO:core.util.config:Loading Mind: mind/boot
DEBUG:main:Configuring Module: Language
DEBUG:modules.language:Updating Language
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.speech.cs.cmu.edu
DEBUG:requests.packages.urllib3.connectionpool:"POST /cgi-bin/tools/lmtool/run HTTP/1.1" 302 0
DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884 HTTP/1.1" 301 273
DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884/ HTTP/1.1" 200 None
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.speech.cs.cmu.edu
DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884/4137.lm HTTP/1.1" 200 995
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): www.speech.cs.cmu.edu
DEBUG:requests.packages.urllib3.connectionpool:"GET /tools/product/1520770653_15884/4137.dic HTTP/1.1" 200 93
DEBUG:main:Configuring Module: Speech Recognition
DEBUG:modules.speech_recognition.gst:Initializing Recognizer
DEBUG:modules.speech_recognition.gst:<core.util.config.Config object at 0x7f4e51d3feb8>
DEBUG:modules.speech_recognition.gst:{'invalid_sentence_command': False, 'continuous': False, 'valid_sentence_command': False, 'interface': None, 'pass_words': False, 'microphone': True, 'history': True}
DEBUG:modules.speech_recognition.gst:autoaudiosrc device="hw:True,0" ! audioconvert ! audioresample ! pocketsphinx lm=mind/boot/cache/lm dict=mind/boot/cache/dic ! appsink sync=false
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+00
-allphone
-allphone_ci no no
-alpha 0.97 9.700000e-01
-ascale 20.0 2.000000e+01
-aw 1 1
-backtrace no no
-beam 1e-48 1.000000e-48
-bestpath yes yes
-bestpathlw 9.5 9.500000e+00
-ceplen 13 13
-cmn current current
-cmninit 8.0 8.0
-compallsen no no
-debug 0
-dict
-dictcase no no
-dither no no
-doublebw no no
-ds 1 1
-fdict
-feat 1s_c_d_dd 1s_c_d_dd
-featparams
-fillprob 1e-8 1.000000e-08
-frate 100 100
-fsg
-fsgusealtpron yes yes
-fsgusefiller yes yes
-fwdflat yes yes
-fwdflatbeam 1e-64 1.000000e-64
-fwdflatefwid 4 4
-fwdflatlw 8.5 8.500000e+00
-fwdflatsfwin 25 25
-fwdflatwbeam 7e-29 7.000000e-29
-fwdtree yes yes
-hmm
-input_endian little little
-jsgf
-keyphrase
-kws
-kws_delay 10 10
-kws_plp 1e-1 1.000000e-01
-kws_threshold 1 1.000000e+00
-latsize 5000 5000
-lda
-ldadim 0 0
-lifter 0 0
-lm
-lmctl
-lmname
-logbase 1.0001 1.000100e+00
-logfn
-logspec no no
-lowerf 133.33334 1.333333e+02
-lpbeam 1e-40 1.000000e-40
-lponlybeam 7e-29 7.000000e-29
-lw 6.5 6.500000e+00
-maxhmmpf 30000 30000
-maxwpf -1 -1
-mdef
-mean
-mfclogdir
-min_endfr 0 0
-mixw
-mixwfloor 0.0000001 1.000000e-07
-mllr
-mmap yes yes
-ncep 13 13
-nfft 512 512
-nfilt 40 40
-nwpen 1.0 1.000000e+00
-pbeam 1e-48 1.000000e-48
-pip 1.0 1.000000e+00
-pl_beam 1e-10 1.000000e-10
-pl_pbeam 1e-10 1.000000e-10
-pl_pip 1.0 1.000000e+00
-pl_weight 3.0 3.000000e+00
-pl_window 5 5
-rawlogdir
-remove_dc no no
-remove_noise yes yes
-remove_silence yes yes
-round_filters yes yes
-samprate 16000 1.600000e+04
-seed -1 -1
-sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03
-smoothspec no no
-svspec
-tmat
-tmatfloor 0.0001 1.000000e-04
-topn 4 4
-topn_beam 0 0
-toprule
-transform legacy legacy
-unit_area yes yes
-upperf 6855.4976 6.855498e+03
-uw 1.0 1.000000e+00
-vad_postspeech 50 50
-vad_prespeech 20 20
-vad_startspeech 10 10
-vad_threshold 2.0 2.000000e+00
-var
-varfloor 0.0001 1.000000e-04
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wbeam 7e-29 7.000000e-29
-wip 0.65 6.500000e-01
-wlen 0.025625 2.562500e-02

INFO:core.assistant:Initializing Assistant
DEBUG:core.assistant:<core.util.config.Config object at 0x7f4e51d3feb8>
Segmentation fault

small update: dmsg shows

autoaudiosrc0-a[7255]: segfault at 10 ip 00007f09a12c23da sp 00007f09977fd810 error 4 in libpocketsphinx.so.3.0.0[7f09a128c000+45000]

update nr. 2: running the pipeline manually also leads to a segfault:

gst-launch-1.0 -vvv autoaudiosrc ! audioconvert ! audioresample ! pocketsphinx lm=mind/boot/cache/lm dict=mind/boot/cache/dic ! appsink sync=false
...
Leitung wird auf PAUSIERT gesetzt ...
Leitung ist aktiv und erfordert keinen VORLAUF …
Leitung wird auf ABSPIELEN gesetzt ...
New clock: GstPulseSrcClock
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: source-output-index = 24
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: actual-buffer-time = 200000
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: actual-latency-time = 10000
Verzögerung neu verteilen …
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls.GstPad:src: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0.GstGhostPad:src: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:src: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstPocketSphinx:pocketsphinx0.GstPad:sink: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:sink: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0.GstGhostPad:src.GstProxyPad:proxypad0: caps = "audio/x-raw,\ format=(string)S16LE,\ layout=(string)interleaved,\ rate=(int)16000,\ channels=(int)1"
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: volume = 1
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: mute = false
/GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: current-device = alsa_input.usb-Kingston_HyperX_7.1_Audio_00000000-00.analog-stereo
Caught SIGSEGV
Spinning. Please run 'gdb gst-launch-1.0 10849' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

[joshuashort]

Thanks @Dennsen! I started dusting off my old development branch.. hope to update some stuff and move to the 'official' OA repository soonish (https://github.com/openassistant) -- I've got a branch that includes -d for executing with debug-level logging. But that's a separate topic..

The first issue I noticed was a line in your debug output where it's printing the pipeline, and it's using True as the device number.. this comes from an incorrect default configuration at https://github.com/vavrek/openassistant/blob/6a51f51c0599d4843310334adffe3a4a6f3404e8/mind/boot/conf/settings.json#L4 -- it shouldn't be true, it should actually be a number representing the audio device.. like 0 or 1. It doesn't look like this version of the code respect run-time arguments when there are defaults set, so try updating that configuration file before running. If your audio device is 0, this block (https://github.com/vavrek/openassistant/blob/6a51f51c0599d4843310334adffe3a4a6f3404e8/modules/speech_recognition/gst.py#L26-L31) doesn't actually do the right thing.. it should be changed to if src is not None -- 0 is a valid device number, but it'll be treated as False for this initialization logic.

I guess I'm most concerned about what's happening when you try to manually run the pipeline.. how recent is your gst/pocketsphinx install? How did you install pocketsphinx? Do you also have the other gst-plugins (i.e. good, bad, ugly)?

[Dennsen]

Hey, thanks for the answer.

I installed pocketsphinx via:
sudo apt install gstreamer1.0-pocketsphinx
All other gst-plugins were also installed.
After installing a new version of pocketsphinx like described here in the link:
https://bakerstreetsystems.com/blog/post/setting-cmu-sphinx-pocketsphinx-ubuntu-1404
it works without a segfault. Don't know, what's wrong with the default lib.

Guess I can live with that for now, but we should keep an eye on this.

[joshuashort]

Good call! I haven't had much luck with the package installs of pocketsphinx.. I think our advice for now is to simply build from source.

Just one more thing, @Dennsen, does the output from pocketsphinx now show the expected paths for lm, dict, and hmm?

[Dennsen]

Good point, I checked the output from the following pipe, seems like the wrong path is used:

gst-launch-1.0 -vvv autoaudiosrc ! audioconvert ! audioresample ! pocketsphinx lm=mind/boot/cache/lm dict=mind/boot/cache/dic ! appsink sync=false
....
-dict /usr/local/share/pocketsphinx/model/en-us/cmudict-en-us.dict
-hmm /usr/local/share/pocketsphinx/model/en-us/en-us
-lm /usr/local/share/pocketsphinx/model/en-us/en-us.lm.bin
...

[S1MB10T3]

Well back on working on my pi project. I can't seem figure out how to get the pocketsphinx built from source to work. I don't think it's installing in the right place. I'm going to try to see if I can get openassistant to work on a vm or a clean install of raspbian.