RaspberryPi 4B source install TensorFlow version issue

[SandroMiccoli]

Describe the bug
I'm trying to train my own wake word on a RaspberryPi 4B. After installing via source, I used precise-collect to record several samples, everything worked ok, but when I try to precise-listen or to precise-train I get these WARNING messages:

With precise-listen:

$ precise-listen ../wakewords/computer/computer-en.pb
WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.COMPILER_VERSION is deprecated. Please use tf.version.COMPILER_VERSION instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.CXX11_ABI_FLAG is deprecated. Please use tf.sysconfig.CXX11_ABI_FLAG instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.ConditionalAccumulator is deprecated. Please use tf.compat.v1.ConditionalAccumulator instead.
...

And precise-train:

$ precise-train -e 60 myword.net myword/
Using TensorFlow backend.
WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.COMPILER_VERSION is deprecated. Please use tf.version.COMPILER_VERSION instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.CXX11_ABI_FLAG is deprecated. Please use tf.sysconfig.CXX11_ABI_FLAG instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.ConditionalAccumulator is deprecated. Please use tf.compat.v1.ConditionalAccumulator instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3138: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/keras/optimizers.py:757: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.
.....

Those two warnings appear at the beggninng but the rest of outputs seem ok. But when I try to precise-listen my new .net file, an Error appears:

 $ precise-listen myword.net
WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.COMPILER_VERSION is deprecated. Please use tf.version.COMPILER_VERSION instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.CXX11_ABI_FLAG is deprecated. Please use tf.sysconfig.CXX11_ABI_FLAG instead.

WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/__init__.py:98: The name tf.ConditionalAccumulator is deprecated. Please use tf.compat.v1.ConditionalAccumulator instead.

Using TensorFlow backend.
Traceback (most recent call last):
  File "/home/pi/mycroft-precise/.venv/bin/precise-listen", line 33, in <module>
    sys.exit(load_entry_point('mycroft-precise', 'console_scripts', 'precise-listen')())
  File "/home/pi/mycroft-precise/precise/scripts/base_script.py", line 43, in run_main
    script = cls(args)
  File "/home/pi/mycroft-precise/precise/scripts/listen.py", line 58, in __init__
    self.listener = Listener(args.model, args.chunk_size)
  File "/home/pi/mycroft-precise/precise/network_runner.py", line 107, in __init__
    self.runner = runner_cls(model_name)
  File "/home/pi/mycroft-precise/precise/network_runner.py", line 85, in __init__
    self.model = load_precise_model(model_name)
  File "/home/pi/mycroft-precise/precise/model.py", line 54, in load_precise_model
    return load_keras().models.load_model(model_name)
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/keras/models.py", line 242, in load_model
    model_config = json.loads(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'

I've tried download the dev branch but the same error happens.

To Reproduce
Steps to reproduce the behavior:

1. Clone the repository
2. Run the ./setup.sh script
3. Activate python environment with source .venv/bin/activate
4. Try to precise-listen any model (I used some community data and also a test of my own after collecting several recordings and generating a .net with precise-train)

Expected behavior
I expected to run my created model but got an error there.

Environment

• Device type: Raspberry Pi 4B
• OS: Raspbian GNU/Linux 10 (buster)
• Linux voice 5.4.72-v7l+ #1356 SMP Thu Oct 22 13:57:51 BST 2020 armv7l GNU/Linux

Additional context
I'm using a clean buster installation. I'm using voice2json and everything works fine there.

[MatthewScholefield]

Thanks for the thorough report! Hmm, yeah this definitely looks like some problem related to the keras/tensorflow version.

You might be able to get it working by using code off of PR #141 since that uses Tensorflow 2 which should resolve a lot of version related issues.

Let me know if this helps.

[SandroMiccoli]

Thanks for the quick reply!

Ok, got the #141 and tried running the ./setup.sh, this was the output:

$ ./setup.sh
Reading package lists... Done
Building dependency tree
Reading state information... Done
curl is already the newest version (7.64.0-4+deb10u1).
cython is already the newest version (0.29.2-2).
libatlas-base-dev is already the newest version (3.10.3-8+rpi1).
libhdf5-dev is already the newest version (1.10.4+repack-10).
libopenblas-dev is already the newest version (0.3.5+ds-3+rpi1).
libpulse-dev is already the newest version (12.2-4+deb10u1).
portaudio19-dev is already the newest version (19.6.0-1).
python3-h5py is already the newest version (2.8.0-3).
python3-scipy is already the newest version (1.1.0-7).
swig is already the newest version (3.0.12-2).
python3-pip is already the newest version (18.1-5+rpt1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Obtaining file:///home/pi/mycroft-precise/runner
Requirement already satisfied: pyaudio in ./.venv/lib/python3.7/site-packages (from precise-runner==0.3.1) (0.2.11)
Installing collected packages: precise-runner
  Attempting uninstall: precise-runner
    Found existing installation: precise-runner 0.3.1
    Uninstalling precise-runner-0.3.1:
      Successfully uninstalled precise-runner-0.3.1
  Running setup.py develop for precise-runner
Successfully installed precise-runner
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Obtaining file:///home/pi/mycroft-precise
ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.2.0 (from mycroft-precise)
ERROR: No matching distribution found for tensorflow-gpu==2.2.0

Should I uninstall/remove something before the new install?

[MatthewScholefield]

Try unpinning tensorflow-gpu==2.2.0 in setup.py and rerunning. Otherwise see if you can find a built .whl pip package for the raspberry pi 4B that you can pip install file.whl within the venv.

[SandroMiccoli]

Ok, so tried unpinning from setup.py but didn't work.

I found this guide on installing tensorflow-2.2.0 on RPi 4 and the installation went ok. Had to install it directly in the venv for it to work with precise though.

Tried running precise-listen and didn't give any warnings and the outputs look promissing.

I'm running precise-train on my recordings now and apparently, it's going well... Although a warning message appeared here also:

$ precise-train -e 60 myword.net myword/
Data: <TrainData wake_words=20 not_wake_words=0 test_wake_words=9 test_not_wake_words=0>
Loading wake-word...
Loading not-wake-word...
Loading wake-word...
Loading not-wake-word...
Inputs shape: (20, 29, 13)
Outputs shape: (20, 1)
Test inputs shape: (9, 29, 13)
Test outputs shape: (9, 1)
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
net (GRU)                    (None, 20)                2100
_________________________________________________________________
dense (Dense)                (None, 1)                 21
=================================================================
Total params: 2,121
Trainable params: 2,121
Non-trainable params: 0
_________________________________________________________________
Epoch 1/60
1/1 [==============================] - ETA: 0s - loss: 0.1328 - accuracy: 0.65002020-12-03 15:57:52.315038: W tensorflow/python/util/util.cc:329] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.


WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.


1/1 [==============================] - 13s 13s/step - loss: 0.1328 - accuracy: 0.6500
Epoch 2/60
1/1 [==============================] - 13s 13s/step - loss: 0.0818 - accuracy: 0.7500
Epoch 3/60
1/1 [==============================] - 13s 13s/step - loss: 0.0745 - accuracy: 0.7500
....

After the training finishes, I'll test it out and post here how it went.

[SandroMiccoli]

Well, still no success... I'm following these steps to train my own word.

Everything went well until precise-training-incremental:

$ precise-train-incremental myword.net myword/ -r data/random/
Loading wake-word...
Loading not-wake-word...
Loading wake-word...
Loading not-wake-word...
Starting file data/random/pencil sharpening.wav...
31.60919540229885%%
Saved to: myword/not-wake-word/generated/pencil sharpening-55.wav
32.758620689655174%
Saved to: myword/not-wake-word/generated/pencil sharpening-57.wav
37.35632183908046%%
Saved to: myword/not-wake-word/generated/pencil sharpening-65.wav
39.6551724137931%%
Saved to: myword/not-wake-word/generated/pencil sharpening-69.wav
48.85057471264368%%
Saved to: myword/not-wake-word/generated/pencil sharpening-85.wav
49.42528735632184%
Saved to: myword/not-wake-word/generated/pencil sharpening-86.wav
67.816091954023%3%%
Saved to: myword/not-wake-word/generated/pencil sharpening-118.wav
70.6896551724138%%
Saved to: myword/not-wake-word/generated/pencil sharpening-123.wav
72.41379310344827%
Saved to: myword/not-wake-word/generated/pencil sharpening-126.wav
72.98850574712644%
Saved to: myword/not-wake-word/generated/pencil sharpening-127.wav
Loading wake-word...
Loading not-wake-word...
Loading wake-word...
Loading not-wake-word...

Epoch 61/61
1/1 [==============================] - ETA: 0s - loss: 1.1070 - accuracy: 0.50002020-12-03 17:15:57.780733: W tensorflow/python/util/util.cc:329] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
1/1 [==============================] - 14s 14s/step - loss: 1.1070 - accuracy: 0.5000 - val_loss: 0.1028 - val_accuracy: 0.8889
Traceback (most recent call last):
  File "/home/pi/mycroft-precise/.venv/bin/precise-train-incremental", line 33, in <module>
    sys.exit(load_entry_point('mycroft-precise', 'console_scripts', 'precise-train-incremental')())
  File "/home/pi/mycroft-precise/precise/scripts/base_script.py", line 49, in run_main
    script.run()
  File "/home/pi/mycroft-precise/precise/scripts/train_incremental.py", line 150, in run
    self.train_on_audio(fn)
  File "/home/pi/mycroft-precise/precise/scripts/train_incremental.py", line 137, in train_on_audio
    self.retrain()
  File "/home/pi/mycroft-precise/precise/scripts/train_incremental.py", line 111, in retrain
    rename(self.args.model + '.h5', self.args.model) # Rename with original
IsADirectoryError: [Errno 21] Is a directory: 'myword.net.h5' -> 'myword.net'

I followed the "method 2" in Reducing False Activations. Downloaded all the files and ran the ffmpeg script on all of them.

This is how my folder looks like after running all of these scripts:

$ ls -l
total 72
drwxr-xr-x 2 pi pi  4096 Dec  3 11:56 computer
drwxr-xr-x 3 pi pi  4096 Dec  3 16:56 data
drwxr-xr-x 2 pi pi  4096 Dec  3 11:56 heycomputer
drwxr-xr-x 5 pi pi  4096 Dec  3 12:21 myword
-rw-r--r-- 1 pi pi     2 Dec  3 17:16 myword.epoch
drwxr-xr-x 4 pi pi  4096 Dec  3 17:16 myword.net
-rw-r--r-- 1 pi pi 44072 Dec  3 17:16 myword.net.h5
-rw-r--r-- 1 pi pi   224 Dec  3 17:15 myword.net.params

[MatthewScholefield]

Hmm, I think I actually fixed that bug on my personal tf2 branch. Could you try checking it out with this?:

git remote add matthew https://github.com/MatthewScholefield/mycroft-precise
git fetch matthew
git checkout feature/tf2

[SandroMiccoli]

After changing to your tf2 branch I left the precise-train-incremental running over night (process started at 23h and finished at 5:30). Now I tried to test the data set and this is the result:

$ precise-test myword.net myword/
Loading wake-word...
Loading not-wake-word...
Traceback (most recent call last):
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/saved_model/loader_impl.py", line 98, in parse_saved_model
    saved_model.ParseFromString(file_content)
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/google/protobuf/message.py", line 199, in ParseFromString
    return self.MergeFromString(serialized)
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/google/protobuf/internal/python_message.py", line 1145, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/google/protobuf/internal/python_message.py", line 1195, in InternalParse
    raise message_mod.DecodeError('Field number 0 is illegal.')
google.protobuf.message.DecodeError: Field number 0 is illegal.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/mycroft-precise/.venv/bin/precise-test", line 33, in <module>    sys.exit(load_entry_point('mycroft-precise', 'console_scripts', 'precise-test')())
  File "/home/pi/mycroft-precise/precise/scripts/base_script.py", line 49, in run_main
    script.run()
  File "/home/pi/mycroft-precise/precise/scripts/test.py", line 52, in run
    predictions = Listener.find_runner(args.model)(args.model).predict(inputs)  File "/home/pi/mycroft-precise/precise/network_runner.py", line 80, in __init__
    self.model = load_precise_model(model_name)
  File "/home/pi/mycroft-precise/precise/model.py", line 55, in load_precise_model
    return load_model(model_name, custom_objects=globals())
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/keras/saving/save.py", line 189, in load_model
    loader_impl.parse_saved_model(filepath)
  File "/home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/saved_model/loader_impl.py", line 101, in parse_saved_model
    raise IOError("Cannot parse file %s: %s." % (path_to_pb, str(e)))
OSError: Cannot parse file b'myword.net/saved_model.pb': Field number 0 is illegal..

[SandroMiccoli]

I saved the output of the precise-train-incremental to a log file, here are the last lines:

...
Starting file data/random/snow_globe_turned_around.wav...
100%
Starting file data/random/STE-028.wav...
100%
Starting file data/random/snow_globe_shaken.wav...
100%
Starting file data/random/electric_roof_window_opening.wav...
100%
Starting file data/random/chirrp.wav...
100%
Starting file data/random/picooz_indoor_helicopter_flight_collage.wav...
100%
Starting file data/random/sliding_door_of_veranda_closing.wav...
100%
Starting file data/random/ritschratsch.wav...
100%
Starting file data/random/planebirdsandkitchen.wav...
100%
Starting file data/random/mnium-mnium-mnium.wav...
100%

Not sure if a specific message should end that script...

[kbromer]

Wanted to confirm I'm seeing the same issue/output on the same hardware/setup as @SandroMiccoli I have not yet tried upgrading the venv to TF2.2, were you able to get the incremental training to complete?

[SandroMiccoli]

Nope, I tried 3 times but on all of them I got the same Field number 0 is illegal. message.

[kbromer]

Had success here. Trained the set with precise-train-incremental. It did error out during its training run (I believe it was due to having an existing model of the same name in the directory) but I was able to successfully retrain after clearing the existing models from previous runs with precise-train. precise-test, precise-listen succeeded as expected. precise-convert also succeeded.

Of course, the app I'm using (Rhasspy) requires .pb/.pb.param files for the integration. Anybody have suggestions on that?

Error from the precise-train-incremental:

100%
Starting file data/random/STE-032.wav...
0.0%Loading wake-word...
Loading not-wake-word...
Loading wake-word...
Loading not-wake-word...

Epoch 61/61
1/1 [==============================] - ETA: 0s - loss: 0.0608 - accuracy: 0.96312020-12-08 16:05:10.570036: W tensorflow/python/util/util.cc:329] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:tensorflow:From /home/pi/mycroft-precise/.venv/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
1/1 [==============================] - 14s 14s/step - loss: 0.0608 - accuracy: 0.9631 - val_loss: 1.2190 - val_accuracy: 0.2000
Traceback (most recent call last):
File "/home/pi/mycroft-precise/.venv/bin/precise-train-incremental", line 33, in
sys.exit(load_entry_point('mycroft-precise', 'console_scripts', 'precise-train-incremental')())
File "/home/pi/mycroft-precise/precise/scripts/base_script.py", line 49, in run_main
script.run()
File "/home/pi/mycroft-precise/precise/scripts/train_incremental.py", line 150, in run
self.train_on_audio(fn)
File "/home/pi/mycroft-precise/precise/scripts/train_incremental.py", line 137, in train_on_audio
self.retrain()
File "/home/pi/mycroft-precise/precise/scripts/train_incremental.py", line 111, in retrain
rename(self.args.model + '.h5', self.args.model) # Rename with original
IsADirectoryError: [Errno 21] Is a directory: 'hey-computer.net.h5' -> 'hey-computer.net'

[kbromer]

I should note: I'm using 141, not @MatthewScholefield repo/branch

[MatthewScholefield]

I haven't looked into field number 0 is illegal yet.

@kbromer Looks like you can set wake.engine.engine_path in your rhasspy config to your source install executable at .venv/bin/precise-engine and then you should be able to use the .tflite model file in place of the .pb.

[kbromer]

Oh wow, I'll try that, thanks!