Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(36)

Side by Side Diff: webrtc/modules/audio_processing/test/py_quality_assessment/README.md

Issue 2813883002: APM QA refactoring: render stream support, echo path simulation, new export engine (Closed)
Patch Set: merge Created 3 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # APM Quality Assessment tool 1 # APM Quality Assessment tool
2 2
3 Python wrapper of `audioproc_f` with which quality assessment can be 3 Python wrapper of `audioproc_f` with which quality assessment can be
4 automatized. The tool allows to simulate different noise conditions, input 4 automatized. The tool allows to simulate different noise conditions, input
5 signals, APM configurations and it computes different scores. 5 signals, APM configurations and it computes different scores.
6 Once the scores are computed, the results can be easily exported to an HTML page 6 Once the scores are computed, the results can be easily exported to an HTML page
7 which allows to listen to the APM input and output signals and also the 7 which allows to listen to the APM input and output signals and also the
8 reference one used for evaluation. 8 reference one used for evaluation.
9 9
10 ## Dependencies 10 ## Dependencies
11 11
12 - OS: Linux 12 - OS: Linux
13 - Python 2.7 13 - Python 2.7
14 - Python libraries: numpy, scipy, pydub (0.17.0+) 14 - Python libraries: numpy, scipy, pydub (0.17.0+), pandas (0.20.1+)
15 - It is recommended that a dedicated Python environment is used 15 - It is recommended that a dedicated Python environment is used
16 - install `virtualenv` 16 - install `virtualenv`
17 - `$ sudo apt-get install python-virtualenv` 17 - `$ sudo apt-get install python-virtualenv`
18 - setup a new Python environment (e.g., `my_env`) 18 - setup a new Python environment (e.g., `my_env`)
19 - `$ cd ~ && virtualenv my_env` 19 - `$ cd ~ && virtualenv my_env`
20 - activate the new Python environment 20 - activate the new Python environment
21 - `$ source ~/my_env/bin/activate` 21 - `$ source ~/my_env/bin/activate`
22 - add dependcies via `pip` 22 - add dependcies via `pip`
23 - `(my_env)$ pip install numpy pydub scipy` 23 - `(my_env)$ pip install numpy pydub scipy pandas`
24 - PolqaOem64 (see http://www.polqa.info/) 24 - PolqaOem64 (see http://www.polqa.info/)
25 - Tested with POLQA Library v1.180 / P863 v2.400 25 - Tested with POLQA Library v1.180 / P863 v2.400
26 - Aachen Impulse Response (AIR) Database 26 - Aachen Impulse Response (AIR) Database
27 - Download https://www2.iks.rwth-aachen.de/air/air_database_release_1_4.zip 27 - Download https://www2.iks.rwth-aachen.de/air/air_database_release_1_4.zip
28 - Input probing signals and noise tracks (you can make your own dataset - *1) 28 - Input probing signals and noise tracks (you can make your own dataset - *1)
29 29
30 ## Build 30 ## Build
31 31
32 - Compile WebRTC 32 - Compile WebRTC
33 - Go to `out/Default/py_quality_assessment` and check that 33 - Go to `out/Default/py_quality_assessment` and check that
(...skipping 27 matching lines...) Expand all
61 - Run `apm_quality_assessment.sh` 61 - Run `apm_quality_assessment.sh`
62 - The script will end by opening the browser and showing ALL the computed 62 - The script will end by opening the browser and showing ALL the computed
63 scores 63 scores
64 64
65 ## Usage (export reports) 65 ## Usage (export reports)
66 66
67 Showing all the results at once can be confusing. You therefore may want to 67 Showing all the results at once can be confusing. You therefore may want to
68 export separate reports. In this case, you can use the 68 export separate reports. In this case, you can use the
69 `apm_quality_assessment_export.py` script as follows: 69 `apm_quality_assessment_export.py` script as follows:
70 70
71 - Set --output_dir to the same value used in `apm_quality_assessment.sh` 71 - Set `--output_dir, -o` to the same value used in `apm_quality_assessment.sh`
72 - Use regular expressions to select/filter out scores by 72 - Use regular expressions to select/filter out scores by
73 - APM configurations: `--config_names, -c` 73 - APM configurations: `--config_names, -c`
74 - probing signals: `--input_names, -i` 74 - capture signals: `--capture_names, -i`
75 - render signals: `--render_names, -r`
76 - echo simulator: `--echo_simulator_names, -e`
75 - test data generators: `--test_data_generators, -t` 77 - test data generators: `--test_data_generators, -t`
76 - scores: `--eval_scores, -e` 78 - scores: `--eval_scores, -s`
77 - Assign a suffix to the report name using `-f <suffix>` 79 - Assign a suffix to the report name using `-f <suffix>`
78 80
79 For instance: 81 For instance:
80 82
81 ``` 83 ```
82 $ ./apm_quality_assessment-export.py \ 84 $ ./apm_quality_assessment-export.py \
83 -o ~/data/apm_quality_assessment \ 85 -o output/ \
84 -e \(polqa\) \
85 -n \(echo\) \
86 -c "(^default$)|(.*AE.*)" \ 86 -c "(^default$)|(.*AE.*)" \
87 -t \(white_noise\) \
88 -s \(polqa\) \
87 -f echo 89 -f echo
88 ``` 90 ```
89 91
90 ## Troubleshooting 92 ## Troubleshooting
91 93
92 The input wav file must be: 94 The input wav file must be:
93 - sampled at a sample rate that is a multiple of 100 (required by POLQA) 95 - sampled at a sample rate that is a multiple of 100 (required by POLQA)
94 - in the 16 bit format (required by `audioproc_f`) 96 - in the 16 bit format (required by `audioproc_f`)
95 - encoded in the Microsoft WAV signed 16 bit PCM format (Audacity default 97 - encoded in the Microsoft WAV signed 16 bit PCM format (Audacity default
96 when exporting) 98 when exporting)
97 99
98 Depending on the license, the POLQA tool may take “breaks” as a way to limit the 100 Depending on the license, the POLQA tool may take “breaks” as a way to limit the
99 throughput. When this happens, the APM Quality Assessment tool is slowed down. 101 throughput. When this happens, the APM Quality Assessment tool is slowed down.
100 For more details about this limitation, check Section 10.9.1 in the POLQA manual 102 For more details about this limitation, check Section 10.9.1 in the POLQA manual
101 v.1.18. 103 v.1.18.
102 104
103 In case of issues with the POLQA score computation, check 105 In case of issues with the POLQA score computation, check
104 `py_quality_assessment/eval_scores.py` and adapt 106 `py_quality_assessment/eval_scores.py` and adapt
105 `PolqaScore._parse_output_file()`. 107 `PolqaScore._parse_output_file()`.
106 The code can be also fixed directly into the build directory (namely, 108 The code can be also fixed directly into the build directory (namely,
107 `out/Default/py_quality_assessment/eval_scores.py`). 109 `out/Default/py_quality_assessment/eval_scores.py`).
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698