Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(122)

Side by Side Diff: webrtc/modules/audio_processing/test/py_quality_assessment/README.md

Issue 2805653002: APM-QA tool, renaming noise generators into input-reference generators. (Closed)
Patch Set: final changes Created 3 years, 8 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # APM Quality Assessment tool 1 # APM Quality Assessment tool
2 2
3 Python wrapper of `audioproc_f` with which quality assessment can be 3 Python wrapper of `audioproc_f` with which quality assessment can be
4 automatized. The tool allows to simulate different noise conditions, input 4 automatized. The tool allows to simulate different noise conditions, input
5 signals, APM configurations and it computes different scores. 5 signals, APM configurations and it computes different scores.
6 Once the scores are computed, the results can be easily exported to an HTML page 6 Once the scores are computed, the results can be easily exported to an HTML page
7 which allows to listen to the APM input and output signals and also the 7 which allows to listen to the APM input and output signals and also the
8 reference one used for evaluation. 8 reference one used for evaluation.
9 9
10 ## Dependencies 10 ## Dependencies
(...skipping 28 matching lines...) Expand all
39 - e.g., `$ export POLQA_PATH=/var/opt/PolqaOem64` 39 - e.g., `$ export POLQA_PATH=/var/opt/PolqaOem64`
40 - Deploy the AIR Database and set the `AECHEN_IR_DATABASE_PATH` environment 40 - Deploy the AIR Database and set the `AECHEN_IR_DATABASE_PATH` environment
41 variable 41 variable
42 - e.g., `$ export AECHEN_IR_DATABASE_PATH=/var/opt/AIR_1_4` 42 - e.g., `$ export AECHEN_IR_DATABASE_PATH=/var/opt/AIR_1_4`
43 - Deploy probing signal tracks into 43 - Deploy probing signal tracks into
44 - `out/Default/py_quality_assessment/probing_signals` (*1) 44 - `out/Default/py_quality_assessment/probing_signals` (*1)
45 - Deploy noise tracks into 45 - Deploy noise tracks into
46 - `out/Default/py_quality_assessment/noise_tracks` (*1, *2) 46 - `out/Default/py_quality_assessment/noise_tracks` (*1, *2)
47 47
48 (*1) You can use custom files as long as they are mono tracks sampled at 48kHz 48 (*1) You can use custom files as long as they are mono tracks sampled at 48kHz
49 encoded in the 16 bit signed format (it is recommended that the tracks are 49 encoded in the 16 bit signed format (it is recommended that the tracks are
50 converted and exported with Audacity). 50 converted and exported with Audacity).
51 51
52 (*2) Adapt `EnvironmentalNoiseGenerator._NOISE_TRACKS` accordingly in 52 (*2) Adapt `EnvironmentalNoiseTestDataGenerator._NOISE_TRACKS` accordingly in
53 `out/Default/py_quality_assessment/quality_assessment/noise_generation.py`. 53 `out/Default/py_quality_assessment/quality_assessment/test_data_generation.py`.
54 54
55 ## Usage (scores computation) 55 ## Usage (scores computation)
56 56
57 - Go to `out/Default/py_quality_assessment` 57 - Go to `out/Default/py_quality_assessment`
58 - Check the `apm_quality_assessment.sh` as an example script to parallelize the 58 - Check the `apm_quality_assessment.sh` as an example script to parallelize the
59 experiments 59 experiments
60 - Adjust the script according to your preferences (e.g., output path) 60 - Adjust the script according to your preferences (e.g., output path)
61 - Run `apm_quality_assessment.sh` 61 - Run `apm_quality_assessment.sh`
62 - The script will end by opening the browser and showing ALL the computed 62 - The script will end by opening the browser and showing ALL the computed
63 scores 63 scores
64 64
65 ## Usage (export reports) 65 ## Usage (export reports)
66 66
67 Showing all the results at once can be confusing. You therefore may want to 67 Showing all the results at once can be confusing. You therefore may want to
68 export separate reports. In this case, you can use the 68 export separate reports. In this case, you can use the
69 `apm_quality_assessment_export.py` script as follows: 69 `apm_quality_assessment_export.py` script as follows:
70 70
71 - Set --output_dir to the same value used in `apm_quality_assessment.sh` 71 - Set --output_dir to the same value used in `apm_quality_assessment.sh`
72 - Use regular expressions to select/filter out scores by 72 - Use regular expressions to select/filter out scores by
73 - APM configurations: `--config_names, -c` 73 - APM configurations: `--config_names, -c`
74 - probing signals: `--input_names, -i` 74 - probing signals: `--input_names, -i`
75 - noise generators: `--noise_generators, -n` 75 - test data generators: `--test_data_generators, -t`
76 - scores: `--eval_scores, -e` 76 - scores: `--eval_scores, -e`
77 - Assign a suffix to the report name using `-f <suffix>` 77 - Assign a suffix to the report name using `-f <suffix>`
78 78
79 For instance: 79 For instance:
80 80
81 ``` 81 ```
82 $ ./apm_quality_assessment-export.py \ 82 $ ./apm_quality_assessment-export.py \
83 -o ~/data/apm_quality_assessment \ 83 -o ~/data/apm_quality_assessment \
84 -e \(polqa\) \ 84 -e \(polqa\) \
85 -n \(echo\) \ 85 -n \(echo\) \
(...skipping 12 matching lines...) Expand all
98 Depending on the license, the POLQA tool may take “breaks” as a way to limit the 98 Depending on the license, the POLQA tool may take “breaks” as a way to limit the
99 throughput. When this happens, the APM Quality Assessment tool is slowed down. 99 throughput. When this happens, the APM Quality Assessment tool is slowed down.
100 For more details about this limitation, check Section 10.9.1 in the POLQA manual 100 For more details about this limitation, check Section 10.9.1 in the POLQA manual
101 v.1.18. 101 v.1.18.
102 102
103 In case of issues with the POLQA score computation, check 103 In case of issues with the POLQA score computation, check
104 `py_quality_assessment/eval_scores.py` and adapt 104 `py_quality_assessment/eval_scores.py` and adapt
105 `PolqaScore._parse_output_file()`. 105 `PolqaScore._parse_output_file()`.
106 The code can be also fixed directly into the build directory (namely, 106 The code can be also fixed directly into the build directory (namely,
107 `out/Default/py_quality_assessment/eval_scores.py`). 107 `out/Default/py_quality_assessment/eval_scores.py`).
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698