Overwrite the Elastic Rally track parameters
Each track allows to overwrite the corresponding parameters using –track-params.
For example, the track geonames allows to overwrite the following parameters.
- bulk_size (default: 5000)
- bulk_indexing_clients (default: 8): Number of clients that issue bulk indexing requests.
- ingest_percentage (default: 100): A number between 0 and 100 that defines how much of the document corpus should be ingested.
- conflicts (default: “random”): Type of id conflicts to simulate. Valid values are: ‘sequential’ (A document id is replaced with a document id with a sequentially increasing id), ‘random’ (A document id is replaced with a document id with a random other id).
- conflict_probability (default: 25): A number between 0 and 100 that defines the probability of id conflicts. This requires to run the respective challenge. Combining conflicts=sequential and conflict-probability=0 makes Rally generate index ids by itself, instead of relying on Elasticsearch’s automatic id generation https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#_automatic_id_generation_.
- on_conflict (default: “index”): Whether to use an “index” or an “update” action when simulating an id conflict.
- recency (default: 0): A number between 0 and 1 that defines whether to bias towards more recent ids when simulating conflicts. See the Rally docs for the full definition of this parameter. This requires to run the respective challenge.
- number_of_replicas (default: 0)
- number_of_shards (default: 5)
- max_num_segments: The maximum number of segments to force-merge to.
- source_enabled (default: true): A boolean defining whether the _source field is stored in the index.
- index_settings: A list of index settings. Index settings defined elsewhere (e.g. number_of_replicas) need to be overridden explicitly.
- cluster_health (default: “green”): The minimum required cluster health.
- error_level (default: “non-fatal”): Available for bulk operations only to specify ignore-response-error-level.
The default track parameters can be verified in the index.json.
$ cat .rally/benchmarks/tracks/default/geonames/index.json
{
"settings": {
"index.number_of_shards": {{number_of_shards | default(5)}},
"index.number_of_replicas": {{number_of_replicas | default(0)}},
"index.store.type": "{{store_type | default('fs')}}",
"index.requests.cache.enable": false
},
[..]
}
You can change the track parameters when you run the race as below.
$ esrally race --pipeline=benchmark-only --target-host=10.10.10.1:39200,10.10.10.2:39200,10.10.10.3:39200 --track=geonames --track-params="number_of_shards:3,number_of_replicas:1" --challenge=append-no-conflicts --on-error=abort --race-id=${RACE_ID}
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
[INFO] Race id is [734bb4b3-8b7a-4c0b-9fa6-aaeb4659569f]
[INFO] Racing on track [geonames], challenge [append-no-conflicts] and car ['external'] with version [7.17.0].
[WARNING] indexing_total_time is 19846 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 5970 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index [100% done]
Running create-index [100% done]
Running check-cluster-health [100% done]
Running index-append [100% done]
Running refresh-after-index [100% done]
Running force-merge [100% done]
Running refresh-after-force-merge [100% done]
Running wait-until-merges-finish [100% done]
Running index-stats [100% done]
Running node-stats [100% done]
Running default [100% done]
Running term [100% done]
Running phrase [100% done]
Running country_agg_uncached [100% done]
Running country_agg_cached [100% done]
Running scroll [100% done]
Running expression [100% done]
Running painless_static [100% done]
Running painless_dynamic [100% done]
Running decay_geo_gauss_function_score [100% done]
Running decay_geo_gauss_script_score [100% done]
Running field_value_function_score [100% done]
Running field_value_script_score [100% done]
Running large_terms [100% done]
Running large_filtered_terms [100% done]
Running large_prohibited_terms [100% done]
Running desc_sort_population [100% done]
Running asc_sort_population [100% done]
Running asc_sort_with_after_population [100% done]
Running desc_sort_geonameid [100% done]
Running desc_sort_with_after_geonameid [100% done]
Running asc_sort_geonameid [100% done]
Running asc_sort_with_after_geonameid [100% done]
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
------------------------------------------------------
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------------------:|----------------:|--------:|
| Cumulative indexing time of primary shards | | 14.3313 | min |
| Min cumulative indexing time across primary shards | | 0 | min |
| Median cumulative indexing time across primary shards | | 0.00696667 | min |
| Max cumulative indexing time across primary shards | | 4.85003 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 9.33973 | min |
| Cumulative merge count of primary shards | | 93 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0.00276667 | min |
| Max cumulative merge time across primary shards | | 3.20492 | min |
| Cumulative merge throttle time of primary shards | | 2.39933 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 0.8346 | min |
| Cumulative refresh time of primary shards | | 1.4791 | min |
| Cumulative refresh count of primary shards | | 408 | |
| Min cumulative refresh time across primary shards | | 0 | min |
| Median cumulative refresh time across primary shards | | 0.0191417 | min |
| Max cumulative refresh time across primary shards | | 0.532383 | min |
| Cumulative flush time of primary shards | | 0.143 | min |
| Cumulative flush count of primary shards | | 14 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0.000191667 | min |
| Max cumulative flush time across primary shards | | 0.0622167 | min |
| Total Young Gen GC time | | 30.76 | s |
| Total Young Gen GC count | | 3790 | |
| Total Old Gen GC time | | 7.34 | s |
| Total Old Gen GC count | | 107 | |
| Store size | | 5.79046 | GB |
| Translog size | | 8.19564e-07 | GB |
| Heap used for segments | | 0.452423 | MB |
| Heap used for doc values | | 0.0294952 | MB |
| Heap used for terms | | 0.340546 | MB |
| Heap used for norms | | 0.0440674 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0.0383148 | MB |
| Segment count | | 74 | |
| Total Ingest Pipeline count | | 0 | |
| Total Ingest Pipeline time | | 0 | s |
| Total Ingest Pipeline failed | | 0 | |
| Min Throughput | index-append | 80680.2 | docs/s |
| Mean Throughput | index-append | 81105.1 | docs/s |
| Median Throughput | index-append | 81128.5 | docs/s |
| Max Throughput | index-append | 81286.2 | docs/s |
| 50th percentile latency | index-append | 338.821 | ms |
| 90th percentile latency | index-append | 843.169 | ms |
| 99th percentile latency | index-append | 1132.18 | ms |
| 100th percentile latency | index-append | 1158.98 | ms |
| 50th percentile service time | index-append | 338.821 | ms |
| 90th percentile service time | index-append | 843.169 | ms |
| 99th percentile service time | index-append | 1132.18 | ms |
| 100th percentile service time | index-append | 1158.98 | ms |
| error rate | index-append | 0 | % |
[..]
----------------------------------
[INFO] SUCCESS (took 4314 seconds)
----------------------------------