jq parsing failed if not starting with . ( dot ) #570
Description
Description
When parsing large API response I got error
/opt/newrelic-infra/newrelic-integrations/bin/nri-flex --verbose --pretty --config_file /tmp/test.yml
...
ERRO[0000] jq: failed to parse api= error="unexpected token \"{\""
...Expected Behavior
to get jq parsed and values added to output
{
"flowInputThroughput": 4123,
"openFD": 309
}
Troubleshooting or NR Diag results
Provide any other relevant log data.
TIP: Scrub logs and diagnostic information for sensitive information
Steps to Reproduce
config file:/tmp/test.yml
integrations:
- name: nri-flex
interval: 60s
config:
name: logstashStats
apis:
- event_type: LogStashNodeStatSample
#url: http://127.0.0.1:9600/_node/stats
commands:
- run: cat /tmp/example.json
jq: .
- event_type: LogStashNodeStatProcesSample
#url: http://127.0.0.1:9600/_node/stats
commands:
- run: cat /tmp/example.json
jq: |
.process | {
openFD: .open_file_descriptors
}
- event_type: LogStashNodeStatLimitedSample
#url: http://127.0.0.1:9600/_node/stats
commands:
- run: cat /tmp/example.json
jq: |
{
openFD: .process.open_file_descriptors,
flowInputThroughput: .flow.input_throughput.current
}json data: /tmp/example.json
{
"flow": {
"filter_throughput": {
"current": 4116,
"last_15_minutes": 5227,
"last_1_hour": 4839,
"last_1_minute": 5361,
"last_24_hours": 4175,
"last_5_minutes": 5169,
"lifetime": 4107
},
"input_throughput": {
"current": 4123,
"last_15_minutes": 5227,
"last_1_hour": 4839,
"last_1_minute": 5359,
"last_24_hours": 4175,
"last_5_minutes": 5166,
"lifetime": 4107
},
"output_throughput": {
"current": 4116,
"last_15_minutes": 5227,
"last_1_hour": 4839,
"last_1_minute": 5361,
"last_24_hours": 4175,
"last_5_minutes": 5169,
"lifetime": 4107
},
"queue_backpressure": {
"current": 0.1115,
"last_15_minutes": 0.2143,
"last_1_hour": 0.1869,
"last_1_minute": 0.1691,
"last_24_hours": 0.1293,
"last_5_minutes": 0.1916,
"lifetime": 0.1263
},
"worker_concurrency": {
"current": 1.111,
"last_15_minutes": 1.241,
"last_1_hour": 1.129,
"last_1_minute": 1.601,
"last_24_hours": 1.406,
"last_5_minutes": 1.315,
"lifetime": 1.216
}
},
"process": {
"cpu": {
"load_average": {
"15m": 2.51,
"1m": 1.28,
"5m": 2.05
},
"percent": 12,
"total_in_millis": 81757710
},
"max_file_descriptors": 16384,
"mem": {
"total_virtual_in_bytes": 14300155904
},
"open_file_descriptors": 309,
"peak_open_file_descriptors": 337
}
}command:
DEBU[0000] Function.isAvailable: enter
DEBU[0000] Function.isAvailable: exit status: false
INFO[0000] com.newrelic.nri-flex GOARCH=amd64 GOOS=linux version=1.16.5
DEBU[0000] config: git sync configuration not set
WARN[0000] config: testing agent config, agent features will not be available
DEBU[0000] config: running async name=logstashStats
DEBU[0000] config: processing apis apis=3 name=logstashStats
DEBU[0000] fetch: collect data name=logstashStats
DEBU[0000] commands: executing count=1 name=logstashStats
DEBU[0000] processor-data: running data handler name=logstashStats DEBU[0000] fetch: collect data name=logstashStats DEBU[0000] commands: executing count=1 name=logstashStats DEBU[0000] processor-data: running data handler name=logstashStats DEBU[0000] fetch: collect data name=logstashStats DEBU[0000] commands: executing count=1 name=logstashStats DEBU[0000] processor-data: running data handler name=logstashStats ERRO[0000] jq: failed to parse api= error="unexpected token \"{\"" DEBU[0000] config: finished variable processing apis apis=3 name=logstashStats INFO[0000] flex: completed processing configs configs=1
{
"name": "com.newrelic.nri-flex",
"protocol_version": "3",
"integration_version": "1.16.5",
"data": [
{
"metrics": [
{
"event_type": "LogStashNodeStatSample",
"flow.filter_throughput.current": 4116,
"flow.filter_throughput.last_15_minutes": 5227,
"flow.filter_throughput.last_1_hour": 4839,
"flow.filter_throughput.last_1_minute": 5361,
"flow.filter_throughput.last_24_hours": 4175,
"flow.filter_throughput.last_5_minutes": 5169,
"flow.filter_throughput.lifetime": 4107,
"flow.input_throughput.current": 4123,
"flow.input_throughput.last_15_minutes": 5227,
"flow.input_throughput.last_1_hour": 4839,
"flow.input_throughput.last_1_minute": 5359,
"flow.input_throughput.last_24_hours": 4175,
"flow.input_throughput.last_5_minutes": 5166,
"flow.input_throughput.lifetime": 4107,
"flow.output_throughput.current": 4116,
"flow.output_throughput.last_15_minutes": 5227,
"flow.output_throughput.last_1_hour": 4839,
"flow.output_throughput.last_1_minute": 5361,
"flow.output_throughput.last_24_hours": 4175,
"flow.output_throughput.last_5_minutes": 5169,
"flow.output_throughput.lifetime": 4107,
"flow.queue_backpressure.current": 0.1115,
"flow.queue_backpressure.last_15_minutes": 0.2143,
"flow.queue_backpressure.last_1_hour": 0.1869,
"flow.queue_backpressure.last_1_minute": 0.1691,
"flow.queue_backpressure.last_24_hours": 0.1293,
"flow.queue_backpressure.last_5_minutes": 0.1916,
"flow.queue_backpressure.lifetime": 0.1263,
"flow.worker_concurrency.current": 1.111,
"flow.worker_concurrency.last_15_minutes": 1.241,
"flow.worker_concurrency.last_1_hour": 1.129,
"flow.worker_concurrency.last_1_minute": 1.601,
"flow.worker_concurrency.last_24_hours": 1.406,
"flow.worker_concurrency.last_5_minutes": 1.315,
"flow.worker_concurrency.lifetime": 1.216,
"integration_name": "com.newrelic.nri-flex",
"integration_version": "1.16.5",
"process.cpu.load_average.15m": 2.51,
"process.cpu.load_average.1m": 1.28,
"process.cpu.load_average.5m": 2.05,
"process.cpu.percent": 12,
"process.cpu.total_in_millis": 81757710,
"process.max_file_descriptors": 16384,
"process.mem.total_virtual_in_bytes": 14300155904,
"process.open_file_descriptors": 309,
"process.peak_open_file_descriptors": 337
},
{
"event_type": "LogStashNodeStatProcesSample",
"integration_name": "com.newrelic.nri-flex",
"integration_version": "1.16.5",
"openFD": 309
},
{
"event_type": "flexStatusSample",
"flex.Hostname": "XXXX",
"flex.IntegrationVersion": "1.16.5",
"flex.counter.ConfigsProcessed": 1,
"flex.counter.EventCount": 2,
"flex.counter.EventDropCount": 0,
"flex.counter.LogStashNodeStatSample": 1,
"flex.counter.LogStashNodeStatSampleOpenFD": 1,
"flex.time.elapsedMs": 23,
"flex.time.endMs": 1748588436601,
"flex.time.startMs": 1748588436578
}
],
"inventory": {},
"events": []
}
]
}also I tested all jq parsing using jq and gojq commands:
$ cat /tmp/example.json |jq '
{
openFD: .process.open_file_descriptors,
flowInputThroughput: .flow.input_throughput.current
}'
{
"openFD": 309,
"flowInputThroughput": 4123
}
$ cat /tmp/example.json | gojq_v0.12.16_linux_amd64/gojq '
{
openFD: .process.open_file_descriptors,
flowInputThroughput: .flow.input_throughput.current
}'also tested with . | { and .. as $parent | $parent | {
Your Environment
OS: Red Hat Enterprise Linux 8
NewRelic Infra: 1.64.0
Additional context
Add any other context about the problem here. For example, relevant community posts or support tickets.
For Maintainers Only or Hero Triaging this bug
Suggested Priority (P1,P2,P3,P4,P5):
Suggested T-Shirt size (S, M, L, XL, Unknown):