Jq iterating array objects to create new json objects

▲
0
▼

I've been thinking and searching for a long time, but I didn't find out what I'm looking for. I'm using JQ to parse tshark (-ek) json output, but I'm a jq newby When a frame is multivalue I have a JSON similar to this:

 {
      "timestamp": "1525627021656",
      "layers": {
        "frame_time_epoch": [
          "1525627021.656417000"
        ],
        "ip_src": [
          "10.10.10.10"
        ],
        "ip_src_host": [
          "test"
        ],
        "ip_dst": [
          "10.10.10.11"
        ],
        "ip_dst_host": [
          "dest_test"
        ],
        "diameter_Event-Timestamp": [
          "May  6, 2018 19:17:02.000000000 CEST",
          "May  6, 2018 19:17:02.000000000 CEST"
        ],
        "diameter_Origin-Host": [
          "TESTHOST",
          "TESTHOST"
        ],
        "diameter_Destination-Host": [
          "DESTHOST",
          "DESTHOST"
        ],
        "diameter_CC-Request-Type": [
          "2",
          "2"
        ],
        "diameter_CC-Request-Number": [
          "10",
          "3"
        ],
        "diameter_Rating-Group": [
          "9004",
          "9001"
        ],
        "diameter_Called-Station-Id": [
          "testing",
          "testing"
        ],
        "diameter_User-Name": [
          "testuser",
          "testuser"
        ],
        "diameter_Subscription-Id-Data": [
          "66666666666",
          "77777777777"
        ],
        "gtp_qos_version": [
          "0x00000008",
          "0x00000005"
        ],
        "gtp_qos_max_dl": [
          "8640",
          "42"
        ],
        "diameter_Session-Id": [
          "test1;sessionID1;test1",
          "test2;sessionID2;test2"
        ]
      }
    }

As you can see, many keys are array and I want to iterate them to create different json objects in a result like this:

{
    "frame_time_epoch": [
      "1525627021.656417000"
    ],
    "ip_src": [
      "10.10.10.10"
    ],
    "ip_src_host": [
      "test"
    ],
    "ip_dst": [
      "10.10.10.11"
    ],
    "ip_dst_host": [
      "dest_test"
    ],
    "diameter_Event-Timestamp": [
      "May  6, 2018 19:17:02.000000000 CEST"
    ],
    "diameter_Origin-Host": [
      "TESTHOST"
    ],
    "diameter_Destination-Host": [
      "DESTHOST"
    ],
    "diameter_CC-Request-Type": [
      "2"
    ],
    "diameter_CC-Request-Number": [
      "3"
    ],
    "diameter_Rating-Group": [
      "9001"
    ],
    "diameter_Called-Station-Id": [
      "testing"
    ],
    "diameter_User-Name": [
      "testuser"
    ],
    "diameter_Subscription-Id-Data": [
      "77777777777"
    ],
    "gtp_qos_version": [
      "0x00000005"
    ],
    "gtp_qos_max_dl": [
      "42"
    ],
    "diameter_Session-Id": [
      "test2;sessionID2;test2"
    ]
  }
 {
    "frame_time_epoch": [
      "1525627021.656417000"
    ],
    "ip_src": [
      "10.10.10.10"
    ],
    "ip_src_host": [
      "test"
    ],
    "ip_dst": [
      "10.10.10.11"
    ],
    "ip_dst_host": [
      "dest_test"
    ],
    "diameter_Event-Timestamp": [
      "May  6, 2018 19:17:02.000000000 CEST"
    ],
    "diameter_Origin-Host": [
      "TESTHOST"
    ],
    "diameter_Destination-Host": [
      "DESTHOST"
    ],
    "diameter_CC-Request-Type": [
      "2"
    ],
    "diameter_CC-Request-Number": [
      "10"
    ],
    "diameter_Rating-Group": [
      "9004"
    ],
    "diameter_Called-Station-Id": [
      "testing"
    ],
    "diameter_User-Name": [
      "testuser"
    ],
    "diameter_Subscription-Id-Data": [
      "66666666666"
    ],
    "gtp_qos_version": [
      "0x00000008"
    ],
    "gtp_qos_max_dl": [
      "8640"
    ],
    "diameter_Session-Id": [
      "test1;sessionID1;test1"
    ]
  }

Another hand made example: INPUT:

{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value1" , "value2"],
    "any_key_name": ["value4" ,"value5"]
}
{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value6" , "value7", "value8"],
    "any_key_name": ["value9" ,"value10" , "value11"]
}

Desired output:

{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value1"],
    "any_key_name": ["value4"],
}
{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value2"],
    "any_key_name": ["value5"],
}
{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value6"],
    "any_key_name": ["value9"],
}
{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value7"],
    "any_key_name": ["value10"],
}
{
    "key_single": ["single_value"],
    "key2": ["single_value"],
    "multiple_value_key": ["value8"],
    "any_key_name": ["value11"],
}

Could you help Me?

Thanks in advance.

json
slice
jq
asked on Stack Overflow May 9, 2018 by Roberto • edited May 22, 2018 by peak

2 Answers

▲
1
▼

It looks like you want to take, in turn, the i-th element of the selected arrays. Using your second example, this could be done like so:

range(0; .multiple_value_key|length) as $i
| . + { multiple_value_key: [.multiple_value_key[$i]],
        any_key_name:       [.any_key_name[$i]] }

The output in compact form:

{"key_single":["single_value"],"key2":["single_value"],"multiple_value_key":["value1"],"any_key_name":["value4"]}
{"key_single":["single_value"],"key2":["single_value"],"multiple_value_key":["value2"],"any_key_name":["value5"]}
{"key_single":["single_value"],"key2":["single_value"],"multiple_value_key":["value6"],"any_key_name":["value9"]}
{"key_single":["single_value"],"key2":["single_value"],"multiple_value_key":["value7"],"any_key_name":["value10"]}
{"key_single":["single_value"],"key2":["single_value"],"multiple_value_key":["value8"],"any_key_name":["value11"]}
answered on Stack Overflow May 9, 2018 by peak
▲
0
▼

Here is a simple solution to the problem as described in the "comments", though the output differs slightly from that shown in the Q.

For clarity, a helper function is defined for producing the $i-th slice of an object, that is, for all array-valued keys with array-length greater than 1, the value is replaced by the $i-th item in the array.

def slice($i):
    map_values(if (type == "array" and length>1)
               then [.[$i]]
               else . end);

The solution is then simply:

.layers
| range(0;  [.[] | length] | max) as $i
| slice($i)

Output

{
  "frame_time_epoch": [
    "1525627021.656417000"
  ],
  "ip_src": [
    "10.10.10.10"
  ],
  "ip_src_host": [
    "test"
  ],
  "ip_dst": [
    "10.10.10.11"
  ],
  "ip_dst_host": [
    "dest_test"
  ],
  "diameter_Event-Timestamp": [
    "May  6, 2018 19:17:02.000000000 CEST"
  ],
  "diameter_Origin-Host": [
    "TESTHOST"
  ],
  "diameter_Destination-Host": [
    "DESTHOST"
  ],
  "diameter_CC-Request-Type": [
    "2"
  ],
  "diameter_CC-Request-Number": [
    "10"
  ],
  "diameter_Rating-Group": [
    "9004"
  ],
  "diameter_Called-Station-Id": [
    "testing"
  ],
  "diameter_User-Name": [
    "testuser"
  ],
  "diameter_Subscription-Id-Data": [
    "66666666666"
  ],
  "gtp_qos_version": [
    "0x00000008"
  ],
  "gtp_qos_max_dl": [
    "8640"
  ],
  "diameter_Session-Id": [
    "test1;sessionID1;test1"
  ]
}
{
  "frame_time_epoch": [
    "1525627021.656417000"
  ],
  "ip_src": [
    "10.10.10.10"
  ],
  "ip_src_host": [
    "test"
  ],
  "ip_dst": [
    "10.10.10.11"
  ],
  "ip_dst_host": [
    "dest_test"
  ],
  "diameter_Event-Timestamp": [
    "May  6, 2018 19:17:02.000000000 CEST"
  ],
  "diameter_Origin-Host": [
    "TESTHOST"
  ],
  "diameter_Destination-Host": [
    "DESTHOST"
  ],
  "diameter_CC-Request-Type": [
    "2"
  ],
  "diameter_CC-Request-Number": [
    "3"
  ],
  "diameter_Rating-Group": [
    "9001"
  ],
  "diameter_Called-Station-Id": [
    "testing"
  ],
  "diameter_User-Name": [
    "testuser"
  ],
  "diameter_Subscription-Id-Data": [
    "77777777777"
  ],
  "gtp_qos_version": [
    "0x00000005"
  ],
  "gtp_qos_max_dl": [
    "42"
  ],
  "diameter_Session-Id": [
    "test2;sessionID2;test2"
  ]
}
answered on Stack Overflow May 12, 2018 by peak • edited May 12, 2018 by peak

User contributions licensed under CC BY-SA 3.0