Is There A Way To Execute Jq From Python
Solution 1:
From the jq FAQ:
Q: What bindings are available for Python?
A:
pip install jq # For details, see https://pypi.python.org/pypi/jq
pip install pyjq # For details, see https://pypi.python.org/pypi/pyjq
As for your nested array, looping through it sounds like something that can (and maybe should) be done within jq.
Solution 2:
I believe the accepted answer should be peak's one, as the proper way to use a C api in python is via a python binding lib, and both https://pypi.python.org/pypi/jq and https://pypi.python.org/pypi/pyjq should work as expected.
That being said, since we are talking about python, I would like to bring an alternative that is much more pythonic: glom (pip install glom
, https://glom.readthedocs.io/)
Instead of using a DSL like in jq
, with glom
you just declare the output in the format you want, using pure python (this output format is called spec
). In this case, you want a simple dict
:
spec = {'Name': 'name',
'street': 'address.0.1.street'}
and then just call glom on your data:
output_data = glom(input_data, spec)
Just like jq
, you can also use glom
on command line:
cat filename.json | glom "{'Name': 'name', 'street': 'address.0.1.street'}"
A complete python example:
import json
from pprint import pprint
from glom import glom
with open('filename.json', 'rt') as f:
input_data = json.load(f)
spec = {'Name': 'name',
'street': 'address.0.1.street'}
output_data = glom(input_data, spec)
pprint(output_data)
Solution 3:
The sh
module makes it easy invoke a jq subprocess from python. e.g.
import sh
cmd = sh.jq('-M', '{Name:.name, address:.address[0][1].street}', 'filename.json')
print "cmd returned >>%s<<" % cmd.stdout
Solution 4:
- Can I execute this command from a python script
Yes, using subprocess. Example:
jsonFile = '/path/to/your/filename.json'
jq_cmd = "/bin/jq '{Name:.name, address:.address[0][1].street}' " + jsonFile
jq_proc = subprocess.Popen(jq_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
# read JSON object, convert to string, store as a dictionary
jDict = json.loads(jq_proc.stdout.read())
jq_proc.stdout.close()
- If it can be done, then how would I loop through for the nested array elements in the sample data give above (address[][].street)
It would help to see a JSON data set with a few records. For looping through JSON sets in python with jq, it is easy to get a count of the objects and then iterate. A slight bit of overhead but it makes the code easy to understand.
# count number of JSON records from the root level
jq_cmd = "/bin/jq '. | length' " + jsonFile
jq_proc = subprocess.Popen(jq_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
jObjCount = int(jq_proc.stdout.read())
jq_proc.stdout.close()
# iterate over each root level JSON record
for ix in range(jObjCount):
jq_cmd = "jq '. | .[" + str(ix) + "]' " + jsonFile
jq_proc = subprocess.Popen(jq_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
# read object, convert to string, store as a python dictionary
jDict = json.loads(jq_proc.stdout.read())
# iterate over nested objects within a root level object
# as before, count number items but here for each root level JSON object
jq_cmd = "/bin/jq '. | .[" + str(ix) + "].sub_Item_Key | length' " + jsonFile
jq_proc = subprocess.Popen(jq_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
jItemCount = int(jq_proc.stdout.read())
jq_proc.stdout.close()
for jx in range(jItemCount):
jq_cmd = "/bin/jq '. | .[" + str(ix) + "].sub_Item_Key[" + str(jx) + "]' " + jsonFile
jq_proc = subprocess.Popen(jq_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
# read JSON item, convert to string, store as a python dictionary
jItemDict = json.loads(jq_proc.stdout.read())
Enjoy!
Solution 5:
Yes. Using plumbum.
from plumbum.cmd import jq, cat
(cat["filename.json"] | jq ["{Name:.name, address:.address[0][1].street}"])()
The result of the above command is a JSON object, that can be parsed to a Python object using json.loads
.
You might also be interested in jello, which is like jq
but uses Python as query language.
Post a Comment for "Is There A Way To Execute Jq From Python"