For fun I adapted my answer to another somewhat related question to make it do what you want. Note that currently it only changes the formatting of a
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
7 if it's in a list.import _ctypes
import json
import re
class OneDictPerLine[object]:
def __init__[self, value]:
self.value = value
def __repr__[self]:
if not isinstance[self.value, list]:
return repr[self.value]
else: # Sort the representation of any dicts in the list.
reps = ['{{{}}}'.format[', '.join[
['{!r}: {}'.format[k, v] for k, v in sorted[v.items[]]]
]] if isinstance[v, dict]
else
repr[v] for v in self.value]
return '[' + ',\n'.join[reps] + ']'
def di[obj_id]:
""" Reverse of id[] function. """
# from //stackoverflow.com/a/15012814/355230
return _ctypes.PyObj_FromPtr[obj_id]
class MyEncoder[json.JSONEncoder]:
FORMAT_SPEC = "@@{}@@"
regex = re.compile[FORMAT_SPEC.format[r"[\d+]"]]
def default[self, obj]:
return [self.FORMAT_SPEC.format[id[obj]] if isinstance[obj, OneDictPerLine]
else super[MyEncoder, self].default[obj]]
def encode[self, obj]:
format_spec = self.FORMAT_SPEC # Local var to expedite access.
json_repr = super[MyEncoder, self].encode[obj] # Default JSON repr.
# Replace any marked-up object ids in the JSON repr with the value
# returned from the repr[] of the corresponding Python object.
for match in self.regex.finditer[json_repr]:
id = int[match.group[1]]
# Replace marked-up id with actual Python object repr[].
json_repr = json_repr.replace[
'"{}"'.format[format_spec.format[id]], repr[di[id]]]
return json_repr
Sample usage:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
Output:
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
json.dump[s] & json.load[s]
bogotobogo.com site search:
json.load[s] & json.dump[s]
There are two ways of reading in [load/loads] the following json file, in.json:
{"alpha": 1, "beta": 2}
- string:
import json io = open["in.json","r"] string = io.read[] # json.loads[str] dictionary = json.loads[string] # or one-liner # dictionary = json.loads[open["in.json","r"].read[]] print[dictionary]
- dictionary:
import json # json.load[_io] io = open["in.json","r"] dictionary = json.load[io] # or one-liner # dictionary = json.load[open["in.json","r"]] print[dictionary]
Both will print out:
{'alpha': 1, 'beta': 2}
Note that while the json.loads[] requires string, load[s,...], the json.load[] requires file descriptor, load[fp...].
Similarly, we can write a [dump/dumps] json file:
- string:
import json d = {'alpha': 1, 'beta': 2} s = json.dumps[d] open["out.json","w"].write[s]
- dictionary:
import json d = {'alpha': 1, 'beta': 2} json.dump[d, open["out.json","w"]]
Note that the json.dump[] requires file descriptor as well as an obj, dump[obj, fp...].
In the following example, we'll convert Python dictionary to JSON and write it to a text file. Then, we'll read in back from the file and play with it.
Initially we'll construct Python dictionary like this:
# Four Fundamental Forces with JSON d = {} d ["gravity"] = { "mediator":"gravitons", "relative strength" : "1", "range" : "infinity" } d ["weak"] = { "mediator":"W/Z bosons", "relative strength" : "10^25", "range" : "10^-18" } d ["electromagnetic"] = { "mediator":"photons", "relative strength" : "10^36", "range" : "infinity" } d ["strong"] = { "mediator":"gluons", "relative strength" : "10^38", "range" : "10^-15" } print[d]
The output looks like this:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
0Now, we want to convert the dictionary to a string using json.dumps:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
1Output:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
2Note that the "json.dumps[]" returns a string as indicated by the "s" at the end of "dumps". This process is called encoding.
Let's write it to a file:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
3Now that the file is written. Let's reads it back and decoding the JSON-encoded string back into a Python dictionary data structure:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
4Let's play with the dictionary a little bit.
What's the relative strength of electromagnetic compared to gravity?
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
5Who's the mediator for "strong" force?
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
6Ok, here is the full code:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
7If we prefer working with files instead of strings, we may want to use json.dump[] / json.load[] to encode / decode JSON data using the data from the previous example:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
8 Here is another example [json.dump[]/json.load[]] using simpler data:
# Sample usage
data = [
{"key01":"value","key02":"value"},
{"key11":"value","key12":"value"},
{"key21":"value","key22":"value"},
{'key{:02d}:"value"'.format[k] for k in range[100]}
]
print[json.dumps[OneDictPerLine[data], cls=MyEncoder]]
9 Usage for string version: json.loads[]/json.dumps[]:
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
0 The following example sends a syslog to logstash fargate containers behind AWS NLB:
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
1note that for the HOST, we can also use FQDN instead of the NLB's domain name.
Also, as usual, instead of the long line of code, we may want to use a simple linux command,
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
8:[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
2If the NLB listener protocol is TLS, we can use
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
9 echo to the TLS NLB:[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
3 Another example: AWS API response.
When we make an AWS API call, the response can be an invalid json due to datetime:
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
4We need to serialize it [ow to overcome “datetime.datetime not JSON serializable”?].
Here is a boto3 code for an API call to EC2 describe:
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
5The r.json with jq looks like this:
[{'key01': value, 'key02': value},
{'key11': value, 'key12': value},
{'key21': value, 'key22': value},
{'key93:"value"', 'key05:"value"', 'key00:"value"', 'key33:"value"', 'key55:"value"', 'key91:"value"', 'key18:"value"', 'key76:"value"', 'key25:"value"', 'key72:"value"', 'key21:"value"', 'key54:"value"', 'key12:"value"', 'key61:"value"', 'key96:"value"', 'key87:"value"', 'key71:"value"', 'key03:"value"', 'key66:"value"', 'key58:"value"', 'key85:"value"', 'key11:"value"', 'key64:"value"', 'key75:"value"', 'key27:"value"', 'key86:"value"', 'key29:"value"', 'key31:"value"', 'key69:"value"', 'key15:"value"', 'key62:"value"', 'key45:"value"', 'key49:"value"', 'key40:"value"', 'key39:"value"', 'key78:"value"', 'key98:"value"', 'key28:"value"', 'key19:"value"', 'key42:"value"', 'key60:"value"', 'key04:"value"', 'key84:"value"', 'key56:"value"', 'key83:"value"', 'key10:"value"', 'key34:"value"', 'key77:"value"', 'key80:"value"', 'key68:"value"', 'key99:"value"', 'key38:"value"', 'key67:"value"', 'key59:"value"', 'key52:"value"', 'key57:"value"', 'key23:"value"', 'key14:"value"', 'key26:"value"', 'key90:"value"', 'key09:"value"', 'key07:"value"', 'key35:"value"', 'key73:"value"', 'key41:"value"', 'key17:"value"', 'key48:"value"', 'key44:"value"', 'key82:"value"', 'key65:"value"', 'key47:"value"', 'key95:"value"', 'key88:"value"', 'key97:"value"', 'key63:"value"', 'key22:"value"', 'key51:"value"', 'key50:"value"', 'key36:"value"', 'key06:"value"', 'key30:"value"', 'key32:"value"', 'key08:"value"', 'key79:"value"', 'key89:"value"', 'key20:"value"', 'key70:"value"', 'key46:"value"', 'key94:"value"', 'key53:"value"', 'key92:"value"', 'key81:"value"', 'key13:"value"', 'key43:"value"', 'key24:"value"', 'key16:"value"', 'key02:"value"', 'key74:"value"', 'key01:"value"', 'key37:"value"'}]
6
Python tutorial
Python Home
Introduction
Running Python Programs [os, sys, import]
Modules and IDLE [Import, Reload, exec]
Object Types - Numbers, Strings, and None
Strings
- Escape Sequence, Raw String, and Slicing
Strings - Methods
Formatting Strings - expressions and method calls
Files and os.path
Traversing directories recursively
Subprocess Module
Regular Expressions with Python
Regular Expressions Cheat Sheet
Object Types - Lists
Object Types - Dictionaries and Tuples
Functions
def, *args, **kargs
Functions lambda
Built-in Functions
map, filter, and reduce
Decorators
List Comprehension
Sets [union/intersection] and itertools - Jaccard coefficient and shingling to check plagiarism
Hashing [Hash tables and hashlib]
Dictionary Comprehension with zip
The yield keyword
Generator Functions and Expressions
generator.send[] method
Iterators
Classes and Instances [__init__, __call__, etc.]
if__name__ == '__main__'
argparse
Exceptions
@static method vs class method
Private attributes and private methods
bits, bytes, bitstring, and constBitStream
json.dump[s] and json.load[s]
Python Object Serialization - pickle and json
Python Object Serialization - yaml and json
Priority queue and heap queue data structure
Graph data structure
Dijkstra's shortest path algorithm
Prim's spanning tree algorithm
Closure
Functional programming in Python
Remote running a local
file using ssh
SQLite 3 - A. Connecting to DB, create/drop table, and insert data into a table
SQLite 3 - B. Selecting, updating and deleting data
MongoDB with PyMongo I - Installing MongoDB ...
Python HTTP Web Services - urllib, httplib2
Web scraping with Selenium for checking domain
availability
REST API : Http Requests for Humans with Flask
Blog app with Tornado
Multithreading ...
Python Network Programming I - Basic Server / Client : A Basics
Python Network Programming I - Basic Server / Client : B File Transfer
Python Network Programming II - Chat Server / Client
Python Network Programming III - Echo Server using socketserver network framework
Python Network Programming IV - Asynchronous Request Handling : ThreadingMixIn and ForkingMixIn
Python Coding Questions I
Python Coding Questions II
Python Coding Questions III
Python Coding Questions IV
Python
Coding Questions V
Python Coding Questions VI
Python Coding Questions VII
Python Coding Questions VIII
Image processing with Python image library Pillow
Python and C++ with SIP
PyDev with Eclipse
Matplotlib
Redis with Python
NumPy array basics A
NumPy Matrix and Linear Algebra
Pandas with NumPy and Matplotlib
Celluar Automata
Batch gradient descent algorithm
Longest Common Substring Algorithm
Python Unit Test - TDD using unittest.TestCase class
Simple tool - Google page ranking by keywords
Google App Hello World
Google App webapp2 and WSGI
Uploading Google App Hello World
Python 2 vs Python 3
virtualenv and virtualenvwrapper
Uploading a big file to AWS S3 using boto module
Scheduled stopping and starting an AWS instance
Cloudera CDH5 - Scheduled stopping and starting services
Removing Cloud Files - Rackspace API with curl and
subprocess
Checking if a process is running/hanging and stop/run a scheduled task on Windows
Apache Spark 1.3 with PySpark [Spark Python API] Shell
Apache Spark 1.2 Streaming
bottle 0.12.7 - Fast and simple WSGI-micro framework for small web-applications ...
Flask app with Apache WSGI on
Ubuntu14/CentOS7 ...
Fabric - streamlining the use of SSH for application deployment
Ansible Quick Preview - Setting up web servers with Nginx, configure enviroments, and deploy an App
Neural Networks with backpropagation for XOR using one hidden layer
NLP - NLTK [Natural Language Toolkit] ...
RabbitMQ[Message broker server]
and Celery[Task queue] ...
OpenCV3 and Matplotlib ...
Simple tool - Concatenating slides using FFmpeg ...
iPython - Signal Processing with NumPy
iPython and Jupyter - Install Jupyter, iPython Notebook, drawing with Matplotlib, and publishing it to Github
iPython and Jupyter Notebook with Embedded D3.js
Downloading YouTube videos using youtube-dl embedded with Python
Machine Learning : scikit-learn ...
Django 1.6/1.8 Web Framework ...