If using a third-party package would be okay then you could use iteration_utilities.unique_everseen
:
>>> from iteration_utilities import unique_everseen
>>> l = [{'a': 123}, {'b': 123}, {'a': 123}]
>>> list[unique_everseen[l]]
[{'a': 123}, {'b': 123}]
It preserves the order of the original list and ut can also handle unhashable items like dictionaries by falling back on a slower algorithm [O[n*m]
where n
are the elements in the original list and m
the unique elements in the original list instead of O[n]
]. In case both keys and values are hashable you can use the key
argument of that function to create hashable items
for the "uniqueness-test" [so that it works in O[n]
].
In the case of a dictionary [which compares independent of order] you need to map it to another data-structure that compares like that, for example frozenset
:
>>> list[unique_everseen[l, key=lambda item: frozenset[item.items[]]]]
[{'a': 123}, {'b': 123}]
Note that you shouldn't use a simple tuple
approach [without sorting] because equal dictionaries don't necessarily have the same order [even in Python 3.7 where insertion order - not absolute order - is guaranteed]:
>>> d1 = {1: 1, 9: 9}
>>> d2 = {9: 9, 1: 1}
>>> d1 == d2
True
>>> tuple[d1.items[]] == tuple[d2.items[]]
False
And even sorting the tuple might not work if the keys aren't sortable:
>>> d3 = {1: 1, 'a': 'a'}
>>> tuple[sorted[d3.items[]]]
TypeError: '