How do you remove duplicates from a comma separated string in python?

How to Remove Duplicates From a Python list?

In Python, there are numerous methods for removing duplicates from a list. To remove duplicates from a given list, you can make use of these methods and get your work done. Let's take a look at them: 

Method 1 - Naive Method 

To remove duplicates from a list in Python, iterate through the elements of the list and store the first occurrence of an element in a temporary list while ignoring any other occurrences of that element.

The basic approach is implemented in the naive method by:

• Using a For-loop to traverse the list
• If the elements do not already exist in a temporary list, they are added to it
• The temporary list has been assigned to the main list

Example: 

# removing duplicates from the list using naive methods 

# initializing list 

sam_list = [11, 13, 15, 16, 13, 15, 16, 11] 

print ("The list is: " + str(sam_list)) 

# remove duplicates from list

result = [] 

for i in sam_list: 

if i not in result: 

result.append(i) 

# printing list after removal 

print ("The list after removing duplicates : " + str(result))

Output: 

 The list is: [11, 13, 15, 16, 13, 15, 16, 11]

 The list after removing duplicates: [11, 13, 15, 16]

Method 2 - Using List Comprehension

Instead of using the For-loop to implement the Naive method of removing duplicates from a list, we can use Python's List comprehension functionality to do so in just one line of code.

Example:

# removing duplicates from the list using list comprehension 

# initializing list 

sam_list = [11, 13, 15, 16, 13, 15, 16, 11] 

print ("The list is: " + str(sam_list)) 

# to remove duplicates from list 

result = [] 

[result.append(x) for x in sam_list if x not in result] 

# printing list after removal 

print ("The list after removing duplicates: " + str(result))

Output:

The list is: [11, 13, 15, 16, 13, 15, 16, 11]

The list after removing duplicates: [11, 13, 15, 16]

Method 3 - Using set()

This is the most commonly used method for removing a duplicate from a Python list. This is due to the fact that duplicates are not permitted in the set data structure. However, the order of the elements is lost when using this method.

Example: 

# removing duplicates from the list using set() 

# initializing list 

sam_list = [11, 15, 13, 16, 13, 15, 16, 11] 

print ("The list is: " + str(sam_list)) 

# to remove duplicates from list 

sam_list = list(set(sam_list)) 

# printing list after removal 

# ordering distorted

print ("The list after removing duplicates: " + str(sam_list))

Output:

The list is: [11, 15, 13, 16, 13, 15, 16, 11]

The list after removing duplicates: [16, 11, 13, 15]

Method 4 - Using list comprehension + enumerate ()

We find the distinct elements and store them in a temporary list using the List comprehension method. When we use the List comprehension in conjunction with the enumerate() function, the programme checks for previously encountered elements and skips adding them to the temporary list. The enumerate function takes an iterable as an argument and returns it as an enumerating object (index, element), which adds a counter to each iterable element.

Example:

# removing duplicates from the list using list comprehension + enumerate() 

# initializing list 

sam_list = [11, 15, 13, 16, 13, 15, 16, 11] 

print ("The list is: " + str(sam_list)) 

# to remove duplicates from list 

result = [i for n, i in enumerate(sam_list) if i not in sam_list[:n]] 

# printing list after removal 

print ("The list after removing duplicates: " + str(result))

Output:

The list is: [11, 13, 15, 16, 13, 15, 16, 11]

The list after removing duplicates: [11, 13, 15, 16]

Method 5 - Using collections.OrderedDict.fromkeys()

This is the quickest way to accomplish the goal of removing duplicates from the Python list. This method first removes duplicates before returning a dictionary that has been converted to a list. In addition, this method works well with strings.

Example:

# removing duplicates from list using collections.OrderedDict.fromkeys() 

from collections import OrderedDict 

# initializing list 

sam_list = [11, 15, 13, 16, 13, 15, 16, 11] 

print ("The list is: " + str(sam_list)) 

# to remove duplicates from list 

result = list(OrderedDict.fromkeys(sam_list)) 

# printing list after removal 

print ("The list after removing duplicates: " + str(result))

Output:

The list is: [11, 15, 13, 16, 13, 15, 16, 11]

The list after removing duplicates: [11, 15, 13, 16]

Learn data operations in Python, strings, conditional statements, error handling, and the commonly used Python web framework Django. Check out Simplilearn's Python Training course.