How to remove duplicate values from list in python

In Python, we make use of a list, when we need to store a collection of data. The thing is that in the list, we can have data of multiple data types, and along with that, duplicate elements are also allowed in the list.

But at times, there can be a requirement, that we need to remove the duplicate elements from the list. So, in this article, we are going to have a look at how can we remove duplicate elements from the list in python. There can be different ways to achieve this goal, and we are going to have a look at some of the different and simple ways to do this.
Have a look at some example input outputs, which would make us more clear about the task –

Example:
Input: [1, 2, 1, 1, 3, 45]
Output: [1, 2, 3, 45]

As you can see, we have a list as input, which contains the duplicate elements, and as an output, we get a list, with the duplicate elements removed. So, now let’s have a look at what different things can we do, to remove the duplicate values from the list.

How to remove duplicate values from list in Python

Way 1 – Manually adding an element to another list, if it is not already present

Here, we are going to have a list, which may contain some duplicate elements, and we are going to create another list, in which, we are going to append the element from the given list, only if they are not already present in the list. The thing is that here, we need an extra list, so it would consume some more memory, but let’s have a look at the program, which tries to demonstrate the same thing.

list1 = [1, 2, 1, 1, 45, 2, 3]
list2 = []
for number in list1:
    if number not in list2:
        list2.append(number)
print(list2)

As you can see in the above program, we have a list, which contains the duplicate elements, and then we have created another list, which is initially empty. After that, we are looping through list 1, and we are adding the element from list 1 to list 2, only if it is not already present in list 2. So doing this, we just avoid adding the element which is already present in the list, again to the list.

After we are done looping, we are printing list 2 as our output list. In this list, we do not have any duplicate elements.
You can try to execute the above program and try to have a look at the output of the program. You can also try changing the values for the list.

Way 2 – Using list comprehension

In the first method, we had another list, in which, we were adding the element, only if that element is not already present in the list. But here, we are going to use list comprehension, to create another list, which would not contain duplicate items. Using list comprehension, we can just specify what we are required to do, in just one line of code. Let’s have a look at the program now –

list1 = [1, 2, 1, 1, 45, 2, 3]
list2 = []
[list2.append(number) for number in list1 if number not in list2]
print(list2)

As you can see in the above program, we are just having the list, and we have also created another list, and after that, we are specifying what we are required to do in a single line. So, if you try to execute the above program, you can find that the new list now does not have duplicate elements.

Way 3 – Converting our list to set, removing all duplicates, and then converting the set back to list.

This way is a very simple way to remove duplicate elements from the list. In this method, we just need to convert our list to a set. The thing is that set is another built-in datatype in python, which is used to store the collection of data. In the set, we can only have unique elements. So, when we convert our list to a set, we get a set, in which, there are no duplicate elements.

But now, we have a set, and we need a list, so we can convert the set back to the list, and so, we have a list, with all the duplicate elements removed from the list. Let’s have a look at the program now, which tries to demonstrate the same thing.

list1 = [1, 2, 1, 1, 45, 2, 3]
set1 = set(list1) # We get a set, with all the duplicates removed
list1 = list(set1) # converting to list again
print(list1)

As you can see in the above program, we have a list, and then we are trying to convert the list to a set. With this, we have a set, and all the duplicate elements from the set are removed. After that, we are going to convert the set back to the list. So with this, we have a list, with all the duplicate elements removed from the list.

Conclusion

In this article, we have seen some simple methods to remove duplicate elements from the set. There can be some more methods to achieve this task, but we have seen some simple methods to remove duplicate elements from the list.

FAQ About How to remove duplicate values from list in Python