To remove duplicate lines in the first file1.txt and output the results to the second file.

uniq myfile1.txt > myfile2.txt

syntax :

uniq [option] filename

The options of uniq command are:

• c : Count of occurrence of each line.
• d : Prints only duplicate lines.
• D : Print all duplicate lines
• f : Avoid comparing first N fields.
• i : Ignore case when comparing.
• s : Avoid comparing first N characters.
• u : Prints only unique lines.
• w : Compare no more than N characters in lines

The default behavior of the uniq command is to suppress the 

duplicate line. Note that, you have to pass sorted input to 

the uniq, as it compares only successive lines. 

If the lines in the file are not in sorted order, then use the 

sort command and then pipe the output to the uniq command. 

> sort example.txt | uniq