Məlumat Mərkəzi

  1. Ana səhifə
  2. Məlumat Mərkəzi
  3. Scripts
  4. Linux - Delete lines with duplicate data

Linux - Delete lines with duplicate data

If for example you dont want the same email address to be in your .csv files you would run the following command to
awk '!x[$2]++' FS="," FILENAME.csv > newname.csv

FILENAME.csv needs to be replaced with your .csv file and [$2] needs to be edited to represent the line you want to check for duplicate content.

Bu mövzu kifayət qədər köməkçi oldu mu?

 Çap

Digər Başlıqlar

Linux - Only keep certain country data

< FileToExtract.csv awk 'BEGIN {FS=","} tolower($19)~/usa|united states/' > USAList.txtThis...

Linux - How to extract .gz files (gunzip)

For this guide we will be using gunzip. You may need to installed it.CentOS - yum install gunzip...

Linux - How to delete .csv file containing "X" data

After extracting the .csv.gz files you can delete certain unwanted data by running the following...

Linux - How to split large file

For this example we will split a large .csv into multiple files each "X" megabytes in size....

Linux - How to combine multiple .csv files

To combine multiple files with the letters "2014" and "2015" in their name run the following...