php删除文本文件中重复行的方法

本文实例讲述了php删除文本文件中重复行的方法。分享给大家供大家参考。具体分析如下：

这个php函数用来删除文件中的重复行，还可以指定是否忽略大小写，和指定换行符

/** * RemoveDuplicatedLines * This function removes all duplicated lines of the given text file. * * @param   string * @param   bool * @return  string */function RemoveDuplicatedLines($Filepath, $IgnoreCase=false, $NewLine=""){  if (!file_exists($Filepath)){    $ErrorMsg = "RemoveDuplicatedLines error: ";    $ErrorMsg .= "The given file " . $Filepath . " does not exist!";    die($ErrorMsg);  }  $Content = file_get_contents($Filepath);  $Content = RemoveDuplicatedLinesByString($Content, $IgnoreCase, $NewLine);  // Is the file writeable?  if (!is_writeable($Filepath)){    $ErrorMsg = "RemoveDuplicatedLines error: ";    $ErrorMsg .= "The given file " . $Filepath . " is not writeable!";      die($ErrorMsg);  }  // Write the new file  $FileResource = fopen($Filepath, "w+");     fwrite($FileResource, $Content);      fclose($FileResource);  } /** * RemoveDuplicatedLinesByString * This function removes all duplicated lines of the given string. * * @param   string * @param   bool * @return  string */function RemoveDuplicatedLinesByString($Lines, $IgnoreCase=false, $NewLine=""){  if (is_array($Lines))    $Lines = implode($NewLine, $Lines);  $Lines = explode($NewLine, $Lines);  $LineArray = array();  $Duplicates = 0;  // Go trough all lines of the given file  for ($Line=0; $Line < count($Lines); $Line++){    // Trim whitespace for the current line    $CurrentLine = trim($Lines[$Line]);    // Skip empty lines    if ($CurrentLine == "")      continue;    // Use the line contents as array key    $LineKey = $CurrentLine;    if ($IgnoreCase)      $LineKey = strtolower($LineKey);    // Check if the array key already exists,    // if not add it otherwise increase the counter    if (!isset($LineArray[$LineKey]))      $LineArray[$LineKey] = $CurrentLine;        else              $Duplicates++;  }  // Sort the array  asort($LineArray);  // Return how many lines got removed  return implode($NewLine, array_values($LineArray));  }

使用范例：

// Example 1// Removes all duplicated lines of the file definied in the first parameter.$RemovedLinesCount = RemoveDuplicatedLines("test.txt");print "Removed $RemovedLinesCount duplicate lines from the test.txt file.";// Example 2 (Ignore case)// Same as above, just ignores the line case.RemoveDuplicatedLines("test.txt", true);// Example 3 (Custom new line character)// By using the 3rd parameter you can define which character// should be used as new line indicator. In this case// the example file looks like "foo;bar;foo;foo" and will// be replaced with "foo;bar" RemoveDuplicatedLines("test.txt", false, ";");

希望本文所述对大家的php程序设计有所帮助。