uniq

The uniq utility shall read an input file comparing adjacent lines, and write one copy of each input line on the output. The second and succeeding copies of repeated adjacent input lines shall not be written. The trailing of each line in the input shall be ignored when doing comparisons. Repeated lines in the input shall not be detected if they are not adjacent. Precede each output line with a count of the number of times the line occurred in the input. Suppress the writing of lines that are not repeated in the input. Ignore the first fields fields on each input line when doing comparisons, where fields is a positive decimal integer. A field is the maximal string matched by the basic regular expression: If the fields option-argument specifies more fields than appear on an input line, a null string shall be used for comparison. Ignore the first chars characters when doing comparisons, where chars shall be a positive decimal integer. If specified in conjunction with the -f option, the first chars characters after the first fields fields shall be ignored. If the chars option-argument specifies more characters than remain on an input line, a null string shall be used for comparison. Provide a default value for the internationalization variables that are unset or null. (See XBD Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.) If set to a non-empty string value, override the values of all the other internationalization variables. Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files) and which characters constitute a in the current locale. Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error. [XSI) ] ![[Option Start]]( Determine the location of message catalogs for the processing of LC MESSAGES. ![[Option End]]( The standard output shall be used if no output file operand is specified, and shall be used if the output file operand is '-' and the implementation treats the '-' as meaning standard output. Otherwise, the standard output shall not be used. See the OUTPUT FILES section. If the collating sequence of the current locale has a total ordering of all characters, the sort utility can be used to cause repeated lines to be adjacent in the input file. If the collating sequence does not have a total ordering of all characters, the sort utility should still do this but it might not. To ensure that all duplicate lines are eliminated, and have the output sorted according the collating sequence of the current locale, applications should use: 1. This first example tests the line counting option, comparing each line of the input file data starting from the second field: The number '2', prefixing the fifth line of output, signifies that the uniq utility detected a pair of repeated lines. Given the input data, this can only be true when uniq is run using the -f 1 option (which shall cause uniq to ignore the first field on each input line). 2. The second example tests the option to suppress unique lines, comparing each line of the input file data starting from the second field: 3. This test suppresses repeated lines, comparing each line of the input file data starting from the second field: 4. This suppresses unique lines, comparing each line of the input file data starting from the third character: In the last example, the uniq utility found no input matching the above criteria. Some historical implementations have limited lines to be 1080 bytes in length, which does not meet the implied {LINE MAX} limit. Earlier versions of this standard allowed the - number and + number options. These options are no longer specified by POSIX.1-2017 but may be present in some implementations. The normative text is reworded to avoid use of the term "must"