Linux学习之Perl-One-Liners-Explained系列

perl 单行命令解释

Perl One-Liners Explained, Part I: File Spacing

Hi all! I am starting yet another article series here. Remember my two articles on Awk One-Liners Explained and Sed One-Liners Explained? They have received more than 150,000 views total now and they attract several thousand new visitors every week. Inspired by their success, I am going to create my own perl1line.txt file and explain every single oneliner in it. I hope it becomes as popular as awk1line.txt and sed1line.txt.

A tiny intro to those unfamiliar with the awk1line.txt and sed1line.txt files – they contain around 80 useful awk and sed one-liners for doing various text file manipulations. Some examples are double spacing files, numbering lines, reversing text, and replacing certain words. They were all compiled by Eric Pement.

The article will be divided in seven or more parts. In parts 1 - 6 I’ll create the one-liners and explain them, and in the last part I will release the perl1line.txt file. I found that splitting the article in many parts is much easier to get it written and that’s what I will do.

Here is the general plan:

Part I: File spacing (this part).
Part II: Line Numbering.
Part III: Calculations.
Part IV: String creation. Array creation.
Part V: Text conversion and substitution.
Part VI: Selective printing and deleting of certain lines.
Part VII: Handy regular expressions.
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

These one-liners will make heavy use of Perl special variables. Luckily, a few years ago I compiled all the special vars in a single file and called it Perl special variable cheat-sheet. Even tho it’s mostly copied out of perldoc perlvar, it’s still handy to have in front of you. I suggest that you print it so that you always had them in front of you and you could memorize them faster.

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

Other than that, I can’t wait to start writing the article, so here I go:
File Spacing

Double space a file.
1
perl -pe '$\="\n"'
This one-liner double spaces a file. There are three things to explain in this one-liner. The “-p” and “-e” command line options, and the “$" variable.

First let’s start with the “-e” option. The “-e” option can be used to enter a Perl program directly in the command line. Typically you don’t want to create source files for every small program. By using “-e” you can handily specify the program to execute on the command line.

Next the “-p” switch. Specifying “-p” to a Perl program causes it to assume the following loop around your program:

while (<>) {
    # your program goes here
} continue {
    print or die &#34;-p failed: $!\n&#34;;
}

This construct loops over all the input, executes your code and prints the value of “$”. This way you can effectively modify all or some lines of input. The “$” variable can be explained as an anonymous variable that gets filled with the good stuff.

The “$" variable is similar to ORS in Awk. It gets appended after every “print” operation. Without any arguments “print” prints the contents of “$_” (the good stuff).

In this one-liner the code specified by “-e” is ‘$=“\n”‘, thus the whole program looks like this:

while (<>) {
    $\ = &#34;\n&#34;;
} continue {
    print or die &#34;-p failed: $!\n&#34;;
}

What happens is that after reading each line, “$” gets filled with it (including the existing line’s newline), the “$" gets set to a newline itself and “print” is called. As I already mentioned, without any arguments “print” prints the contents of “$” and “$" gets appended at the end. The result is that each line gets printed unmodified and it’s followed by the “$" which has been set to newline. The input has been double-spaced.

There is actually no need to set “$" to newline on each line. It was just the shortest possible one-liner that double-spaced the file. Here are several others that do the same:

1	perl -pe 'BEGIN { $\="\n" }'

This one sets the “$" to newline just once before Perl does anything (BEGIN block gets executed before everything else).

1	perl -pe '$_ .= "\n";'

This one-liner is equivalent to:

while (<>) {
    $_ = $_ . &#34;\n&#34;
} continue {
    print or die &#34;-p failed: $!\n&#34;;
}

It appends another new-line at the end of each line, then prints it out.

The cleanest and coolest way to do it is probably use the substitution “s///“ operator:

1	perl -pe 's/$/\n/'

It replaces the regular expression “$” that matches at the end of line with a newline, effectively adding a newline at the end.

Double space a file, except the blank lines.
1
perl -pe '$_ .= "\n" unless /^$/'
This one-liner double spaces all lines that are not completely empty. It does it by appending a newline character at the end of each line that is not blank. The “unless” means “if not”. And “unless /^$/“ means “if not ‘beginning is end of line’”. The condition “beginning is end of line” is true only for lines that contain the newline character.

Here is how it looks when expanded:

while (<>) {
    if ($_ !~ /^$/) {
        $_ .= "\n";
    }
} continue {
    print or die &#34;-p failed: $!\n&#34;;
}

A better test that takes spaces and tabs on the line into account is this one:

1	perl -pe '$_ .= "\n" if /\S/'

Here the line is matched against “\S”. “\S” is a regular expression that is the inverse of “\s”. The inverse of “\s” is any non-whitespace character. The result is that every line that has at least one non-whitespace character (tab, vertical tab, space, etc) gets double spaced.

Triple space a file.
1
perl -pe '$\="\n\n"'
Or
1
perl -pe '$_.="\n\n"'
They are the same as one-liner #1, except that two new-lines get appended after each line.
N-space a file.
1
perl -pe '$_.="\n"x7'
This one-liner uses inserts 7 new lines after each line. Notice how I used ‘“\n” x 7’ to repeat the newline char 7 times. The “x” operator repeats the thing on the left N times.

For example:

1	perl -e 'print "foo"x5'

would print “foofoofoofoofoo”.

Add a blank line before every line.
1
perl -pe 's//\n/'
This one-liner uses the “s/pattern/replacement/“ operator. It substitutes the first pattern (regular expression) in the “$_” variable with the replacement. In this one-liner the pattern is empty, meaning it matches any position between chars (and in this case it’s the position before first char) and replaces it with “\n”. The effect is that a newline char gets inserted before the line.
Remove all blank lines.
1
perl -ne 'print unless /^$/'
This one-liner uses “-n” flag. The “-n” flag causes Perl to assume to following loop around your program:
1
2
3
4
LINE:
while (<>) {
# your program goes here
}
What happens here is that each line gets read by the diamond “<>” operator and is placed in the dolar underscore special variable “$_”. At this moment you are free to do with it whatever you want. You specify that in your main program text.

In this one-liner the main program is “print unless /^$/“, it gets inserted in the while loop above and the whole Perl program becomes:

LINE:
while (<>) {
    print unless /^$/
}

Unraveling it further:

LINE:
while (<>) {
    print $_ unless $_ =~ /^$/
}

This one-liner prints all non-blank lines.

A few other ways to do the same:

perl -lne ‘print if length’

This one uses the “-l” command line argument. The “-l” automatically chomps the input line (basically gets rid of newline at the end). Next the line is tested for its length. If there are any chars left, it evals to true and the line gets printed.

1	perl -ne 'print if /\S/'

This one-liner behaves differently than the “print unless /^$/“ one-liner. The “print unless /^$/“ one-liner prints lines with spaces and tabs, this one doesn’t.

Remove all consecutive blank lines, leaving just one.
1
perl -00 -pe ''
Ok, this is really tricky, ain’t it? First of all it does not have any code, the -e is empty. Next it has a silly -00 command line option. This command line option turns paragraph slurp mode on. A paragraph is text between two newlines. All the other newlines get ignored. The paragraph gets put in “$_” and the “-p” option prints it out.

Later I found a shorter version of this one-liner:

1	perl -00pe0

Compress/expand all blank lines into N consecutive ones.
1
perl -00 -pe '$_.="\n"x4'
This one-liner combines the previous one and one-liner #4. It slurps lines paragraph wise, then appends (N-1) new-line. This one-liner expands (or compresses) all new-lines to 5 (“\n” x 4 prints four, and there was one at the end of paragraph itself, so 5).

Perl One-Liners Explained, Part II: Line Numbering

This is the second part of a seven-part article on Perl one-liners. In this part I will create various one-liners for line numbering. See part one for introduction of the series.

Perl one-liners is my attempt to create “perl1line.txt” that is similar to “awk1line.txt” and “sed1line.txt” that have been so popular among Awk and Sed programmers.

The article on Perl one-liners will consist of at least seven parts:

Part I: File spacing.
Part II: Line numbering (this part).
Part III: Calculations.
Part IV: String creation. Array creation.
Part V: Text conversion and substitution.
Part VI: Selective printing and deleting of certain lines.
Part VII: Handy regular expressions.
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

The one-liners will make heavy use of Perl special variables. A few years ago I compiled all the Perl special variables in a single file and called it Perl special variable cheat-sheet. Even tho it’s mostly copied out of perldoc perlvar, it’s still handy to have in front of you. Print it!

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

And here are today’s one-liners:
Line Numbering

Number all lines in a file.
1
perl -pe '$_ = "$. $_"'
As I explained in the first one-liner, “-p” causes Perl to assume a loop around the program (specified by “-e”) that reads each line of input into the “ $ “ variable, executes the program and then prints the “ $ “ variable.

In this one-liner I simply modify “ $_ “ and prepend the “ $. “ variable to it. The special variable “ $. “ contains the current line number of input.

The result is that each line gets its line number prepended.

Number only non-empty lines in a file.
1
perl -pe '$_ = ++$a." $_" if /./'
Here we employ the “action if condition” statement that executes “action” only if “condition” is true. In this case the condition is a regular expression “/./“, which matches any character except newline (that is, it matches a non-empty line); and the action is “ $ = ++$a.” $” “, which prepends variable “ $a “ incremented by one to the current line. As we didn’t use strict pragma, $a was created automatically.

The result is that at each non-empty line “ $a “ gets incremented by one and prepended to that line. And at each empty line nothing gets modified and the empty line gets printed as is.

Number and print only non-empty lines in a file (drop empty lines).
1
perl -ne 'print ++$a." $_" if /./'
This one-liner uses the “-n” program argument that places the line in “ $_ “ variable and then executes the program specified by “-e”. Unlike “-p”, it does not print the line after executing code in “-e”, so we have to call “print” explicitly to get it printed.

The one-liner calls “print” only on lines that have at least one character in them. And exactly like in the previous one-liner, it increments the line number in variable “ $a “ by one for each non-empty line.

The empty lines simply get ignored and never get printed.

Number all lines but print line numbers only non-empty lines.
1
perl -pe '$_ = "$. $_" if /./'
This one-liner is similar to one-liner #10. Here I modify the “ $_ “ variable that holds the entire line only if the line has at least one character. All other lines (empty ones) get printed without line numbers.
Number only lines that match a pattern, print others unmodified.
1
perl -pe '$_ = ++$a." $_" if /regex/'
Here we again use the “action if condition” statement but the condition in this case is a pattern (regular expression) “/regex/“. The action is the same as in one-liner #10. I don’t want to repeat, see #10 for explanation.
Number and print only lines that match a pattern.
1
perl -ne 'print ++$a." $_" if /regex/'
This one-liner is almost exactly like #11. The only difference is that it prints numbered lines that match only “/regex/“.
Number all lines, but print line numbers only for lines that match a pattern.
1
perl -pe '$_ = "$. $_" if /regex/'
This one-liner is similar to the previous one-liner and to one-liner #12. Here the line gets its line number prepended if it matches a /regex/, otherwise it just gets printed without a line number.
Number all lines in a file using a custom format (emulate cat -n).
1
perl -ne 'printf "%-5d %s", $., $_'
This one-liner uses the formatted print “printf” function to print the line number together with line. In this particular example the line numbers are left aligned on 5 char boundary.

Some other nice format strings are “%5d” that right-aligns line numbers on 5 char boundary and “%05d” that zero-fills and right-justifies the line numbers.

Here my Perl printf cheat sheet might come handy that lists all the possible format specifiers.

Print the total number of lines in a file (emulate wc -l).
1
perl -lne 'END { print $. }'
This one-liner uses the “END” block that Perl probably took as a feature from Awk language. The END block gets executed after the program has executed. In this case the program is the hidden loop over the input that was created by the “-n” argument. After it has looped over the input, the special variable “ $. “ contains the number of lines there was in the input. The END block prints this variable. The “ -l “ parameter sets the output record separator for “print” to a newline (so that we didn’t have to print “$.\n”).

Another way to do the same is:

1	perl -le 'print $n=()=<>'

This is a tricky one, but easy to understand if you know about Perl contexts. In this one-liner the “ ()=<> “ part causes the <> operator (the diamond operator) to evaluate in list context, that causes the diamond operator to read the whole file in a list. Next, “ $n “ gets evaluated in scalar context. Evaluating a list in a scalar context returns the number of elements in the list. Thus the “ $n=()=<> “ construction is equal to the number of lines in the input, that is number of lines in the file. The print statement prints this number out. The “ -l “ argument makes sure a newline gets added after printing out this number.

This is the same as writing the following, except longer:

1	perl -le 'print scalar(()=<>)'

And completely obvious version:

1	perl -le 'print scalar(@foo=<>)'

Yet another way to do it:

1	perl -ne '}{print $.'

This one-liner uses the eskimo operator “}{“ in conjunction with “-n” command line argument. As I explained in one-liner #11, the “-n” argument forces Perl to assume a “ while(<>) { } “ loop around the program. The eskimo operator forces Perl to escape the loop, and the program turns out to be:

while (<>) {
}{                    # eskimo operator here
    print $.;
}

It’s easy to see that this program just loops over all the input and after it’s done doing so, it prints the “ $. “, which is the number of lines in the input.

Print the number of non-empty lines in a file.
1
perl -le 'print scalar(grep{/./}<>)'
This one-liner uses the “grep” function that is similar to the grep Unix command. Given a list of values, “ grep {condition} “ returns only those values that match condition. In this case the condition is a regular expression that matches at least one character, so the input gets filtered and the “grep{/./}” returns all lines that were non empty. To get the number of characters we evaluate the list in scalar context and print the result. (As I mentioned in the previous one-liner list in scalar context evaluates to number of elements in the list).

A golfer’s version of this one-liner would be to replace “scalar()” with “ ~~ “ (double bitwise negate), thus it can be shortened:

1	perl -le 'print ~~grep{/./}<>'

This can be made even shorter:

1	perl -le 'print~~grep/./,<>'

Print the number of empty lines in a file.
1
perl -lne '$a++ if /^$/; END {print $a+0}'
Here I use variable “ $a “ to count how many empty lines have I encountered. Once I have finished looping over all the lines, I print the value of $a in the END block. I use “ $a+0 “ construction to make sure “ 0 “ gets output if no lines were empty.

I could have also modified the previous one-liner:

1	perl -le 'print scalar(grep{/^$/}<>)'

Or written it with “ ~~ “:

1	perl -le 'print ~~grep{/^$/}<>'

These last two versions are not as effective, as they would read the whole file in memory. Where as the first one would do it line by line.

Print the number of lines in a file that match a pattern (emulate grep -c).
1
perl -lne '$a++ if /regex/; END {print $a+0}'
This one-liner is basically the same as the previous one, except it increments the line counter $a by one in case a line matches a regular expression /regex/.

Perl One-Liners Explained, Part III: Calculations

This is the third part of a seven-part article on Perl one-liners. In this part I will create various one-liners for calculations. See part one for introduction of the series.

Perl one-liners is my attempt to create “perl1line.txt” that is similar to “awk1line.txt” and “sed1line.txt” that have been so popular among Awk and Sed programmers.

The article on Perl one-liners will consist of at least seven parts:

Part I: File spacing.
Part II: Line numbering.
Part III: Calculations (this part).
Part IV: String creation. Array creation.
Part V: Text conversion and substitution.
Part VI: Selective printing and deleting of certain lines.
Part VII: Handy regular expressions.
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

After I’m done explaining all these one-liners, I’ll publish an ebook. Subscribe to my blog to know when that happens!

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

And here are today’s one-liners:
Calculations

Check if a number is a prime.
1
perl -lne '(1x$_) !~ /^1?$|^(11+?)\1+$/ && print "$_ is prime"'
This one-liner uses an ingenious regular expression to detect if a given number is a prime or not. Don’t take it too seriously, though. I included it for its artistic value.

First, the number is converted in its unary representation by “ (1x$_) “. For example, 5 gets converted into “ 1x5 “, which is “ 11111 “.

Next, the unary number gets tested against the ingenious regular expression. If it doesn’t match, the number is a prime, otherwise it’s a composite.

The regular expression works this way. It consists of two parts “ ^1?$ “ and “ ^(11+?)\1+$ “.

The first part matches “ 1 “ and empty string. Clearly, empty string and 1 are not prime numbers, therefore this regular expression matches, which indicated that they are not prime numbers.

The second part determines if two or more 1s repeatedly make up the whole number. If two or mores 1s repeatedly make up the whole number, the regex matches, which means that the number is composite. Otherwise it’s a prime.

Let’s look at the second regex part on numbers 5 and 6.

The number 5 in unary representation is “ 11111 “. The “ (11+?) “ matches first two ones “ 11 “. The back-reference “ \1 “ becomes “ 11 “ and the whole regex now becomes “ ^11(11)+$ “. It can’t match five ones, therefore it fails. But since it used “ +? “, it backtracks and matches the first three ones “ 111 “. The back-reference becomes “ 111 “ and the whole regex becomes “ ^111(111)+$ “. It doesn’t match again. This repeats for “ 1111 “ and “ 11111 “, which also don’t match, therefore the whole regex doesn’t match and the number is a prime.

The number 4 in unary representation is “ 1111 “. The “ (11+?) “ matches the first two ones “ 11 “. The back-reference “ \1 “ becomes “ 11 “ and the regex becomes “ ^11(11)+$ “. It matches the original string, therefore the number is not a prime.

The “ -lne “ command line options have been explained in parts one and two.

Print the sum of all the fields on a line.
1
perl -MList::Util=sum -alne 'print sum @F'
This one-liner turns on field auto-splitting with “ -a “ command line option and imports the “sum” function from “List::Util” module with “ -MList::Util=sum “ option. The “List::Util” is in the Perl core so you don’t need to worry about installing it.

As a result of auto-splitting the split fields end up in the “ @F “ array and the “ sum “ function just sums them up.

The -Mmodule=arg option imports arg from module and is the same as writing:

1	use module qw(arg)

This one-liner is equivalent to the following:

use List::Util qw(sum);
while (<>) {
    @F = split(&#39; &#39;);
    print sum @F, &#34;\n&#34;;
}

Print the sum of all the fields on all lines.
1
perl -MList::Util=sum -alne 'push @S,@F; END { print sum @S }'
This one-liner keeps pushing the split fields in “ @F “ to the “ @S “ array. Once the input is over and perl is about quit, END { } block gets called that outputs the sum of all items in @F. This sum is the sum of all fields over all lines.

This solution isn’t too good - it creates a massive array @S. A better solution is to keep just the running:

1	perl -MList::Util=sum -alne '$s += sum @F; END { print $s }'

Shuffle all fields on a line.
1
perl -MList::Util=shuffle -alne 'print "@{[shuffle @F]}"'
This is almost the same as one-liner #22 above. Instead of summing all fields, it shuffles and prints them.

The “ @{[shuffle @F]} “ construct creates an array reference to the contents of “ shuffle @F “ and “ @ { … } “ dereferences it. This is a tricky way to execute code inside quotes. It was needed to get the values of shuffled @F separated by a space when printing them out.

Another way to do the same is join the elements of @F by a space, but it’s longer:

1	perl -MList::Util=shuffle -alne 'print join " ", shuffle @F'

Find the minimum element on a line.
1
perl -MList::Util=min -alne 'print min @F'
This one-liner uses the “min” function from “List::Util”. It’s similar to all the previous ones. After the line has been automatically split by “ -a “, the “min” function finds minimum element and prints it.
Find the minimum element over all the lines.
1
perl -MList::Util=min -alne '@M = (@M, @F); END { print min @M }'
This one-liner is a combination of the previous one and the #23.

The “@M = (@M, @F)” construct is the same as “push @M, @F”. It appends the contents of @F to the array @M.

This one-liner stores all the data in memory. If you run it on a 10 terabyte file, it will die. Therefore it’s better to keep the running minimum element in memory and print it out at the end:

perl -MList::Util=min -alne ‘$min = min @F; $rmin = $min unless defined $rmin && $min > $rmin; END { print $rmin }’

It finds the minimum of each line and stores in $min, then it checks if $min is smaller than the running minimum. Once the input ends, it prints the running minimum, which is the smallest value over all input.

Find the maximum element on a line.
1
perl -MList::Util=max -alne 'print max @F'
This is the same as #25, except “min” has been replaced with “max”.
Find the maximum element over all the lines.
1
perl -MList::Util=max -alne '@M = (@M, @F); END { print max @M }'
This is the same as #26.

Or:

1	perl -MList::Util=max -alne '$max = max @F; $rmax = $max unless defined $rmax && $max < $rmax; END { print $rmax }'

Replace each field with its absolute value.
1
perl -alne 'print "@{[map { abs } @F]}"'
This one-liner auto-splits the line by “ -a “ command line option. The split fields, as I already explained, end up in the @F variable. Next it calls the absolute value function “abs” on each field by the help of “map” function. Finally it prints it joins all the fields by the help of array interpolation in double quotes.

The “ @{ … } “ construct was explained in one-liner #24.

Find the total number of fields (words) on each line.
1
perl -alne 'print scalar @F'
This one-liner forces to evaluate the @F in scalar context, which in Perl means “the number of elements in @F.” Therefore this one-liner prints out the number of elements on each line.
Print the total number of fields (words) on each line followed by the line.
1
perl -alne 'print scalar @F, " $_"'
This is exactly the same as #30, except “ $ “ is added at the end that prints out the whole line. (Remember that “ -n “ option caused each line to be put in the $ variable.)
Find the total number of fields (words) on all lines.
1
perl -alne '$t += @F; END { print $t}'
Here we just keep adding the number of fields on each line to variable “ $t “, and at the end we print it out. The result is number of words on all lines.
Print the total number of fields that match a pattern.
1
perl -alne 'map { /regex/ && $t++ } @F; END { print $t }'
This one-liner uses the “ map “ function that applies some operation on each of the elements in @F array. In this case the operation checks if each element matches /regex/ and if it does, it increments variable $t. At the end it prints this variable $t that contains the number of fields that matched /regex/ pattern.

A better way to do it is by looping:

1	perl -alne '$t += /regex/ for @F; END { print $t }'

Each element in code>@F</code is tested against regex. If it matches, /regex/ returns 1 (true), which gets added to variable $t. This way the number of matches get counted in $t.

The best way is to use grep in scalar context:

1	perl -alne '$t += grep /regex/, @F; END { print $t }'

Grep in scalar context returns the number of matches. This number gets accumulated in $t.

Print the total number of lines that match a pattern.
1
perl -lne '/regex/ && $t++; END { print $t }'
The /regex/ evaluates to true if the current line of input matches this regular expression. Writing /regex/ && $t++ is the same as if ($_ =~ /regex/) { $t++ }, which increments variable $t if the line matched the pattern. Finally in the END block the variable $t contains the total number of pattern matches and it gets printed out.
Print the number PI to n decimal places.
1
perl -Mbignum=bpi -le 'print bpi(21)'
The bignum package exports bpi function that calculates constant PI to wanted accuracy. This one-liner prints PI to 20 decimal places.

The bignum library also exports constant PI alone to 39 decimal places:

1	perl -Mbignum=PI -le 'print PI'

Print the number E to n decimal places.
1
perl -Mbignum=bexp -le 'print bexp(1,21)'
The bignum library also exports bexp function that takes two arguments - the power to raise e to and accuracy. This one-liner prints the constant e to 20 decimal places.

You can print the value of e^2 to 30 decimal places this way:

1	perl -Mbignum=bexp -le 'print bexp(2,31)'

Just the same as with PI, bignum exports the constant e alone to 39 decimal places:

1	perl -Mbignum=e -le 'print e'

Print UNIX time (seconds since Jan 1, 1970, 00:00:00 UTC).
1
perl -le 'print time'
The built-in function “time” returns seconds since the epoch.
Print GMT (Greenwich Mean Time) and local computer time.
1
perl -le 'print scalar gmtime'
The “gmtime” function is a Perl built-in function. If used in scalar context, it prints the time localized to Greenwich time zone.
1
perl -le 'print scalar localtime'
The “localtime” built-in function acts the same way as “gmtime”, except it prints the computer’s local time.

In array context both “gmtime” and “localtime” return a 9 element list (struct tm) with the following elements.

($second,             [0]
$minute,              [1]
$hour,                [2]
$month_day,           [3]
$month,               [4]
$year,                [5]
$week_day,            [6]
$year_day,            [7]
$is_daylight_saving   [8]
)

You may slice this list, or print individual elements if you need just some part of this information.

For example, to print H:M:S, slice elements 2, 1 and 0 from localtime:

1	perl -le 'print join ":", (localtime)[2,1,0]'

Print yesterday’s date.
1
perl -MPOSIX -le '@now = localtime; $now[3] -= 1; print scalar localtime mktime @now'
Remember that localtime returns a 9-list (see above) of various date elements. The 4th element in the list is current month’s day. If we subtract one from it we get yesterday. The “mktime” function constructs a Unix epoch time from this modified 9-list. And “scalar localtime” construct prints out the new date, which is yesterday.

The POSIX package was needed because it exports mktime function. It’s supposed to normalize negative values.

Print date 14 months, 9 days and 7 seconds ago.
1
perl -MPOSIX -le '@now = localtime; $now[0] -= 7; $now[4] -= 14; $now[7] -= 9; print scalar localtime mktime @now'
This one-liner modifies 0th, 4th, and 7th elements of @now list. The 0th is seconds, the 4th is months and 7th is days (see the table of 9 element time list above).

Next, mktime creates Unix time from this new structure, and localtime, evaluated in scalar context, prints out the date that was 14 months, 9 days and 7 seconds ago.

Calculate factorial.
1
perl -MMath::BigInt -le 'print Math::BigInt->new(5)->bfac()'
This one-liner uses bfac() function from Math::BigInt module that is in the Perl core (no need to install).

Math::BigInt->new(5) construction creates a new Math::BigInt object with value 5, then a method bfac() is called on the newly created object to calculate the factorial of 5. Change 5 to any number you want to find factorial for the value you are interested in.

Another way to calculate factorial is by just multiplying numbers from 1 to n together:

1	perl -le '$f = 1; $f *= $_ for 1..5; print $f'

Here we initially set $f to 1. Then do a loop from 1 to 5 and multiply $f by each of the values. The result is 12345, which is the factorial of 5.

Calculate greatest common divisor.
1
perl -MMath::BigInt=bgcd -le 'print bgcd(@list_of_numbers)'
Math::BigInt has several other useful math functions. One of them is bgcd that calculates the greatest common divisor of a list of numbers.

For example, to find gcd of (20, 60, 30), you’d execute the one-liner this way:

1	perl -MMath::BigInt=bgcd -le 'print bgcd(20,60,30)'

Surely, you can also use Euclid’s algorithm. Given two numbers $n and $m, this one-liner finds the gcd of $n and $m. The result is stored in $m.

1	perl -le '$n = 20; $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $m'

Calculate least common multiple.

Another function from Math::BigInt is lcm - the least common multiplicator. This one-liner finds lcm of (35, 20, 8):

1	perl -MMath::BigInt=blcm -le 'print blcm(35,20,8)'

If you know some number theory, then you’ll recall that there is a connection between gcd and lcm. Given two numbers $n and $m, their lcm is $n*$m/gcd($n,$m), therefore one-liner follows:

1	perl -le '$a = $n = 20; $b = $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $a*$b/$m'

Generate 10 random numbers between 5 and 15 (excluding 15).
1
perl -le '$n=10; $min=5; $max=15; $, = " "; print map { int(rand($max-$min))+$min } 1..$n'
You can modify this one-liner by changing variables $n, $min, $max. The variable $n stands for how many random numbers to generate, and [$min,$max) is the generation range.

The variable $, gets set to a space because it’s the output field separator for print and it’s undef by default. If we didn’t set it to a space, the numbers would get printed concatenated together.

Find and print all permutations of a list.
1
perl -MAlgorithm::Permute -le '$l = [1,2,3,4,5]; $p = Algorithm::Permute->new($l); print @r while @r = $p->next'
This one-liner uses the object-oriented interface of Algorithm::Permute module to find the permutations (all ways to rearrange items).

The constructor of Algorithm::Permute takes an array reference to an array of elements to permute. In this particular one-liner the elements are numbers 1, 2, 3, 4, 5.

The next object function returns the next permutation. Calling it repeatedly iterates over all permutations. Each permutation is put in code>@r</code array and is then printed.

Please note that the output list gets large really quickly. There are n! permutations for a list of n elements.

Another way to print out all permutations is to use the exported permute subroutine:

1	perl -MAlgorithm::Permute -le '@l = (1,2,3,4,5); Algorithm::Permute::permute { print "@l" } @l'

Generate the power set.

1	perl -MList::PowerSet=powerset -le '@l = (1,2,3,4,5); for (@{powerset(@l)}) { print "@$_" }'

Here I use the List::PowerSet module from CPAN.

It exports the powerset function, which takes a list of elements and returns a reference to a list containing references to subset lists.

In the for() loop, I call the powerset function, pass it the list of elements in code>@l</code. Next I dereference the return value of powerset, which is a reference to a list of subsets. Next, I dereference each individual subset code>@$_</code and print it.

For a set of n elements, there are exactly 2n subsets in the powerset.

Convert an IP address to unsigned integer.
1
perl -le '$i=3; $u += ($_<<8*$i--) for "127.0.0.1" =~ /(\d+)/g; print $u'
This one-liner converts the IP address 127.0.0.1 into unsigned integer (which happens to be 2130706433).

It does it by first doing a global match of (\d+) on the IP address. Doing a for loop over a global match iterates over all the matches. These matches are the four parts of the IP address.

Next the matches are added together in the $u variable, with first being bit shifted 83 = 24 places, the second being shifted 82 = 16 places, the third 8 places and the last just getting added to $u.

But this one-liner doesn’t do any error checking on the format of an IP address. You may use a more sophisticated regular expression to add checking, such as /^(\d+).(\d+).(\d+).(\d+)$/g.

I had a discussion about this with a friend and we came up with several more one-liner:

1	perl -le '$ip="127.0.0.1"; $ip =~ s/(\d+)\.?/sprintf("%02x", $1)/ge; print hex($ip)'

This one-liner utilizes the fact that 127.0.0.1 can be easily converted to hex as 7f000001 and then converted to decimal from hex by the hex Perl function.

Another way is to use unpack:

1	perl -le 'print unpack("N", 127.0.0.1)'

This one-liner is probably as short as it can get. It uses the vstring literals (version strings) to express the IP address. A vstring forms a string literal composed of characters with the specified ordinal values. Next, the newly formed string literal is unpacked into a number from a string in Network byte order (Big-Endian order) and it gets printed.

If you have a string with an IP (and not a vstring), then you first have to convert the string with the function inet_aton to byte form:

1	perl -MSocket -le 'print unpack("N", inet_aton("127.0.0.1"))'

Here inet_aton converts the string “ 127.0.0.1 “ to the byte form (which is the same as pure vstring 127.0.0.1) and next it unpacks it as the same was as in previous one-liner.

If you want a reference of pack and unpack templates (such as “N” for Network order), get my Perl pack/unpack cheat sheet!

Convert an unsigned integer to an IP address.
1
perl -MSocket -le 'print inet_ntoa(pack("N", 2130706433))'
Here the integer 2130706433 first gets packed into a number in Big-Endian and then it gets passed to inet_ntoa function that converts a number back to an IP address.

Another way to do it is by bit shifting and printing one byte at a time:

1	perl -le '$ip = 2130706433; print join ".", map { (($ip>>8*($_))&0xFF) } reverse 0..3'

And by the way, join “.” can be replaced by the special variable $, that acts as a value separator for print statement:

1	perl -le '$ip = 2130706433; $, = "."; print map { (($ip>>8*($_))&0xFF) } reverse 0..3'

See my Perl special variable cheat sheet for the list of all variables.

Perl One-Liners Explained, Part IV: String and Array Creation

This is the fourth part of a nine-part article on Perl one-liners. In this part I will create various one-liners for string and array creation. See part one for introduction of the series.

Perl one-liners is my attempt to create “perl1line.txt” that is similar to “awk1line.txt” and “sed1line.txt” that have been so popular among Awk and Sed programmers.

The article on Perl one-liners will consist of nine parts:

Part I: File spacing.
Part II: Line numbering.
Part III: Calculations.
Part IV: String creation and array creation (this part).
Part V: Text conversion and substitution.
Part VI: Selective printing and deleting of certain lines.
Part VII: Handy regular expressions.
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

I decided that there will be two new parts in this series. The most powerful feature in Perl is its regular expressions, therefore I will write a part on “Handy Perl regular expressions.” I also decided to publish an e-book after I am done with the series, so that will be the last part of this series. Subscribe to my blog to know when that happens!

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

I also updated the previous part on calculations with 14 new one-liners on finding values of constants pi and e, doing date calculations, finding factorial, greatest common divisor, least common multiple, generating random numbers, generating permutations, finding power sets and doing some IP address conversions.

Here are today’s one-liners:
String Creation and Array Creation

Generate and print the alphabet.
1
perl -le 'print a..z'
This one-liner prints all the letters from a to z as abcdefghijklmnopqrstuvwxyz. The letters are generated by the range operator … The range operator, when used in the list context (which is forced here by print) on strings, uses the magical auto-increment algorithm that advances the string to the next character. So in this one-liner the auto-increment algorithm on the range a..z produces all the letters from a to z.

I really golfed this one-liner. If you used strict it would not work because of barewords a and z. Semantically more correct version is this:

1	perl -le 'print ("a".."z")'

Remember that the range operator .. produced a list of values. If you want, you may print them comma separated by setting the $, special variable:

1	perl -le '$, = ","; print ("a".."z")'

There are many more special variables. Take a look at my special variable cheat sheet for a complete listing.

Syntactically more appealing is to use join to separate the list with a comma:

1	perl -le 'print join ",", ("a".."z")'

Here the list a..z gets joined by a comma before printing.

Generate and print all the strings from “a” to “zz”.
1
perl -le 'print ("a".."zz")'
Here the range operator .. is used again. This time it does not stop at “z” as in the previous one-liner, but advances z by one-character producing “aa”, then it keeps going, producing “ab”, “ac”, …, until it hits “az”. At this point it advances the string to “ba”, continues with “bb”, “bc”, …, until it reaches “zz”.

Similarly, you may generate all strings from “aa” to “zz” by:

1	perl -le 'print "aa".."zz"'

Here it goes like “aa”, “ab”, …, “az”, “ba”, “bb”, …, “bz”, “ca”, … “zz”.

Create a hex lookup table.
1
@hex = (0..9, "a".."f")
Here the array code>@hex</code gets filled with values 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 and letters a, b, c, d, e, f.

You may use this array to convert a number (in variable $num) from decimal to hex using base conversion formula:

1	perl -le '$num = 255; @hex = (0..9, "a".."f"); while ($num) { $s = $hex[($num%16)&15].$s; $num = int $num/16 } print $s'

Surely, much easier way to convert a number to hex is just using the printf function (or sprintf function) with %x format specifier. (The example above just illustrates a use of a hex lookup table that we created by using the range operator.)

1	perl -le '$hex = sprintf("%x", 255); print $hex'

(See my Perl printf and sprintf format cheat sheet for all the format specifiers.)

To convert the number back from hex to dec use the hex function:

1	perl -le '$num = "ff"; print hex $num'

The hex function takes a hex string (beginning with or without “0x”) and converts it to decimal.

Generate a random 8 character password.
1
perl -le 'print map { ("a".."z")[rand 26] } 1..8'
Here the map function executes (“a”..”z”)[rand 26] code 8 times (because it iterates over the dummy range 1..8). In each iteration the code chooses a random letter from the alphabet. When map is done iterating, it returns the generated list of characters and print function prints it out by concatenating all the characters together.

If you also want to include numbers in the password, add 0..9 to the list of characters to choose from and change 26 to 36 as there are 36 different characters to choose from:

1	perl -le 'print map { ("a".."z", 0..9)[rand 36] } 1..8'

If you need a longer password, change 1..8 to 1..20 to generate a 20 character long password.

Create a string of specific length.
1
perl -le 'print "a"x50'
Operator x is the repetition operator. This one-liner creates a string of 50 letters “a” and prints it.

If the repetition operator is used in list context, it creates a list (instead of scalar) with the given elements repeated:

1	perl -le '@list = (1,2)x20; print "@list"'

This one liner creates a list of twenty repetitions of (1, 2) (it looks like (1, 2, 1, 2, 1, 2, …)).

Create an array from a string.
1
@months = split ' ', "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
Here the code>@months</code gets filled with values from the string containing month names. As each month name is separated by a space, the split function splits them and puts them in code>@months</code. This way $months[0] contains “Jan”, $months[1] contains “Feb”, …, and $months[11] contains “Dec”.

Another way to do the same is by using qw// operator:

1	@months = qw/Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec/

The qw// operator takes a space separated string and creates an array with each word being an array element.

Create a string from an array.
1
@stuff = ("hello", 0..9, "world"); $string = join '-', @stuff
Here the values in array code>@stuff</code get turned in a string $string that has them separated by a hyphen. Turning an array in a string was done by the join function that takes a separator and a list, and concatenates the items in the list in a single string, separated by the separator.
Find the numeric values for characters in the string.
1
perl -le 'print join ", ", map { ord } split //, "hello world"'
This one-liner takes the string “hello world”, splits it into a list of characters by split //, “hello world”, then it maps the ord function onto each of the characters, which returns the numeric, native 8-bit encoding (like ASCII or EBCDIC) of the character. Finally all the numeric values get joined together by a comma and get printed out.

Another way to do the same is use the unpack function and specify C as the unpacking template (C means unsigned character and means as many characters there are):

1	perl -le 'print join ", ", unpack("C*", "hello world")'

Convert a list of numeric ASCII values into a string.
1
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print pack("C*", @ascii)'
Just as we unpacked a string into a list of values with the C* template in the one-liner above, we can pack them back into a string.

Another way to do the same is use the chr function that takes the code point value and returns the corresponding character:

1	perl -le '@ascii = (99, 111, 100, 105, 110, 103); print map { chr } @ascii'

Similar to one-liner #55 above, function chr gets mapped onto each value in the code>@ascii</code producing the characters.

Generate an array with odd numbers from 1 to 100.
1
perl -le '@odd = grep {$_ % 2 == 1} 1..100; print "@odd"'
This one-liner generates an array of odd numbers from 1 to 99 (as 1, 3, 5, 7, 9, 11, …, 99). It uses the grep function that evaluates the given code $_ % 2 == 1 for each element in the given list 1..100 and returns only the elements that had the code evaluate to true. In this case the code tests if the reminder of the number is 1. If it is, the number is odd and it has to be put in the code>@odd</code array.

Another way to write is by remembering that odd numbers have the low-bit set and testing this fact:

1	perl -le '@odd = grep { $_ & 1 } 1..100; print "@odd"'

Expression $_ & 1 isolates the low-bit, and grep selects only the numbers with low-bit set (odd numbers).

See my explanation of bit-hacks for full explanation and other related bit-hacks.

Generate an array with even numbers from 1 to 100.
1
perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'
This is almost the same as the previous one-liner, except the condition grep tests for is “is the number even (reminder dividing by 2 is zero)?”
Find the length of the string.
1
perl -le 'print length "one-liners are great"'
Just for completeness, the length subroutine finds the length of the string.
Find the number of elements in an array.
1
perl -le '@array = ("a".."z"); print scalar @array'
Evaluating an array in a scalar context returns the number of elements in it.

Another way to do the same is by adding one to the last index of the array:

1	perl -le '@array = ("a".."z"); print $#array + 1'

Here $#array returns the last index in array code>@array</code. Since it’s a number one less than the number of elements, we add 1 to the result to find the total number of elements in the array.

Perl One-Liners Explained, Part V: Text conversion and substitution

This is the fifth part of a nine-part article on Perl one-liners. In this part I will create various one-liners for text conversion and substitution. See part one for introduction of the series.

Perl one-liners is my attempt to create “perl1line.txt” that is similar to “awk1line.txt” and “sed1line.txt” that have been so popular among Awk and Sed programmers.

The article on Perl one-liners will consist of nine parts:

Part I: File spacing.
Part II: Line numbering.
Part III: Calculations.
Part IV: String creation and array creation.
Part V: Text conversion and substitution (this part).
Part VI: Selective printing and deleting of certain lines.
Part VII: Handy regular expressions.
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

After I’m done with explaining the one-liners, I’ll release an ebook. Subscribe to my blog to know when that happens!

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

Alright then, here are today’s one-liners:
Text conversion and substitution

ROT13 a string.
1
'y/A-Za-z/N-ZA-Mn-za-m/'
This one-liner uses the y operator (also known as tr operator) to do ROT13. Operators y and tr do string transliteration. Given y/SEARCH/REPLACE/, the operator transliterates all occurrences of the characters found in SEARCH list with the corresponding (position-wise) characters in REPLACE list.

In this one-liner A-Za-z creates the following list of characters:

1	ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz

And N-ZA-Mn-za-m creates this list:

1	NOPQRSTUVWXYZABCDEFGHIJKLMnopqrstuvwxyzabcdefghijklm

If you look closely you’ll notice that the second list is actually the first list offset by 13 characters. Now the y operator translates each character in the first list to a character in the second list, thus performing the ROT13 operation.

If you want to ROT13 the whole file then do this:

1	perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file

The -p argument puts each of file’s line in the $ variable, the y does ROT13, and -p prints the $ out. The -l appends a newline to the output.

Note: remember that applying ROT13 twice produces the same string, i.e., ROT13(ROT13(string)) == string.

Base64 encode a string.
1
perl -MMIME::Base64 -e 'print encode_base64("string")'
This one-liner uses the MIME::Base64 module that is in the core (no need to install it, it comes with Perl). This module exports the encode_base64 function that takes a string and returns base64 encoded version of it.

To base64 encode the whole file do the following:

1	perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' file

Here the -0777 argument together with -n causes Perl to slurp the whole file into the $_ variable. Then the file gets base64 encoded and printed out, just like the string example above.

If we didn’t slurp the file and encoded it line-by-line we’d get a mess.

Base64 decode a string.

1	perl -MMIME::Base64 -le 'print decode_base64("base64string")'

The MIME::Base64 module also exports decode_base64 function that takes a base64-encoded string and decodes it.

The whole file can be similarly decoded by:

1	perl -MMIME::Base64 -ne 'print decode_base64($_)' file

There is no need to slurp the whole file into $_ because each line of a base64 encoded file is exactly 76 characters and decodes nicely.

URL-escape a string.
1
perl -MURI::Escape -le 'print uri_escape($string)'
You’ll need to install the URI::Escape module as it doesn’t come with Perl. The module exports two functions - uri_escape and uri_unescape. The first one does URL-escaping (sometimes also referred to as URL encoding), and the other does URL-unescaping (URL decoding).
URL-unescape a string.
1
perl -MURI::Escape -le 'print uri_unescape($string)'
This one-liner uses the uri_unescape function from URI::Escape module to do URL-unescaping.
HTML-encode a string.
1
perl -MHTML::Entities -le 'print encode_entities($string)'
This one-liner uses the encode_entities function from HTML::Entities module. This function encodes HTML entities. For example, < and > get turned into < and >.
HTML-decode a string.
1
perl -MHTML::Entities -le 'print decode_entities($string)'
This one-liner uses the decode_entities function from HTML::Entities module.
Convert all text to uppercase.
1
perl -nle 'print uc'
This one-liner uses the uc function, which by default operates on the $_ variable and returns an uppercase version of it.

Another way to do the same is to use -p command line option that enables automatic printing of $_ variable and modify it in-place:

1	perl -ple '$_=uc'

The same can also be also achieved by applying the \U escape sequence to string interpolation:

1	perl -nle 'print "\U$_"'

It causes anything after it (or until the first occurrence of \E) to be upper-cased.

Convert all text to lowercase.
1
perl -nle 'print lc'
This one-liner is very similar to the previous. Here the lc function is used that converts the contents of $_ to lowercase.

Or, using escape sequence \L and string interpolation:

1	perl -nle 'print "\L$_"'

Here \L causes everything after it (until the first occurrence of \E) to be lower-cased.

Uppercase only the first word of each line.
1
perl -nle 'print ucfirst lc'
The one-liner first applies the lc function to the input that makes it lower case and then uses the ucfirst function that upper-cases only the first character.

It can also be done via escape codes and string interpolation:

1	perl -nle 'print "\u\L$_"'

First the \L lower-cases the whole line, then \u upper-cases the first character.

Invert the letter case.
1
perl -ple 'y/A-Za-z/a-zA-Z/'
This one-liner does transliterates capital letters A-Z to lowercase letters a-z, and lowercase letters to uppercase letters, thus switching the case.
Camel case each line.
1
perl -ple 's/(\w+)/\u$1/g'
This is a lousy Camel Casing one-liner. It takes each word and upper-cases the first letter of it. It fails on possessive forms like “friend’s car”. It turns them into “Friend’S Car”.

An improvement is:

1	s/(?<!['])(\w+)/\u\1/g

Which checks if the character before the word is not single quote ‘. But I am sure it still fails on some more exotic examples.

Strip leading whitespace (spaces, tabs) from the beginning of each line.
1
perl -ple 's/^[ \t]+//'
This one-liner deletes all whitespace from the beginning of each line. It uses the substitution operator s. Given s/REGEX/REPLACE/ it replaces the matched REGEX by the REPLACE string. In this case the REGEX is ^[ \t]+, which means “match one or more space or tab at the beginning of the string” and REPLACE is nothing, meaning, replace the matched part with empty string.

The regex class [ \t] can actually be replaced by \s+ that matches any whitespace (including tabs and spaces):

1	perl -ple 's/^\s+//'

Strip trailing whitespace (space, tabs) from the end of each line.
1
perl -ple 's/[ \t]+$//'
This one-liner deletes all whitespace from the end of each line.

Here the REGEX of the s operator says “match one or more space or tab at the end of the string.” The REPLACE part is empty again, which means to erase the matched whitespace.

Strip whitespace from the beginning and end of each line.
1
perl -ple 's/^[ \t]+|[ \t]+$//g'
This one-liner combines the previous two. Notice that it specifies the global /g flag to the s operator. It’s necessary because we want it to delete whitespace at the beginning AND end of the string. If we didn’t specify it, it would only delete whitespace at the beginning (assuming it exists) and not at the end.
Convert UNIX newlines to DOS/Windows newlines.
1
perl -pe 's|\n|\r\n|'
This one-liner substitutes the Unix newline \n LF with Windows newline \r\n CRLF on each line. Remember that the s operator can use anything for delimiters. In this one-liner it uses vertical pipes to delimit REGEX from REPLACE to improve readibility.
Convert DOS/Windows newlines to UNIX newlines.
1
perl -pe 's|\r\n|\n|'
This one-liner does the opposite of the previous one. It takes Windows newlines CRLF and converts them to Unix newlines LF.
Convert UNIX newlines to Mac newlines.
1
perl -pe 's|\n|\r|'
Apple Macintoshes used to use \r CR as newlines. This one-liner converts UNIX’s \n to Mac’s \r.
Substitute (find and replace) “foo” with “bar” on each line.
1
perl -pe 's/foo/bar/'
This one-liner uses the s/REGEX/REPLACE/ command to substitute “foo” with “bar” on each line.

To replace all “foos” with “bars”, add the global /g flag:

1	perl -pe 's/foo/bar/g'

Substitute (find and replace) “foo” with “bar” on lines that match “baz”.
1
perl -pe '/baz/ && s/foo/bar/'
This one-liner is equivalent to:
1
2
3
4
5
while (defined($line = <>)) {
if ($line =~ /baz/) {
$line =~ s/foo/bar/
}
}
It puts each line in variable $line, then checks if line matches “baz”, and if it does, it replaces “foo” with “bar” in it.

Perl One-Liners Explained, Part VI: Selective printing and deleting of lines

This is the sixth part of a nine-part article on Perl one-liners. In this part I will create various one-liners for selective printing and deleting of certain lines. See part one for introduction of the series.

Perl one-liners is my attempt to create “perl1line.txt” that is similar to “awk1line.txt” and “sed1line.txt” that have been so popular among Awk and Sed programmers and Linux sysadmins.

The article on Perl one-liners will consist of nine parts:

Part I: File spacing.
Part II: Line numbering.
Part III: Calculations.
Part IV: String creation and array creation.
Part V: Text conversion and substitution.
Part VI: Selective printing and deleting of certain lines (this part).
Part VII: Handy regular expressions.
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

The selective printing and selective deleting of certain lines is actually the same process. If you want to delete certain lines, you just print the lines you’re interested in. Or the other way around! For example, to delete all lines with even line numbers, print the odd lines, and to delete odd lines print the even lines.

After I am done with the 8 parts of the article, I will release the whole article series as a pdf e-book! Please subscribe to my blog to be the first to get it!

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

Here are today’s one-liners:

Print the first line of a file (emulate head -1).
1
perl -ne 'print; exit'
This is the simplest one-liner so far. Here Perl reads in the first line into $ variable thanks to -n option, then it calls print statement that prints the contents of the $ variable. And then it just exists. That’s it. The first line got printed and that’s all we wanted.
Print the first 10 lines of a file (emulate head -10).
1
perl -ne 'print if $. <= 10'
This one-liner uses the $. special variable. This variable stands for “current line number.” Each time Perl reads in the next line, it increments $. by one. Therefore it’s very simple to understand what this one-liner does, it prints the line if the line number is equal to or less than 10.

This one liner can also be written the other way around without use of if statement,

1	perl -ne '$. <= 10 && print'

Here the print statement gets called only if $. <= 10 boolean expression is true, and it’s true only if current line number is less than or equal to 10.

Print the last line of a file (emulate tail -1).
1
perl -ne '$last = $_; END { print $last }'
Printing the last line of the file is a bit tricker, because you always have to maintain the previous line in memory. In this one-liner we always save the current line in $_ to $last variable. When Perl program ends, it always executes code in the END block. Now just before exiting it read in the last line, so when it quits, we print $last that prints the last line.

Another way to do the same is,

1	perl -ne 'print if eof'

This one-liner uses the eof function that returns 1 if the next read will return end of file. Since the next read after the last line in the file will really return eof, this one-liner does what it’s supposed to do.

Print the last 10 lines of a file (emulate tail -10).
1
perl -ne 'push @a, $_; @a = @a[@a-10..$#a]; END { print @a }'
Now this is tricky. Here we push each line to the code>@a</code array, and then we replace with a slice of itself. We do @a = @a[@a-10..$#a], which means, replace code>@a</code with last 10 elements of a. code>@a-10</code is evaluated in scalar context here and it returns number of elements in the array minus 10. #$a is the last index in the code>@a</code array. And code>@a[@a-10..$#a]</code takes the last 10 elements of the array, so code>@a</code always contains just 10 last elements.

Here is an example. Suppose code>@a</code contains (“line1”, “line2”, “line3”, “line4”). And let’s say we want to print last 4 lines of the file. Now when we read the 5th line, the array becomes (“line1”, “line2”, “line3”, “line4”, “line5”). At this moment code>@a-4</code is 1, because code>@a</code in scalar context is 5. The $#a however is 4 because that’s the last index in the array. Now taking the slice, code>@a[@a-4..$#a]</code is code>@a[1..4]</code, which drops the front element from the array and the code>@a</code array becomes (“line2”, “line3”, “line4”, “line5”).

Print only lines that match a regular expression.
1
perl -ne '/regex/ && print'
Here /regex/ is short for $ =~ /regex/. Since the -n operator puts every line in $ variable the /regex/ returns true on all lines that matched the regex. If that happened, print prints the line.
Print only lines that do not match a regular expression.
1
perl -ne '!/regex/ && print'
This is the same as the previous one-liner, except the regular expression match has been negated. So all the lines that don’t match the regex get printed.
Print the line before a line that matches a regular expression.
1
perl -ne '/regex/ && $last && print $last; $last = $_'
In this one-liner every line gets saved to $last variable. Now when the next line matches /regex/ and there has been a previous line $last, then it print $last prints the last line, and then it assigns the current line to the last line variable via $last = $_.
Print the line after a line that matches a regular expression.
1
perl -ne 'if ($p) { print; $p = 0 } $p++ if /regex/'
Here we set the variable $p if the line matches a regex. It indicates that the next line should be printed. Now when the next line is read in and $p is set, then that line gets printed and $p gets set to 0 again to reset the state.
Print lines that match regex AAA and regex BBB in any order.
1
perl -ne '/AAA/ && /BBB/ && print'
This one-liner is basically the same as one-liner #86 above. Here we test if a line matches two regular expressions instead of line. If a line matches both regexes, then it gets printed.
Print lines that don’t match match regexes AAA and BBB.
1
perl -ne '!/AAA/ && !/BBB/ && print'
This one-liner is almost the same as one-liner #87. Here we test if a line doesn’t match two regular expressions in any order. If it doesn’t match /AAA/ and it doesn’t match /BBB/, then we print it.
Print lines that match regex AAA followed by regex BBB followed by CCC.
1
perl -ne '/AAA.*BBB.*CCC/ && print'
Here we simply chain regexes AAA, BBB and CCC with .*, which stands for match anything or nothing at all. If AAA is followed by BBB and that is followed by CCC then we print the line. It also matches AAABBBCCC with nothing in between the regexes.
Print lines that are 80 chars or longer.
1
perl -ne 'print if length >= 80'
This one-liner prints all lines that are 80 chars or longer. In Perl you can sometimes omit the brackets () for function calls. In this one we omitted brackets for length function call. In fact, length, length() and length($_) are the same.
Print lines that are less than 80 chars in length.
1
perl -ne 'print if length < 80'
This is the opposite of previous one-liner. It checks if the length of a line is less than 80 characters.
Print only line 13.
1
perl -ne '$. == 13 && print && exit'
As I explained in one-liner #83, the $. special variable stands for “current line number”. So if $. has value 13, then we print the line and exit.
Print all lines except line 27.
1
perl -ne '$. != 27 && print'
Just like in previous one-liner, we check if the current line is line 27, if it’s not then we print it, otherwise we skip it.

Another way to write the same is to reverse print and $. != 27 and use if statement,

1	perl -ne 'print if $. != 27'

Print only lines 13, 19 and 67.
1
perl -ne 'print if $. == 13 || $. == 19 || $. == 67'
If you have Perl 5.10 or later then you can use the ~~ smart match operator,
1
perl -ne 'print if int($.) ~~ (13, 19, 67)'
The smart matching operator ~~ appeared only in Perl 5.10. You can do all kinds of smart matching with it, for example, check if two arrays are the same, if an array contains an element, and many other use cases (see perldoc perlsyn). In this particular one-liner we use int($.) ~~ (13, 19, 67) that determines if numeric value $. is in the list (13, 19, 69). It’s basically short for, grep { $ == int($.) } (13, 19, 67). If the check succeeds the line gets printed.
Print all lines between two regexes (including lines that match regex).
1
perl -ne 'print if /regex1/../regex2/'
This one-liner uses the flip-flop operator, which becomes true when a line matches regex1 and becomes false after another line matches regex2. Therefore this one-liner prints all lines between (and including) lines that match regex1 and regex2.
Print all lines from line 17 to line 30.
1
perl -ne 'print if $. >= 17 && $. <= 30'
This one-liner is very simple to understand. The $. variable stands for the current line number, so it checks if the current line number is greater than or equal to 17 and less than or equal to 30.

I just thought of another way to write it,

1	perl -ne 'print if int($.) ~~ (17..30)'

This is one-liner uses the Perl 5.10 (and later) smart matching operator ~~. It basically says, is the current line number in the list (17, 18, 19, …, 30). If it is, the smart match succeeds and the line gets printed.

You can write the same idea in older Perls as following,

1	perl -ne 'print if grep { $_ == $. } 17..30'

What happens here is grep checks if the current line number is in the list (17, 18, …, 30). If it is, it returns a list of just one element, and a list of one element is true, and the line gets printed. Otherwise grep returns the empty list, which is false, and nothing gets printed.

Print the longest line.
1
perl -ne '$l = $_ if length($_) > length($l); END { print $l }'
This one-liner keeps the longest line seen so far in the $l variable. In case the current line $_ exceeds the length of currently longest line, it gets replaced. Just before exiting, the END block is executed and it prints the longest line $l.
Print the shortest line.
1
perl -ne '$s = $_ if $. == 1; $s = $_ if length($_) < length($s); END { print $s }'
This one-liner is the opposite of the previous one. But as we’re finding the minimum and $s is not defined for the first line, we have to set it to first line explicitly. Otherwise it’s the same just with the length check reversed length($_) < length($s).
Print all lines that contain a number.
1
perl -ne 'print if /\d/'
This one-liner uses a regular expression \d that stands for “match a number” and checks if a line contains one. If it does, the check succeeds, and the line gets printed.
Find all lines that contain only a number.
1
perl -ne 'print if /^\d+$/'
This one-liner is very similar to the previous one, but instead of matching a number anywhere on the line, it anchors the match to the beginning of the line, and to the end of the line. The regular expression ^\d+$ means “match one or more numbers that start at the beginning of line and end at the end of the line”.
Print all lines that contain only characters.
1
perl -ne 'print if /^[[:alpha:]]+$/
This one-liner checks if the line contains only characters and if it does, it prints it. Here the [[:alpha:]] stands for “match all characters”. You could also write the same as [a-zA-Z] (if you live in ASCII world).
Print every second line.
1
perl -ne 'print if $. % 2'
This one-liner prints first, third, 5th, 7th, etc, line. It does so because $. % 2 is true when the current line number is odd, and it’s false when the current line number is even.
Print every second line, starting the second line.
1
perl -ne 'print if $. % 2 == 0'
This one-liner is very similar to the previous one but except printing 1st, 3rd, 5th, etc, lines, it prints 2nd, 4th, 6th, etc, lines. It prints them because $. % 2 == 0 is true when the current line number is 2, 4, 6, ….
Print all lines that repeat.
1
perl -ne 'print if ++$a{$_} == 2'
This one-liner keeps track of the lines it has seen so far and it also keeps the count of how many times it has seen the line before. If it sees the line the 2nd time, it prints it out because ++$a{$_} == 2 is true. If it sees the line more than 2 times, it just does nothing because the count for this line has gone beyond 2 and the result of the print check is false.
Print all unique lines.
1
perl -ne 'print unless $a{$_}++'
Here the lines get printed only if the hash value $a{$_} for the line is 0. Every time Perl reads in a line, it increments the value for the line and that makes sure that only never before seen lines get printed.

Perl One-Liners Explained, Part VII: Handy Regular Expressions

This is the seventh part of a nine-part article on Perl one-liners. Perl is not Perl without regular expressions, therefore in this part I will come up with and explain various Perl regular expressions. Please see part one for the introduction of the series.

Perl one-liners is my attempt to create “perl1line.txt” that is similar to “awk1line.txt” and “sed1line.txt” that have been so popular among Awk and Sed programmers, and Unix sysadmins. I will release the perl1line.txt in the next part of the series.

The article on Perl one-liners consists of nine parts:

Part I: File spacing.
Part II: Line numbering.
Part III: Calculations.
Part IV: String creation and array creation.
Part V: Text conversion and substitution.
Part VI: Selective printing and deleting of certain lines.
Part VII: Handy regular expressions (this part).
Part VIII: Release of perl1line.txt.
Part IX: Release of Perl One-Liners e-book.

After I am done with the next part of the article, I will release the whole article series as a pdf e-book! Please subscribe to my blog to be the first to get it. You can also follow me on Twitter.

Awesome news: I have written an e-book based on this article series. Check it out:
Perl book

And here are today’s one-liners:

Match something that looks like an IP address.
1
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/
This regex doesn’t guarantee that the thing that got matched is in fact a valid IP. All it does is match something that looks like an IP. It matches a number followed by a dot four times. For example, it matches a valid IP 81.198.240.140 and it also matches an invalid IP such as 923.844.1.999.

Here is how it works. The ^ at the beginning of regex is an anchor that matches the beginning of string. Next \d{1,3} matches one, two or three consecutive digits. The . matches a dot. The $ at the end is an anchor that matches the end of the string. It’s important to use both ^ and $ anchors, otherwise strings like foo213.3.1.2bar would also match.

This regex can be simplified by grouping the first three repeated \d{1,3}. expressions:

1	/^(\d{1,3}\.){3}\d{1,3}$/

Test if a number is in range 0-255.
1
/^([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$/
Here is how it works. A number can either be one digit, two digit or three digit. If it’s a one digit number then we allow it to be anything [0-9]. If it’s two digit, we also allow it to be any combination of [0-9][0-9]. However if it’s a three digit number, it has to be either one hundred-something or two-hundred something. If it’e one hundred-something, then 1[0-9][0-9] matches it. If it’s two hundred-something then it’s either something up to 249, which is matched by 2[0-4][0-9] or it’s 250-255, which is matched by 25[0-5].
Match an IP address.
1
2
3
4
my $ip_part = qr|([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|;
if ($ip =~ /^($ip_part\.){3}$ip_part$/) {
say "valid ip";
}
This regexp combines the previous two. It uses the my $ip_part = qr/…/ operator compiles the regular expression and puts it in $ip_part variable. Then the $ip_part is used to match the four parts of the IP address.
Check if the string looks like an email address.
1
/.+@.+\..+/
This regex makes sure that the string looks like an email address. Notice that I say “looks like”. It doesn’t guarantee it is an email address. Here is how it works - first it matches something up to the code>@</code symbol, then it matches as much as possible until it finds a dot, and then it matches some more. If this succeeds, then it it’s something that at least looks like email address with the code>@</code symbol and a dot in it.

For example, code>cats@catonmat.net</code matches but code>cats@catonmat</code doesn’t because the regex can’t match the dot . that is necessary.

Much more robust way to check if a string is a valid email would be to use Email::Valid module:

1 2	use Email::Valid; print (Email::Valid->address('john@example.com') ? 'valid email' : 'invalid email');

Check if the string is a decimal number.

Checking if the string is a number is really difficult. I based my regex and explanation on the one in Perl Cookbook.

Perl offers \d that matches digits 0-9. So we can start with:

/^\d+$/

This regex matches one or more digits \d starting at the beginning of the string ^ and ending at the end of the string $. However this doesn’t match numbers such as +3 and -3. Let’s modify the regex to match them:

1	/^[+-]?\d+$/

Here the [+-]? means match an optional plus or a minus before the digits. This now matches +3 and -3 but it doesn’t match -0.3. Let’s add that:

1	/^[+-]?\d+\.?\d*$/

Now we have expanded the previous regex by adding .?\d*, which matches an optional dot followed by zero or more numbers. Now we’re in business and this regex also matches numbers like -0.3 and 0.3.

Much better way to match a decimal number is to use Regexp::Common module that offers various useful regexes. For example, to match an integer you can use $RE{num}{int} from Regexp::Common.

How about positive hexadecimal numbers? Here is how:

1	/^0x[0-9a-f]+$/i

This matches the hex prefix 0x followed by hex number itself. The /i flag at the end makes sure that the match is case insensitive. For example, 0x5af matches, 0X5Fa matches but 97 doesn’t, cause it’s just a decimal number.

It’s better to use $RE{num}{hex} because it supports negative numbers, decimal places and number grouping.

Now how about octal? Here is how:

1	/^0[0-7]+$/

Octal numbers are prefixed by 0, which is followed by octal digits 0-7. For example, 013 matches but 09 doesn’t, cause it’s not a valid octal number.

It’s better to use $RE{num}{oct} because of the same reasons as above.

Finally binary:

/^[01]+$/

Binary base consists of just 0s and 1s. For example, 010101 matches but 210101 doesn’t, because 2 is not a valid binary digit.

It’s better to use $RE{num}{bin} because of the same reasons as above.

Check if a word appears twice in the string.
1
/(word).*\1/
This regex matches word followed by something or nothing at all, followed by the same word. Here the (word) captures the word in group 1 and \1 refers to contents of group 1, therefore it’s almost the same as writing /(word).*word/

For example, silly things are silly matches /(silly).*\1/, but silly things are boring doesn’t, because silly is not repeated in the string.

Increase all numbers by one in the string.
1
$str =~ s/(\d+)/$1+1/ge
Here we use the substitution operator s///. It matches all integers (\d+), puts them in capture group 1, then it replaces them with their value incremented by one $1+1. The g flag makes sure it finds all the numbers in the string, and the e flag evaluates $1+1 as a Perl expression.

For example, this 1234 is awesome 444 gets turned into this 1235 is awesome 445.

Extract HTTP User-Agent string from the HTTP headers.
1
/^User-Agent: (.+)$/
HTTP headers are formatted as Key: Value pairs. It’s very easy to parse such strings, you just instruct the regex engine to save the Value part in $1 group variable.

For example, if the HTTP headers contain,

Host: localhost:8000
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_0_0; en-US)
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,/;q=0.5
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

Then the regular expression will extract the Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_0_0; en-US) string.

Match printable ASCII characters.
1
/[ -~]/
This is really tricky and smart. To understand it, take a look at man ascii. You’ll see that space starts at value 0x20 and the ~ character is 0x7e. All the characters between a space and ~ are printable. This regular expression matches exactly that. The [ -~] defines a range of characters from space till ~. This is my favorite regexp of all time.

You can invert the match by placing ^ as the first character in the group:

/[^ -~]/

This matches the opposite of [ -~].

Match text between two HTML tags.
1
m|([^<]*)|
This regex matches everything between … HTML tags. The trick here is the ([^<]*), which matches as much as possible until it finds a < character, which starts the next tag.

Alternatively you can write:

1	m\|<strong>(.*?)</strong>\|

But this is a little different. For example, if the HTML is hello then the first regex doesn’t match anything because the < follows and ([^<]) matches as little as possible. The second regex matches hello because the (.?) matches as little as possible until it finds , which happens to be hello.

However don’t use regular expressions for matching and parsing HTML. Use modules like HTML::TreeBuilder to accomplish the task cleaner.

Replace all tags with

1
$html =~ s|<(/)?b>|<$1strong>|g

Here I assume that the HTML is in variable $html. Next the <(/)?b> matches the opening and closing tags, captures the optional closing tag slash in group $1 and then replaces the matched tag with either or , depending on if it was an opening or closing tag.

Extract all matches from a regular expression.

1
my @matches = $text =~ /regex/g;

Here the regular expression gets evaluated in the list context that makes it return all the matches. The matches get put in the code>@matches</code variable.

For example, the following regex extracts all numbers from a string:

1
2
my $t = "10 hello 25 moo 31 foo";
my @nums = $text =~ /\d+/g;

code>@nums</code now contains (10, 25, 30).

Announcing perl1line.txt – A Collection of Handy Perl Scripts

The ultimate goal of the Perl One-Liners Explained article series was to release the perl1line.txt file. Last week I finished the series and today I am happy to announce perl1line.txt – a collection of handy Perl one-liner scripts.

Perl1line.txt file contains over a hundred short Perl one-line scripts for various text processing tasks. These tasks include – changing file spacing, numbering lines, doing calculations, creating strings and arrays, converting and substituting text, selective printing and deleting of certain lines, and modifying text with regular expressions.

The latest version of perl1line.txt is always at:

https://catonmat.net/ftp/perl1line.txt

Here is the full table of contents of perl1line.txt.

File Spacing. Line Numbering. Calculations. String Creation and Array Creation. Text Conversion and Substitution. Selective Printing and Deleting of Certain Lines. Handy Regular Expressions.

You can send me bug fixes and updates via GitHub. I put the file in its own perl1line.txt repository. I also accept translations. Send them in!

Awesome news: I have written an e-book based on the one-liners in this file. Check it out:
Perl book

Here is the whole file of perl1line.txt at version 1.0:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
Useful One-Line Scripts for Perl version 1.0
-------------------------------- -----------

Compiled by Peteris Krumins (peter@catonmat.net, @pkrumins on Twitter)
https://catonmat.net -- good coders code, great reuse

Latest version of this file is always at:

https://catonmat.net/ftp/perl1line.txt

This file is also available in other languages:
(None at the moment.)
Please email me peter@catonmat.net if you want to translate it.

I am also writing "Perl One-Liners Explained" ebook that's based on
this file. It explains all the one-liners here. Get it soon at:

https://catonmat.net/perl-book/

These one-liners work both on UNIX systems and Windows. Most likely your
UNIX system already has Perl. For Windows get the Strawberry Perl at:

http://www.strawberryperl.com/

Table of contents:

1. File Spacing
2. Line Numbering
3. Calculations
4. String Creation and Array Creation
5. Text Conversion and Substitution
6. Selective Printing and Deleting of Certain Lines
7. Handy Regular Expressions

FILE SPACING
------------

# Double space a file
perl -pe '$\="\n"'
perl -pe 'BEGIN { $\="\n" }'
perl -pe '$_ .= "\n"'
perl -pe 's/$/\n/'

# Double space a file, except the blank lines
perl -pe '$_ .= "\n" unless /^$/'
perl -pe '$_ .= "\n" if /\S/'

# Triple space a file
perl -pe '$\="\n\n"'
perl -pe '$_.="\n\n"'

# N-space a file
perl -pe '$_.="\n"x7'

# Add a blank line before every line
perl -pe 's//\n/'

# Remove all blank lines
perl -ne 'print unless /^$/'
perl -lne 'print if length'
perl -ne 'print if /\S/'

# Remove all consecutive blank lines, leaving just one
perl -00 -pe ''
perl -00pe0

# Compress/expand all blank lines into N consecutive ones
perl -00 -pe '$_.="\n"x4'

LINE NUMBERING
--------------

# Number all lines in a file
perl -pe '$_ = "$. $_"'

# Number only non-empty lines in a file
perl -pe '$_ = ++$a." $_" if /./'

# Number and print only non-empty lines in a file (drop empty lines)
perl -ne 'print ++$a." $_" if /./'

# Number all lines but print line numbers only non-empty lines
perl -pe '$_ = "$. $_" if /./'

# Number only lines that match a pattern, print others unmodified
perl -pe '$_ = ++$a." $_" if /regex/'

# Number and print only lines that match a pattern
perl -ne 'print ++$a." $_" if /regex/'

# Number all lines, but print line numbers only for lines that match a pattern
perl -pe '$_ = "$. $_" if /regex/'

# Number all lines in a file using a custom format (emulate cat -n)
perl -ne 'printf "%-5d %s", $., $_'

# Print the total number of lines in a file (emulate wc -l)
perl -lne 'END { print $. }'
perl -le 'print $n=()=<>'
perl -le 'print scalar(()=<>)'
perl -le 'print scalar(@foo=<>)'
perl -ne '}{print $.'

# Print the number of non-empty lines in a file
perl -le 'print scalar(grep{/./}<>)'
perl -le 'print ~~grep{/./}<>'
perl -le 'print~~grep/./,<>'

# Print the number of empty lines in a file
perl -lne '$a++ if /^$/; END {print $a+0}'
perl -le 'print scalar(grep{/^$/}<>)'
perl -le 'print ~~grep{/^$/}<>'

# Print the number of lines in a file that match a pattern (emulate grep -c)
perl -lne '$a++ if /regex/; END {print $a+0}'

CALCULATIONS
------------

# Check if a number is a prime
perl -lne '(1x$_) !~ /^1?$|^(11+?)\1+$/ && print "$_ is prime"'

# Print the sum of all the fields on a line
perl -MList::Util=sum -alne 'print sum @F'

# Print the sum of all the fields on all lines
perl -MList::Util=sum -alne 'push @S,@F; END { print sum @S }'
perl -MList::Util=sum -alne '$s += sum @F; END { print $s }'

# Shuffle all fields on a line
perl -MList::Util=shuffle -alne 'print "@{[shuffle @F]}"'
perl -MList::Util=shuffle -alne 'print join " ", shuffle @F'

# Find the minimum element on a line
perl -MList::Util=min -alne 'print min @F'

# Find the minimum element over all the lines
perl -MList::Util=min -alne '@M = (@M, @F); END { print min @M }'
perl -MList::Util=min -alne '$min = min @F; $rmin = $min unless defined $rmin && $min > $rmin; END { print $rmin }'

# Find the maximum element on a line
perl -MList::Util=max -alne 'print max @F'

# Find the maximum element over all the lines
perl -MList::Util=max -alne '@M = (@M, @F); END { print max @M }'

# Replace each field with its absolute value
perl -alne 'print "@{[map { abs } @F]}"'

# Find the total number of fields (words) on each line
perl -alne 'print scalar @F'

# Print the total number of fields (words) on each line followed by the line
perl -alne 'print scalar @F, " $_"'

# Find the total number of fields (words) on all lines
perl -alne '$t += @F; END { print $t}'

# Print the total number of fields that match a pattern
perl -alne 'map { /regex/ && $t++ } @F; END { print $t }'
perl -alne '$t += /regex/ for @F; END { print $t }'
perl -alne '$t += grep /regex/, @F; END { print $t }'

# Print the total number of lines that match a pattern
perl -lne '/regex/ && $t++; END { print $t }'

# Print the number PI to n decimal places
perl -Mbignum=bpi -le 'print bpi(n)'

# Print the number PI to 39 decimal places
perl -Mbignum=PI -le 'print PI'

# Print the number E to n decimal places
perl -Mbignum=bexp -le 'print bexp(1,n+1)'

# Print the number E to 39 decimal places
perl -Mbignum=e -le 'print e'

# Print UNIX time (seconds since Jan 1, 1970, 00:00:00 UTC)
perl -le 'print time'

# Print GMT (Greenwich Mean Time) and local computer time
perl -le 'print scalar gmtime'
perl -le 'print scalar localtime'

# Print local computer time in H:M:S format
perl -le 'print join ":", (localtime)[2,1,0]'

# Print yesterday's date
perl -MPOSIX -le '@now = localtime; $now[3] -= 1; print scalar localtime mktime @now'

# Print date 14 months, 9 days and 7 seconds ago
perl -MPOSIX -le '@now = localtime; $now[0] -= 7; $now[4] -= 14; $now[7] -= 9; print scalar localtime mktime @now'

# Calculate factorial of 5
perl -MMath::BigInt -le 'print Math::BigInt->new(5)->bfac()'
perl -le '$f = 1; $f *= $_ for 1..5; print $f'

# Calculate greatest common divisor (GCM)
perl -MMath::BigInt=bgcd -le 'print bgcd(@list_of_numbers)'

# Calculate GCM of numbers 20 and 35 using Euclid's algorithm
perl -le '$n = 20; $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $m'

# Calculate least common multiple (LCM) of numbers 35, 20 and 8
perl -MMath::BigInt=blcm -le 'print blcm(35,20,8)'

# Calculate LCM of 20 and 35 using Euclid's formula: n*m/gcd(n,m)
perl -le '$a = $n = 20; $b = $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $a*$b/$m'

# Generate 10 random numbers between 5 and 15 (excluding 15)
perl -le '$n=10; $min=5; $max=15; $, = " "; print map { int(rand($max-$min))+$min } 1..$n'

# Find and print all permutations of a list
perl -MAlgorithm::Permute -le '$l = [1,2,3,4,5]; $p = Algorithm::Permute->new($l); print @r while @r = $p->next'

# Generate the power set
perl -MList::PowerSet=powerset -le '@l = (1,2,3,4,5); for (@{powerset(@l)}) { print "@$_" }'

# Convert an IP address to unsigned integer
perl -le '$i=3; $u += ($_<<8*$i--) for "127.0.0.1" =~ /(\d+)/g; print $u'
perl -le '$ip="127.0.0.1"; $ip =~ s/(\d+)\.?/sprintf("%02x", $1)/ge; print hex($ip)'
perl -le 'print unpack("N", 127.0.0.1)'
perl -MSocket -le 'print unpack("N", inet_aton("127.0.0.1"))'

# Convert an unsigned integer to an IP address
perl -MSocket -le 'print inet_ntoa(pack("N", 2130706433))'
perl -le '$ip = 2130706433; print join ".", map { (($ip>>8*($_))&0xFF) } reverse 0..3'
perl -le '$ip = 2130706433; $, = "."; print map { (($ip>>8*($_))&0xFF) } reverse 0..3'

STRING CREATION AND ARRAY CREATION
----------------------------------

# Generate and print the alphabet
perl -le 'print a..z'
perl -le 'print ("a".."z")'
perl -le '$, = ","; print ("a".."z")'
perl -le 'print join ",", ("a".."z")'

# Generate and print all the strings from "a" to "zz"
perl -le 'print ("a".."zz")'
perl -le 'print "aa".."zz"'

# Create a hex lookup table
@hex = (0..9, "a".."f")

# Convert a decimal number to hex using @hex lookup table
perl -le '$num = 255; @hex = (0..9, "a".."f"); while ($num) { $s = $hex[($num%16)&15].$s; $num = int $num/16 } print $s'
perl -le '$hex = sprintf("%x", 255); print $hex'
perl -le '$num = "ff"; print hex $num'

# Generate a random 8 character password
perl -le 'print map { ("a".."z")[rand 26] } 1..8'
perl -le 'print map { ("a".."z", 0..9)[rand 36] } 1..8'

# Create a string of specific length
perl -le 'print "a"x50'

# Create a repeated list of elements
perl -le '@list = (1,2)x20; print "@list"'

# Create an array from a string
@months = split ' ', "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
@months = qw/Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec/

# Create a string from an array
@stuff = ("hello", 0..9, "world"); $string = join '-', @stuff

# Find the numeric values for characters in the string
perl -le 'print join ", ", map { ord } split //, "hello world"'

# Convert a list of numeric ASCII values into a string
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print pack("C*", @ascii)'
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print map { chr } @ascii'

# Generate an array with odd numbers from 1 to 100
perl -le '@odd = grep {$_ % 2 == 1} 1..100; print "@odd"'
perl -le '@odd = grep { $_ & 1 } 1..100; print "@odd"'

# Generate an array with even numbers from 1 to 100
perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'

# Find the length of the string
perl -le 'print length "one-liners are great"'

# Find the number of elements in an array
perl -le '@array = ("a".."z"); print scalar @array'
perl -le '@array = ("a".."z"); print $#array + 1'

TEXT CONVERSION AND SUBSTITUTION
--------------------------------

# ROT13 a string
'y/A-Za-z/N-ZA-Mn-za-m/'

# ROT 13 a file
perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file

# Base64 encode a string
perl -MMIME::Base64 -e 'print encode_base64("string")'
perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' file

# Base64 decode a string
perl -MMIME::Base64 -le 'print decode_base64("base64string")'
perl -MMIME::Base64 -ne 'print decode_base64($_)' file

# URL-escape a string
perl -MURI::Escape -le 'print uri_escape($string)'

# URL-unescape a string
perl -MURI::Escape -le 'print uri_unescape($string)'

# HTML-encode a string
perl -MHTML::Entities -le 'print encode_entities($string)'

# HTML-decode a string
perl -MHTML::Entities -le 'print decode_entities($string)'

# Convert all text to uppercase
perl -nle 'print uc'
perl -ple '$_=uc'
perl -nle 'print "\U$_"'

# Convert all text to lowercase
perl -nle 'print lc'
perl -ple '$_=lc'
perl -nle 'print "\L$_"'

# Uppercase only the first word of each line
perl -nle 'print ucfirst lc'
perl -nle 'print "\u\L$_"'

# Invert the letter case
perl -ple 'y/A-Za-z/a-zA-Z/'

# Camel case each line
perl -ple 's/(\w+)/\u$1/g'
perl -ple 's/(?<!['])(\w+)/\u\1/g'

# Strip leading whitespace (spaces, tabs) from the beginning of each line
perl -ple 's/^[ \t]+//'
perl -ple 's/^\s+//'

# Strip trailing whitespace (space, tabs) from the end of each line
perl -ple 's/[ \t]+$//'

# Strip whitespace from the beginning and end of each line
perl -ple 's/^[ \t]+|[ \t]+$//g'

# Convert UNIX newlines to DOS/Windows newlines
perl -pe 's|\n|\r\n|'

# Convert DOS/Windows newlines to UNIX newlines
perl -pe 's|\r\n|\n|'

# Convert UNIX newlines to Mac newlines
perl -pe 's|\n|\r|'

# Substitute (find and replace) "foo" with "bar" on each line
perl -pe 's/foo/bar/'

# Substitute (find and replace) all "foo"s with "bar" on each line
perl -pe 's/foo/bar/g'

# Substitute (find and replace) "foo" with "bar" on lines that match "baz"
perl -pe '/baz/ && s/foo/bar/'

SELECTIVE PRINTING AND DELETING OF CERTAIN LINES
------------------------------------------------

# Print the first line of a file (emulate head -1)
perl -ne 'print; exit'

# Print the first 10 lines of a file (emulate head -10)
perl -ne 'print if $. <= 10'
perl -ne '$. <= 10 && print'

# Print the last line of a file (emulate tail -1)
perl -ne '$last = $_; END { print $last }'
perl -ne 'print if eof'

# Print the last 10 lines of a file (emulate tail -10)
perl -ne 'push @a, $_; @a = @a[@a-10..$#a]; END { print @a }'

# Print only lines that match a regular expression
perl -ne '/regex/ && print'

# Print only lines that do not match a regular expression
perl -ne '!/regex/ && print'

# Print the line before a line that matches a regular expression
perl -ne '/regex/ && $last && print $last; $last = $_'

# Print the line after a line that matches a regular expression
perl -ne 'if ($p) { print; $p = 0 } $p++ if /regex/'

# Print lines that match regex AAA and regex BBB in any order
perl -ne '/AAA/ && /BBB/ && print'

# Print lines that don't match match regexes AAA and BBB
perl -ne '!/AAA/ && !/BBB/ && print'

# Print lines that match regex AAA followed by regex BBB followed by CCC
perl -ne '/AAA.*BBB.*CCC/ && print'

# Print lines that are 80 chars or longer
perl -ne 'print if length >= 80'

# Print lines that are less than 80 chars in length
perl -ne 'print if length < 80'

# Print only line 13
perl -ne '$. == 13 && print && exit'

# Print all lines except line 27
perl -ne '$. != 27 && print'
perl -ne 'print if $. != 27'

# Print only lines 13, 19 and 67
perl -ne 'print if $. == 13 || $. == 19 || $. == 67'
perl -ne 'print if int($.) ~~ (13, 19, 67)'

# Print all lines between two regexes (including lines that match regex)
perl -ne 'print if /regex1/../regex2/'

# Print all lines from line 17 to line 30
perl -ne 'print if $. >= 17 && $. <= 30'
perl -ne 'print if int($.) ~~ (17..30)'
perl -ne 'print if grep { $_ == $. } 17..30'

# Print the longest line
perl -ne '$l = $_ if length($_) > length($l); END { print $l }'

# Print the shortest line
perl -ne '$s = $_ if $. == 1; $s = $_ if length($_) < length($s); END { print $s }'

# Print all lines that contain a number
perl -ne 'print if /\d/'

# Find all lines that contain only a number
perl -ne 'print if /^\d+$/'

# Print all lines that contain only characters
perl -ne 'print if /^[[:alpha:]]+$/

# Print every second line
perl -ne 'print if $. % 2'

# Print every second line, starting the second line
perl -ne 'print if $. % 2 == 0'

# Print all lines that repeat
perl -ne 'print if ++$a{$_} == 2'

# Print all unique lines
perl -ne 'print unless $a{$_}++'

HANDY REGULAR EXPRESSIONS
-------------------------

# Match something that looks like an IP address
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/
/^(\d{1,3}\.){3}\d{1,3}$/

# Test if a number is in range 0-255
/^([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$/

# Match an IP address
my $ip_part = qr|([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|;
if ($ip =~ /^($ip_part\.){3}$ip_part$/) {
say "valid ip";
}

# Check if the string looks like an email address
/.+@.+\..+/

# Check if the string is a decimal number
/^\d+$/
/^[+-]?\d+$/
/^[+-]?\d+\.?\d*$/

# Check if the string is a hexadecimal number
/^0x[0-9a-f]+$/i

# Check if the string is an octal number
/^0[0-7]+$/

# Check if the string is binary
/^[01]+$/

# Check if a word appears twice in the string
/(word).*\1/

# Increase all numbers by one in the string
$str =~ s/(\d+)/$1+1/ge

# Extract HTTP User-Agent string from the HTTP headers
/^User-Agent: (.+)$/

# Match printable ASCII characters
/[ -~]/

# Match unprintable ASCII characters
/[^ -~]/

# Match text between two HTML tags
m|([^<]*)|
m|(.*?)|

# Replace all tags with 
$html =~ s|<(/)?b>|<$1strong>|g

# Extract all matches from a regular expression
my @matches = $text =~ /regex/g;

PERL ONE-LINERS EXPLAINED E-BOOK
--------------------------------

I am writing an ebook based on the one-liners in this file. If you want to
support my work and learn more about these one-liners, you can get a copy
of my ebook soon at:

https://catonmat.net/perl-book/

The ebook is based on the 7-part article series that I wrote on my blog.
In the ebook I reviewed all the one-liners, improved explanations and added
new ones.

You can read the original article series here:

https://catonmat.net/perl-one-liners-explained-part-one/
https://catonmat.net/perl-one-liners-explained-part-two/
https://catonmat.net/perl-one-liners-explained-part-three/
https://catonmat.net/perl-one-liners-explained-part-four/
https://catonmat.net/perl-one-liners-explained-part-five/
https://catonmat.net/perl-one-liners-explained-part-six/
https://catonmat.net/perl-one-liners-explained-part-seven/

HAVE FUN
--------

I hope you found these one-liners useful and have fun!

#---end of file---

Perl One-Liners Explained Article Series

If you’re curious how the one-liners work, take a look at the Perl One-Liners Explained article series. I have explained all of them in the following articles:

Part I: File spacing. Part II: Line numbering. Part III: Calculations. Part IV: String creation and array creation. Part V: Text conversion and substitution. Part VI: Selective printing and deleting of certain lines. Part VII: Handy regular expressions.

The next, and final, thing I am doing with these one-liners is releasing Perl One-Liners Explained e-book. Stay tuned!

Update: I finished writing the e-book. Check it out!
Awk and Sed One-Liners Explained

I based the perl1line.txt file on the famous awk1line.txt and sed1line.txt files. That’s how I actually learned awk and sed. I studied these two files inside out and learned everything I could about sed and awk. Later I decided to contribute back to the community and explained all one-liners in these two article series:

Awk One-Liners Explained https://catonmat.net/awk-one-liners-explained-part-one Sed One-Liners Explained https://catonmat.net/sed-one-liners-explained-part-one

Awk posts have been read over 2,000,000 times and sed posts over 1,500,000 times.

Have fun writing one-liners and see you next time!
Read more articles →
Thanks for reading my post. If you enjoyed it and would like to receive my posts automatically, you can subscribe to new posts via rss feed or email.

Perl One-Liners Explained, Part VII: Handy Regular Expressions

Dnode – A Node.js Module You Should Know About
Secret message: Use coupon code JELLYLING to get a discount at my company Browserling!

Announcing my third e-book “Perl One-Liners Explained”

Buy it now for just $19.99

PayPal - The safer, easier way to pay online!

Hello everyone! I’m happy to announce my 3rd e-book called Perl One-Liners Explained. This book is based on the Perl One-Liners Explained article series that I wrote over the last 3 years and that has been read over 1,000,000 times!

I went through all the one-liners in the article series, improved explanations, fixed mistakes and typos, added a bunch of new one-liners, added an introduction to Perl one-liners and a new chapter on Perl’s special variables.
Table of Contents

The e-book explains 130 unique one-liners. Many of one-liners are presented in several different ways so the total number of one-liners in the book is over 200.

The e-book is divided into the following chapters:

Preface. 1. Introduction to Perl One-Liners. 2. Spacing. 3. Numbering. 4. Calculations 5. String Creation and Array Creation. 6. Text Conversion and Substitution. 7. Selective Printing and Deleting of Lines. 8. Handy Regular Expressions. 9. perl1line.txt Appendix A. Perl's Special Variables. Index.

What are Perl One-Liners?

Perl one-liners are small and awesome Perl programs that fit in a single line of code and they do one thing really well. These things include changing line spacing, numbering lines, doing calculations, converting and substituting text, deleting and printing certain lines, parsing logs, editing files in-place, doing statistics, carrying out system administration tasks, updating a bunch of files at once, and many more.

Let’s take a look at several practical examples that you can easily do with one-liners. All these examples and many more are explained in the e-book.

I have also made the first chapter of the book, Introduction to Perl One-Liners, freely available. Please download the e-book preview to read it.

Example 1: Replace a string in multiple files at once

1
perl -p -i.bak -e 's/Config/config/g' conf1 conf2 conf3

Suppose you have 3 configuration files, and you discover that you made a mistake and need to replace all occurrences of Config with config. This one-liner does just that. It executes the s/Config/config/g that replaces all occurrences of Config with config on all lines. And since you’re smart about it, you always do -i.bak to make backup files in case something goes wrong.

I explain the -i, -p, and -e arguments in the e-book in great detail.

Example 2: Generate a random 8 character password

1
perl -le 'print map { ("a".."z")[rand 26] } 1..8'

This one-liner generates and prints a random 8 character password. It uses the list range operator .. operator to produce all strings from “a” to “z”, which is the alphabet. Then a random letter is chosen by rand 26 and this operation is repeated 8 times.

Example 3: URL-escape a string

1
perl -MURI::Escape -lne 'print uri_escape($string)'

Here we use the URI::Escape module from CPAN. It exports the uri_escape function that does URL-escaping.

You can install this module from CPAN by running perl -MCPAN -e’install URI::Escape’ on the command line.

I have this one-liner as an alias actually for both URL-escaping and unescaping URL-escaping as it’s such a common thing to do:

1
2
urlescape () { perl -MURI::Escape -lne 'print uri_escape($_)' <<< "$1" }
urlunescape () { perl -MURI::Escape -lne 'print uri_unescape($_)' <<< "$1"; }

Then I can do this in the shell:

1
2
3
4
5
$ urlescape "https://catonmat.net"
https%3A%2F%2Fcatonmat.net

$ urlunescape https%3A%2F%2Fcatonmat.net
https://catonmat.net

Very useful!

Example 4: Print all lines from line 17 to line 30

1
perl -ne 'print if 17..30'

Here we use the binary flip-flop operator .. that becomes true when the input line number is 17, stays true while the line number is less than or equal to 30, and then becomes false. Combining the flip-flop operator with print if makes it print only lines 17-30.

Example 5: Remove all consecutive blank lines, leaving just one

1
perl -00pe0

I included this one-liner here in the examples just to show you how funny and obscure one-liners can get. This one-liner deletes all repeated blank lines from the input or from the given file. It does it by enabling the paragraph slurp mode through -00 command line argument, which reads the input paragraph-by-paragraph, rather than line-by-line, and prints the paragraphs. This way any number of blank lines between the paragraphs get ignored.

I explain this one-liner in more details in the e-book.

As I hope you can see, knowing how to write one-liners is very useful. It was one of my top priority tasks through the years to become very efficient in the shell. Literally every day when I’m programming, I have to do all kinds of data processing tasks, changing files, verifying output, doing quick calculations, parsing data, etc, and knowing Perl one-liners makes it really fast to get things done.

Now that I have written this e-book, you can become very efficient, too. Enjoy!
Book Preview

I prepared a free book preview that contains the first 13 pages of the book. It includes the table of contents, preface, introduction to Perl one-liners and the first page of the second chapter.
Perl One-Liners Explained. Book Preview.
Buy it now!

The price of the e-book is $19.99 and it can be purchased via PayPal:
PayPal - The safer, easier way to pay online!

After you have made the payment, my automated e-book processing system will send you the PDF e-book in a few minutes!
Tweet about my book!

Help me spread the word about my new book. I prepared a special link that you can use to tweet about it.
What’s next?

I really love writing about programming and I have planned writing many more books. The next few are going to be a book on mastering vim, a practical guide on how to be anonymous on the web, and the catonmat book.
Enjoy!

Enjoy the book and let me know how you liked it.

Also if you’re interested, take a look at my other two e-books. The 1st one is called “Awk One-Liners Explained” and the 2nd one is called “Sed One-Liners Explained” They’re written in a similar style as this e-book and they teach practical Awk and Sed through many examples.

Finally, if you enjoy my writing, you can subscribe to my blog, follow me on Twitter or Google+.
Read more articles →
Thanks for reading my post. If you enjoyed it and would like to receive my posts automatically, you can subscribe to new posts via rss feed or email.
How Browserling Works [art]
Announcing dedicated servers for Browserling
Secret message: Use coupon code JELLYLING to get a discount at my company Browserling!