this post was submitted on 15 Apr 2025
22 points (95.8% liked)

Linux

53639 readers
990 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
22
Help with sed commands (lemmy.dbzer0.com)
submitted 1 week ago* (last edited 1 week ago) by orsetto@lemmy.dbzer0.com to c/linux@lemmy.ml
 

Hi all! I have always only used sed with s///, becouse I've never been able to figure out how to properly make use of its full capabilities. Right now, I'm trying to filter the output of df -h --output=avail,source to only get the available space from /dev/dm-2 (let's ignore that I just realized df accepts a device as parameter, which clearly solves my problem).

This is the command I'm using, which works:

df -h --output=avail,source \
    | grep /dev/dm-2 \
    | sed -E 's/^[[:blank:]]*([0-9]+(G|M|K)).*$/\1/

However, it makes use of grep, and I'd like to get rid of it. So I've tried with a combiantion of t, T, //d and some other stuff, but onestly the output I get makes no sense to me, and I can't figure out what I should do instead.

In short, my question is: given the following output

$ df -h --output=avail,source 
Avail Filesystem
  87G /dev/dm-2
 1.6G tmpfs
  61K efivarfs
  10M dev
...

How do I only get 87G using only sed as a filter?

EDIT:

Nevermind, I've figured it out...

$ df -h --output=avail,source \
    | sed -E 's/^[[:blank:]]*([0-9]+(G|M|K))[[:blank:]]+(\/dev\/dm-2).*$/\1/; t; /.*/d'
85G
top 21 comments
sorted by: hot top controversial new old
[–] smeg@feddit.uk 10 points 1 week ago (2 children)

Are you opposed to using awk? Printing out a specific column is basically the only thing I actually know how to do with it: df -h --output=avail,source | awk '/dm-2/ {print $1}'

[–] thingsiplay@beehaw.org 5 points 1 week ago

A perfect example when to use AWK.

[–] orsetto@lemmy.dbzer0.com 3 points 1 week ago (1 children)

Are you opposed to using awk?

Not at all, I'm just not familiar with it so I find it confusing.

Although, looking at your command, i think I understand what it means

[–] smeg@feddit.uk 2 points 1 week ago* (last edited 1 week ago) (1 children)

I had to check the syntax because I also don't use it enough to be sure, definitely a bit weird. Basically just grep for a regex and print the specified column.

[–] thingsiplay@beehaw.org 2 points 1 week ago

Just a thumb of rule to make sense of it: A column in AWK is by default any space separated part. You can change the column separator to any other character too with -F ":" in example would be a double colon. There is also a way to print all columns, but with certain exceptions. In example print all, but the third and fourth columns: ls -l | awk '{$3=""; $4=""; print $0}' . Admittely I forget this syntax often and have to look for it again.

[–] gnuhaut@lemmy.ml 8 points 1 week ago* (last edited 1 week ago) (2 children)

You can do something like this to emulate grep:

sed -n '/myregex/p'

The -n suppresses auto-printing. That command should interpreted as: find line matching /myregex/ and then print (p) it.

You can then combine this with s (substitute):

sed '/myregex/ s/from/to/ p`

The complete command is then something like:

LANG=C df -h --output=avail,source | sed -n '/\/dev\/nvme0n1p2/ s/^\s*\([0-9.]\+[KMGT]\).*/\1/ p'

Note that the output can be something like 2.3G, but in my locale that would be 2,3G which is why I added LANG=C.

Easier IMHO is awk:

awk '/myregex/ {print $1}'

prints the first field.

[–] orsetto@lemmy.dbzer0.com 2 points 1 week ago (1 children)

The -n suppresses auto-printing

That's something I was missing! It makes the command look definitely less complicated.

Also, what is the difference between separating expressions with a space vs with a semicolon?

Easier IMHO is awk

Eh, that's another beast. For some reason tho I find sed more appealing.

[–] gnuhaut@lemmy.ml 2 points 1 week ago (1 children)

What expressions?

I mean awk is more powerful, it has variables, function, can do arithmetic and format strings, and such proper scripting language features. And the GNU awk manual is rather well written.

[–] orsetto@lemmy.dbzer0.com 2 points 1 week ago (1 children)

I meant to ask what is the difference between, i.e., sed '/myregex/ s/from/to/ p' and sed '/myregex/ s/from/to/ ; p', but while testing to explain what i meant I answered myself, and in the process I also understood what addresses are ehe

Right, awk is a proper programming language, right? that'll be for another day...

[–] gnuhaut@lemmy.ml 2 points 1 week ago (1 children)

Oh yeah that, so technically (and I was confused about this), the p in s/from/to/p is not the same as the p command, it's a flag for the s command that tells it to print the output. You could do multiple commands like /re/ {s/a/b/;p} for the same result, by using a {} block.

If you do, say, /re/ s/a/b/; p those would be separate command, the first does the thing on lines matching /re/, while the p has no address range (e.g. regex) associated with it, so it gets executed for all the lines.

[–] orsetto@lemmy.dbzer0.com 2 points 1 week ago (1 children)

I see. I guess what confused me was that i didn't understand what addresses were.

Thank you for your explanations :)

[–] gnuhaut@lemmy.ml 1 points 1 week ago (1 children)

Yeah no problem.

What's maybe interesting is how sed came to be. Back in early days of Unix, they had ed as their editor (or, as some old manpage says, "Ed is the standard text editor.")

Sed has basically the same commands as ed. You typed 3d to delete the third line, or 10,20p to print lines 10 to 20. They only had teletypes back then, which are basically a keyboard and dot-matrix printer with one of those continuous papers for output, prior to when hardware terminals with CRT screens were a thing.

Anyway, someone thought it would be useful if you could put ed-style editing commands inside shell pipelines, and ed doesn't work for that. It is used for interactively editing files in place, and so gets commands from stdin. You can't pipe any files into it. So the "stream editor", sed, was born.

Also interesting: There were improved modified versions of ed going around like em and later ex. The original vi was a "visual mode" extension for ex, and you can still run ex/ed commands from vi by typing : first, e.g. you can delete line 3 by typing :3d inside vi.

[–] orsetto@lemmy.dbzer0.com 2 points 1 day ago

12 days late, but thanks for the bit of history, I always enjoy this stuff :)

[–] bizdelnick@lemmy.ml 2 points 1 week ago (1 children)

You can then combine this with s (substitute):

sed '/myregex/ s/from/to/ p'

This is not combining commands. In your example p is a modifier to the s/// command.

[–] gnuhaut@lemmy.ml 2 points 1 week ago

Yeah you're right, thanks. I did figure this out on my own already, see my other comment:

Oh yeah that, so technically (and I was confused about this), the p in s/from/to/p is not the same as the p command, it’s a flag for the s command that tells it to print the output.

[–] unlawfulbooger@lemmy.blahaj.zone 4 points 1 week ago* (last edited 1 week ago) (1 children)

The easiest way is probably without sed, which you mentioned:

df -h --output=avail /dev/dm-2| tail -n1

But purely with sed it would be something like this:

df -h --output=avail,source | sed -n ‘/\/dev\/dm-2/s!/dev/dm-2!!p’

-n tells sed to not print lines by default

/[regex]/ selects the likes matching regex. We need to escape the slashes inside the regex.

s/// does search-and-replace, and has a special feature: it can use any character, not just a slash. So I used three exclamation points instead , so that I don’t need to escape the slashes. Here we replace the device with the empty string.

p prints the result

Check the sed man page for more details: https://linux.die.net/man/1/sed

[–] orsetto@lemmy.dbzer0.com 1 points 1 week ago* (last edited 1 week ago) (1 children)

!/dev/dm-2!!p

This weired me out until I read the explanation. I'm so used to the slashes lol

In the end I've used the first command you wrote, because KISS, but I appreciate your explanation

Check the sed man page for more details

Yes that's been my only source so far, but to be honest it's really cryptic. it might just be that I'm used to syscalls man pages (also sed is kinda complex it can't be easy to write a single man page for it)

[–] unlawfulbooger@lemmy.blahaj.zone 1 points 1 week ago (1 children)

In the end I've used the first command you wrote, because KISS, but I appreciate your explanation

There’s no shame in combining multiple tools, that’s what pipelines are all about 😄.

Also there’s a different tool that I would use if I want to output a specific column: awk

df -h —output=avail,source | awk ‘/\/dev\/dm-2/ {print $1}’

For lines matching /dev/dm-2 print the first column. awk splits columns on whitespace by default.

But I would probably use grep+awk.

Sed is definitely a very powerful tool, which leads to complex documentation. But I really like the filtering options before using the search/replace.

You can select specific lines, with regex or by using a line number; or you can select multiple lines by using a comma to specify a range.

E.g. /mystring/,100s/input/output/g: in the lines starting from the first match of /mystring/ until line 100, replace input with output

[–] orsetto@lemmy.dbzer0.com 2 points 1 day ago (1 children)

There’s no shame in combining multiple tools, that’s what pipelines are all about

Not at all, but some times it's just funny

You can select specific lines, with regex or by using a line number; or you can select multiple lines by using a comma to specify a range.

Yep, learning this made sed even more useful to me.

I also gave awk a try and now i know what i've missed all these years

(Also, sorry for the 12 days old reply :))

I get it, sometimes you just do something for the challenge.

It’s really great what you can accomplish when you know a little more than the bare minimum of the tools at your disposal (^^,)

And I had the same experience after learning a bit more about awk for the fist time, hahaha.

[–] bizdelnick@lemmy.ml 2 points 1 week ago* (last edited 1 week ago)

Just disable printing out with the -n option and add the p modifier to the s/// command to print out lines where substitution has occured. sed -n 's/your-regexp/replacement/p'