LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 02-01-2013, 08:55 AM   #1
dedec0
Member
 
Registered: May 2007
Posts: 55

Rep: Reputation: 3
How to make Vim (scripts) write to stdout? (or, how to use Vim as a filter program?)


Hi

This sounds like a simple idea, but I'm having no luck. The basic idea is to use vim as a filter program, like sort is.

I made a script for vim, to be executed and the results written on stdout.

My intended command line should be something like:

Code:
ls -l | vim -s testScript.vim - | cat
I have tried to do ":w! /dev/stdout" inside the Vim script, but it didn't work well.

I know how, but I didn't want to use temporary files, e.g.:

Code:
ls -l > /tmp/file; vim -s testScript.vim /tmp/file; cat /tmp/file| ...
This doesn't really fit my needs, and it's a bit ugly too.
 
Old 02-01-2013, 09:05 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
Why use vi(m), an editor when it comes down to it, in the first place? Other, better tools are available for filtering (sed and awk to name just 2). Vi(m) looks like the wrong tool for the job in this case.

I'm sure that all the actions done in the vi(m) script can also be done with other, more suitable tools.
 
2 members found this post helpful.
Old 02-01-2013, 09:14 AM   #3
schneidz
Senior Member
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-19-live-usb/ aix
Posts: 3,822

Rep: Reputation: 586Reputation: 586Reputation: 586Reputation: 586Reputation: 586Reputation: 586
what are you trying to filter out.?

are you trying to recreate grep for learning purposes ?
 
Old 02-01-2013, 10:29 AM   #4
dedec0
Member
 
Registered: May 2007
Posts: 55

Original Poster
Rep: Reputation: 3
@druuna

Vim is scriptable. Why not use it? Because it is an editor and I'm more familiar with its commands than with awk? I think not.

Vim looks like the wrong tool? Why? The example command I gave was just to show the chained use.

I know how to do everything I need now, in Vim. I only stuck in the final part, the standard output... since the problem is this simple, I didn't even bother to comment what "processing" I'm doing with the Vim script, it is irrelevant.

@schneidz

I'm not trying to filter out things, in this sense. I'll use Vim to make transformations on text and give the output to another command. I can do these transforms in Vim, but not in other programs (although I know how to use, a bit, a few).
 
Old 02-01-2013, 11:04 AM   #5
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 1,990

Rep: Reputation: 512Reputation: 512Reputation: 512Reputation: 512Reputation: 512Reputation: 512
Quote:
Originally Posted by dedec0 View Post
@druuna

Vim is scriptable. Why not use it? Because it is an editor and I'm more familiar with its commands than with awk? I think not.
Scripted vim is sed. Most of the commands used in vim are designed for use with a file going backward/forward in direction - scripting an edit is not a stream.
Quote:
Vim looks like the wrong tool? Why? The example command I gave was just to show the chained use.
vim has at its base the editing commands of sed. sed is a much smaller and faster working program than vim.
Quote:

I know how to do everything I need now, in Vim. I only stuck in the final part, the standard output... since the problem is this simple, I didn't even bother to comment what "processing" I'm doing with the Vim script, it is irrelevant.

@schneidz

I'm not trying to filter out things, in this sense. I'll use Vim to make transformations on text and give the output to another command. I can do these transforms in Vim, but not in other programs (although I know how to use, a bit, a few).
Which is exactly what sed is for.
 
Old 02-01-2013, 11:17 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
Quote:
Originally Posted by dedec0 View Post
@druuna

Vim is scriptable. Why not use it? Because it is an editor and I'm more familiar with its commands than with awk? I think not.

Vim looks like the wrong tool? Why? The example command I gave was just to show the chained use.

I know how to do everything I need now, in Vim. I only stuck in the final part, the standard output... since the problem is this simple, I didn't even bother to comment what "processing" I'm doing with the Vim script, it is irrelevant.

@schneidz

I'm not trying to filter out things, in this sense. I'll use Vim to make transformations on text and give the output to another command. I can do these transforms in Vim, but not in other programs (although I know how to use, a bit, a few).
vim is the wrong tool for the job. Why? Well, you just started a thread about one of the reasons.... This tool is not meant to be used in this way.

(s)ed is what is mostly used by vi(m). Why not take the jump and start using sed?

Btw: There is a command that lets you define an editor to be used in a pipe: vipe (from the moreutils package). Never used it myself and I don't know if any (automated) vim scripting is allowed/possible.
 
Old 02-01-2013, 11:34 AM   #7
dedec0
Member
 
Registered: May 2007
Posts: 55

Original Poster
Rep: Reputation: 3
All right, it's nice to discuss these things, but it doesn't answer the question. If it is *possible* to do it with Vim, I want to find out how.

If someday after learning to work better with other tools I laugh at this, well, there is nothing wrong with it.

@jpollard
Your definition of "right tool" is not like mine, for this situation.

Even the "base of Vim" being the same commands sed has, Vim has more, agree?? They're different. Or else, there should be a tool to convert Vim editing scripts to sed scripts available...

Quote:
Quote:
make transformations on text and give the output to another command
Which is exactly what sed is for.
Not only sed, but a gazillion programs (read text on stdin, write *anything* to stdout). It was a very loose definition.
 
Old 02-01-2013, 12:15 PM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
@dedec0: You cannot dot it with using just vim.

I just played around a bit with vipe (see post #6) and with a little trickery you can achieve what you are after (command | <vim+script> | command).

vipe expects the EDITOR variable to be set to your editor (EDITOR='vim'). You can add the -s testScript.vim part, i.e.: EDITOR='vim -s testScript.vim'.

The following can now be done:
Code:
command | vipe | command
WARNING: Other commands might also look at the EDITOR variable and will start behaving unexpected when the -s part is added!!

This will set the EDITOR variable for use by vipe only:
Code:
command | EDITOR='vim -s script.vim' vipe | command
Like I said, wrong tool for the job
 
1 members found this post helpful.
Old 02-01-2013, 12:20 PM   #9
schneidz
Senior Member
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-19-live-usb/ aix
Posts: 3,822

Rep: Reputation: 586Reputation: 586Reputation: 586Reputation: 586Reputation: 586Reputation: 586
why dont you explain what you are trying to achive (and provide a before and after picture) and maybe someone will provide a solution.
 
1 members found this post helpful.
Old 02-01-2013, 12:25 PM   #10
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,953
Blog Entries: 54

Rep: Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732Reputation: 2732
Heh, that reminded me I used to script Vi before I learned to use other tools ;-p


Quote:
Originally Posted by dedec0 View Post
Your definition of "right tool" is not like mine, for this situation.
You don't say. About any post you find about using Vim in a pipe says it's like using a small tactical nuke where a double tap would do... Anyway how about pipe.vim : make vim part of a unix pipe and allow it to edit the pipe contents?
 
1 members found this post helpful.
Old 02-01-2013, 12:41 PM   #11
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
I don't think it's possible to "stream" input into vim, but you CAN use it in scripts. You can supply it with ex-mode commands, which it will execute directly on the given file without opening up the buffer visually.

You just need to use the -e option for ex mode, and either the -c option to pass commands directly as arguments or the -s option to control it through stdin (see the help documentation for details).

And the "file" can be a process substitution or named pipe (or just use the 'r !' ex command, for that matter), so you can operate on the output of commands too.

As an example, the following code will strip ascii color codes from the output of ls:

Code:
printf '%s\n' '%s/\e[[][0-9;]\+m//g' '%p' | vim -es <( ls --color=always )
Check the web for lots more on what vim can do. This wiki, for example:
http://vim.wikia.com/

Edit: One more hint for you. I recently discovered that there's a 'norm' command, which lets you execute normal mode commands in ex mode, giving you even more power.


In any case though, I'd recommend using ed instead for much of this. It operates similarly to how vim does above, in a more limited, but lighter fashion.

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)

Notice that ed can also operate on the output of commands directly, by using them in place of a filename, prepended with "!".


Of course if you want to get really fancy, switch to perl or something like that.

Last edited by David the H.; 02-01-2013 at 01:02 PM.
 
1 members found this post helpful.
Old 02-01-2013, 01:11 PM   #12
dedec0
Member
 
Registered: May 2007
Posts: 55

Original Poster
Rep: Reputation: 3
@druuna
I can do it with Vim. I have done it now.

And now I heard about vipe and moreutils package. Really good!


@unSpawn
Quote:
Heh, that reminded me I used to script Vi before I learned to use other tools ;-p
LOL, I feel that's exactly the situation I'm in!

My situation don't need efficiency. So, no worry about the bloat of Vim in the face of sed or awk.

That pipe.vim script is very cool! It can make you feel like a filter program...

Tested pipe.vim (ugly source, Perl does, eek!), but works charmly!


-------------------

I only like to think of using sed when I can easily think of the "editing" in one pass. Moving around with SED, multiline spaces, etc... gets too ugly for me - maybe for the lack of practice. It is a matter of time, anyway. I'm confident that I can do it with awk, after I learn it. Not now...

Thank you all!
 
Old 02-01-2013, 01:54 PM   #13
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 1,990

Rep: Reputation: 512Reputation: 512Reputation: 512Reputation: 512Reputation: 512Reputation: 512
Quote:
Originally Posted by dedec0 View Post
...
@jpollard
Your definition of "right tool" is not like mine, for this situation.

Even the "base of Vim" being the same commands sed has, Vim has more, agree?? They're different. Or else, there should be a tool to convert Vim editing scripts to sed scripts available...
Where do you thing vi/vim started from? vi was just a front end to ed, which was a file level sed rather than a stream sed (you could go back to the beginning of a file).

There are more commands... but they are also aimed at a terminal and generate escape sequences, highlighting, backlighting, terminal dependent tabbing, cursor movement, restarting edits, undoing edits, ... None of which work when there is no terminal for such interactive functions, and thus unusable.
Quote:

Not only sed, but a gazillion programs (read text on stdin, write *anything* to stdout). It was a very loose definition.
Yup. Things that vi/vim isn't designed for and is not very good at.
 
Old 02-04-2013, 05:45 AM   #14
dedec0
Member
 
Registered: May 2007
Posts: 55

Original Poster
Rep: Reputation: 3
Thanks, [b]David the H.[b]. I didn't had the time to read your post last week. You pointed some nice things. I have used Vim in scripts before, but not using Ex mode like you showed.
 
  


Reply

Tags
vim


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How do you write <Esc> in the vim command line (:vim command). stf92 Linux - General 6 06-26-2011 05:33 AM
LXer: Personalize and Optimize Vim editor using Packtís new Vim 7.2 book LXer Syndicated Linux News 0 05-20-2010 10:20 PM
Switching from vim to vim -g from inside vim iDragoon Linux - Software 4 05-15-2009 11:46 AM
Editor comparison: vim VS vim-lite, Cleaning vim Ruler2112 *BSD 4 04-13-2009 04:26 PM
LXer: Five scripts that make life easier with Vim LXer Syndicated Linux News 0 06-07-2007 05:46 PM


All times are GMT -5. The time now is 11:20 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration