LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-06-2017, 08:52 AM   #1
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,143

Rep: Reputation: 43
Question A quick one: in Vim, how do I erase the \n in the end of a file?


I have a text file with one line. The command 'hd' outputs this for it:

Code:
$ hd entrada.txt 
00000000  7b 7b 31 2c 32 2c 33 2c  30 7d 2c 7b 31 2c 31 2c  |{{1,2,3,0},{1,1,|
00000010  31 2c 32 7d 2c 7b 30 2c  2d 31 2c 33 2c 39 7d 7d  |1,2},{0,-1,3,9}}|
00000020  0a                                                |.|
00000021
I do not want that \n to end the file. The last byte must be "}". I opened my usual editor, Vim, used the command :%s/\n//ge . File showed as changed, so I saved it, quit the editor and hexdumped it again. The \n is still there! (actually, the file did not change any byte)

Second try: editing the file with "vim -b" instead. Did not work either (or I did not know how to use it, please tell me how!).

About Vim's -b:

Quote:
-b Binary mode. A few options will be set that makes it possible to edit a binary or executable file.
Can I make it with Vim? Reversing the hexdump is easy for this file, but I may need to do it to others, and quickly. One line commands to use in "debianish" distros (my usual) are wanted! :D

==================================================================================================
Edit: The hexdumping command that can revert it is xxd, not hd or hexdump. See these terminal commands (with some comment in them) and their output:

Code:
$ # Do a plain hexdump (which is only the concatenated hexvalues of file bytes)
$ xxd -p entrada.txt
7b7b312c322c332c307d2c7b312c312c312c327d2c7b302c2d312c332c39
7d7d0a

$ # Make the dump again, saving it to a file
$ xxd -p entrada.txt > plain.hexdump.txt

$ # Manually edit the dump file, removing the two "0a" chars, the end of last line
$ vim plain.hexdump.txt

$ # Revert the new hexdump, saving it in file "entrada.txt"
$ xxd -p -r plain.hexdump.txt > entrada.txt

$ # Check that the file does have the unwanted '\n'
$ xxd -p entrada.txt
7b7b312c322c332c307d2c7b312c312c312c327d2c7b302c2d312c332c39
7d7d

$  # Great! Now, how to do it without the manual editing step?

Last edited by dedec0; 05-06-2017 at 09:19 AM.
 
Old 05-06-2017, 09:38 AM   #2
mithrantor
LQ Newbie
 
Registered: May 2017
Location: France
Posts: 6

Rep: Reputation: Disabled
Hi,
I do not know how to solve your problem using Vim but I was able (I think) to reproduce it. I used emacs to solve it. I don't know if you can use emacs but if you can, it may help you.
Open your file with emacs then move to the end of the last line. Select the last \n : [Ctrl+Space] then move down to the next line. Copy it with [Meta+w] (Alt key + w). Then move to the previous line. Search and replace the las t \n : [Meta+%] to enter search and replace mode, the paste the \n using [Ctrl+Y] and finally hit [Enter] two times and then [!] to confirm the replacement.
[Ctr+x s] to save and [Ctrl+x c] to exit.
It works for me, the \n was present when I hd my file even if I can't see while editing the file and after that it disappeared. See :
Before :
Code:
hd test 
00000000  74 65 73 74 0a                                    |test.|
00000005
After :
Code:
00000000  74 65 73 74                                       |test|
00000004
I hope it can help you.
 
Old 05-06-2017, 09:44 AM   #3
mithrantor
LQ Newbie
 
Registered: May 2017
Location: France
Posts: 6

Rep: Reputation: Disabled
May be you can do it that way :
Code:
$ head --bytes=-1 entrada.txt > entrada_copy.txt
It will display the all file into entrada_copy.txt except the last byte (the unwanted \n).

Last edited by mithrantor; 05-06-2017 at 10:30 AM. Reason: Add a missing s to byte
 
1 members found this post helpful.
Old 05-06-2017, 10:03 AM   #4
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,143

Original Poster
Rep: Reputation: 43
Talking

Quote:
Originally Posted by mithrantor View Post
May be you can do it that way :
Code:
$ head --byte=-1 entrada.txt > entrada_copy.txt
It will display the all file into entrada_copy.txt except the last byte (the unwanted \n).
Maybe you are right! ;) hehehehe

Thank you very much! I did not imagine using the head command to deal with raw bytes - never done it before, until now.

Just a small detail: head's manual page says the option is "--bytes", and not "--byte":

Code:
       -c, --bytes=[-]N
              print the first N bytes of each  file;  with  the  leading  `-',
              print all but the last N bytes of each file

Although the command works as you gave, I do not know why.

Last edited by dedec0; 05-06-2017 at 10:06 AM.
 
Old 05-06-2017, 10:12 AM   #5
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,143

Original Poster
Rep: Reputation: 43
Just a note: I am not marking this thread as solved because a solution in Vim was not yet provided (as asked in the title).

My main problem is solved, though, as I can write a command to remove the last N bytes of any file and generate another one without them.
 
Old 05-06-2017, 10:26 AM   #6
mithrantor
LQ Newbie
 
Registered: May 2017
Location: France
Posts: 6

Rep: Reputation: Disabled
Glad it helped you. (Sorry for missing s in the command line, it worked without ? Some magic in the air...).I was able to solve the problem by using the search and replace mode in emacs (by replacing the last invisible \n by nothing) and it worked. But I don't know how to do this in Vim...

Last edited by mithrantor; 05-06-2017 at 10:31 AM.
 
Old 05-06-2017, 10:45 AM   #7
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,401
Blog Entries: 10

Rep: Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212
i'm not using vim, but other programming editors (nano, geany) automatically add a newline at the end of a file.
usually this is configurable.
 
Old 05-06-2017, 11:35 AM   #8
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,143

Original Poster
Rep: Reputation: 43
@mithrantor: Then the missing S, and the head command working without it, is a mistery we cannot explain - a good one! (: Strange that a global replace of '\n' with nothing works in Emacs but not in Vim. Vim replaces the last newline with nothing, as we ask it to do. But when we save the file and quit the editor, it is added again (yes, a kind of paradox). Both are very mature editors with many advanced features. (...) You said you do not know how to do that in Vim, but I said in the original post! It is with the command ":%s/\n//ge". This should be clear to every Vim user, I imagine. I assume we are in Vim's normal mode, and then we type all but the quotes and then hit the enter key. It will replace all "\n" bytes with nothing, removing them. All but the one ending the last line, as I can say now.

@ondoho I guessed it was configurable, but I could not find what is the setting for such thing in Vim. I tried the binary mode editing, imagining it would solve the "issue", but it did not.

Last edited by dedec0; 05-06-2017 at 11:37 AM.
 
Old 05-06-2017, 11:47 AM   #9
mithrantor
LQ Newbie
 
Registered: May 2017
Location: France
Posts: 6

Rep: Reputation: Disabled
@dedec0 yep I saw how you tried to do that using Vim but in emacs I do not use \n in search mode, instead of that I literally copy/past the last line break. It's in that way that I don't know if you can do it in Vim (I don't even know if it makes a difference)...

Last edited by mithrantor; 05-06-2017 at 01:01 PM.
 
Old 05-06-2017, 01:33 PM   #10
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,432

Rep: Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055
Quote:
Originally Posted by dedec0 View Post
I have a text file with one line. The command 'hd' outputs this for it:

Code:
$ hd entrada.txt 
00000000  7b 7b 31 2c 32 2c 33 2c  30 7d 2c 7b 31 2c 31 2c  |{{1,2,3,0},{1,1,|
00000010  31 2c 32 7d 2c 7b 30 2c  2d 31 2c 33 2c 39 7d 7d  |1,2},{0,-1,3,9}}|
00000020  0a                                                |.|\n
00000021\n
As I read it, your file is ending in TWO line feeds (\n)...indicated above with my manually entered \n
I only use vim when I have to...I prefer nano...but positioning the cursor on line 21 and keying dd should delete that last, empty, line. Yes?
If so, that would be your "In vim" solution.

Let us know...

Sean
 
Old 05-06-2017, 02:08 PM   #11
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,143

Original Poster
Rep: Reputation: 43
Quote:
Originally Posted by scasey View Post
As I read it, your file is ending in TWO line feeds (\n)...indicated above with my manually entered \n
I only use vim when I have to...I prefer nano...but positioning the cursor on line 21 and keying dd should delete that last, empty, line. Yes?
If so, that would be your "In vim" solution.

Let us know...
I am a bit confused with your post. The file does *not* end with two '\n' bytes (which have 0ah value, as we see in many *nix text files). You seem to understand wrong the "strange" line for the 21th byte in the file. This byte does not exist, as showed in the hexdump. If it existed, we would see *another* "0a" word in the center columns of the hexdump! The non printable characters (tab, newlines, and a few others) are represented as dots (a real dot!) in the hexdump. That is why you see a dot between the two "|" that limit the raw dump of 20th byte, in the right section of the hexdump. I did not edit hd output to show it here (there are just a few empty lines after it, but this is a LinuxQuestions issue). Is it clearer for you, now?
 
Old 05-07-2017, 12:17 PM   #12
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,432

Rep: Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055
Quote:
Originally Posted by dedec0 View Post
I am a bit confused with your post. The file does *not* end with two '\n' bytes (which have 0ah value, as we see in many *nix text files). You seem to understand wrong the "strange" line for the 21th byte in the file. This byte does not exist, as showed in the hexdump. If it existed, we would see *another* "0a" word in the center columns of the hexdump! The non printable characters (tab, newlines, and a few others) are represented as dots (a real dot!) in the hexdump. That is why you see a dot between the two "|" that limit the raw dump of 20th byte, in the right section of the hexdump. I did not edit hd output to show it here (there are just a few empty lines after it, but this is a LinuxQuestions issue). Is it clearer for you, now?
Yes. Sorry, and oops! I didn't (but should have) realize that was a hexdump.
Yes, there is only one \n. Is that at the end of the last line?
In my experience, not terminating the last line of a file with a line feed will (often has) created problems. That said, I don't have a vim-only solution for you, sorry.

Sean
 
Old 05-07-2017, 02:15 PM   #13
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,432

Rep: Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055
I do find this LQ thread: http://www.linuxquestions.org/questi...he-end-236635/
 
Old 05-07-2017, 03:20 PM   #14
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,143

Original Poster
Rep: Reputation: 43
Quote:
Originally Posted by scasey View Post
Indeed. That thread repeats a few things discussed here, but goes a bit further, solving the question in this thread's title:

We need to set both the binary and noeol options. Calling "vim -b" is a way I knew and tried; another way is to make ":set bin". But there is the option endofline! (short is just eol) So, doing ":set noeol" will turn it off. And ":h eol" is very informative. (:

As for other options, we may add them to our vimrc.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Running a Crafted bash script yelds 'Unexpected end of file' at the end of the file MCLover1337 Linux - General 5 10-15-2011 08:29 AM
Vim - string replacement from current position to end of file armandino Linux - General 1 01-24-2011 03:12 PM
VIM: ^M at end of line baikonur Programming 3 09-12-2006 04:16 AM
vim remove LF at the end michal017 Linux - Software 4 11-19-2004 12:26 AM
quick vim question mbegovic Linux - Software 2 05-14-2004 12:23 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 01:20 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration