Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back > Forums > Non-*NIX Forums > Linux User Groups (LUG)
User Name
Linux User Groups (LUG) This forum allows Linux User Groups (LUG) to gain exposure. It also allows people looking for a LUG in their area to find one.


  Search this Thread
Old 08-08-2017, 01:37 PM   #1
Registered: Jan 2008
Posts: 75

Rep: Reputation: 15
Convert fields in one column to single line using common field separator in the column

I have a file

I want following o/p

--- _id:abc config:qa
--- _id:lmn config:dev
--- _id:xyz config:xyz
I tried using it with awk with no luck

awk '{BEGIN OFS = "---"; ORS =" " } {print}'
Old 08-08-2017, 02:04 PM   #2
LQ Guru
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,050
Blog Entries: 14

Rep: Reputation: 1214Reputation: 1214Reputation: 1214Reputation: 1214Reputation: 1214Reputation: 1214Reputation: 1214Reputation: 1214Reputation: 1214
awk '/---/ {printf $0" "}
/_id/ {printf $0" "}
/config/ {printf $0 "\n"}' <filename>
echo ""
You can have have awk look for multiple lines based on separate search patterns and output on each based on that line. Using printf rather than print it will output the combined 3 lines on one line.

So in the above we search for the line that contains literal "---" and print that followed by a space, we then look for the line that contains the literal "_id" and print that followed by a space and finally look for the line that contains literal "config" and print that followed by a newline.
At end we do the echo to put in a blank line.

Notice on the third line you specify the file you're using where I put <filename>.

The above of course assumes that the lines are all as you show in your sample. You'd have to play with regex for other differences or similarities.

Last edited by MensaWater; 08-08-2017 at 04:20 PM.
Old 08-08-2017, 06:24 PM   #3
Registered: Mar 2015
Distribution: Linux Mint
Posts: 531

Rep: Reputation: 239Reputation: 239Reputation: 239
Python3 Program

This is a good use of positive lookup in regular expressions.
This starts with ---, then continues until the next character matches --- and repeats. Interestingly, This does not read line by line, but as a single stream. Because it iterates the results as it reads it, it'll begin printing out results immediately and uses minimal memory.

$ ./ -d'---' -c'\n' myfile # Default action
--- _id:abc config:qa 
--- _id:lmn config:dev 
--- _id:xyz config:xyz
#!/usr/bin/env python3

import re
import sys
import argparse
import codecs

def unescaped_str(arg_str):
    return codecs.decode(str(arg_str), 'unicode_escape')

parser = argparse.ArgumentParser(description='Convert delimiter to singlelines')
parser.add_argument('-d', '--row-delimiter',
parser.add_argument('-c', '--column-delimiter',
args, other_args = parser.parse_known_args()

dash_column_regex = re.compile('{d}.*?(?={d})'.format(d=args.row_delimiter), re.DOTALL)
data = (other_args if other_args else sys.stdin)

def column_to_rows(regex_object, file_object):
    file_object =
    return (, ' ')
            for column in regex_object.finditer(file_object))

if data is sys.stdin:
    results = [column_to_rows(dash_column_regex, data)]
    results = list()
    for _file in data:
        with open(_file) as f:
            results.append(column_to_rows(dash_column_regex, f))

for _file in results:
    for line in _file:

Last edited by Sefyir; 08-09-2017 at 12:32 PM.
Old 08-09-2017, 07:56 AM   #4
LQ Guru
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,779

Rep: Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888
awk 'BEGIN{RS="---\n"; FS="\n"; ORS="\n"; OFS=" "} { print "---",$1,$2,$3 }'


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] grep single column only in line keif Programming 4 12-04-2013 02:45 PM
[SOLVED] AWK fill column from previuos line column akeka Programming 4 01-30-2013 08:16 PM
How to display 2 different column field values as one column value in mysql VijayaRaghavanLakshman Linux - General 2 04-16-2012 10:56 AM
Split single line into multiple lines with 3 column each udiubu Programming 4 10-28-2011 12:20 AM
awk multiple column into single column ilukacevic Programming 49 07-19-2010 08:23 PM > Forums > Non-*NIX Forums > Linux User Groups (LUG)

All times are GMT -5. The time now is 05:50 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration