InfinityQuest - Programming Code Tutorials and Examples with Python, C++, Java, PHP, C#, JavaScript, Swift and more

Menu
  • Home
  • Sitemap

Python Programming Language Best Tutorials and Code Examples

Learn Python Right Now!
Home
Bash
Converting a Data File to CSV in bash
Bash

Converting a Data File to CSV in bash

InfinityCoder February 21, 2017

You have a data file that you need to convert to a Comma Separated Values (CSV) file.

Use awk to convert the data into CSV format:

1
2
3
4
5
6
$ awk 'BEGIN { FS="\t"; OFS="\",\"" } { gsub(/"/, "\"\""); $1 = $1; printf "\"%s\"\
n", $0}' tab_delimited
"Line 1","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"
"Line 2","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"
"Line 3","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"
"Line 4","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"

You can do the same thing in Perl also:

1
2
3
4
5
6
$ perl -naF'\t' -e 'chomp @F; s/"/""/g for @F; print q(").join(q(","), @F).qq("\n);'
tab_delimited
"Line 1","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"
"Line 2","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"
"Line 3","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"
"Line 4","Field 2","Field 3","Field 4","Field 5 with ""internal"" double-quotes"

First of all, it’s tricky to define exactly what CSV really means. There is no formal specification, and various vendors have implemented various versions.

Our version here is very simple, and should hopefully work just about anywhere.

We place double quotes around all fields (some implementations only quote strings, or strings with internal commas), and we double internal double quotes.
To do that, we have awk split up the input fields using a tab as the field separator, and set the output field separator (OFS) to “,”.

We then globally replace any double quotes with two double quotes, make an assignment so awk rebuilds the record  and print out the record with leading and trailing double quotes.

We have to escape double quotes in several places, which looks a little cluttered, but otherwise this is very straightforward.

Share
Tweet
Email
Prev Article
Next Article

Related Articles

Selecting from a List of Options in bash
You need to provide the user with a list of …

Selecting from a List of Options in bash

Searching for Text While Ignoring Case in bash
You need to search for a string (e.g., “error”) in …

Searching for Text While Ignoring Case in bash

About The Author

InfinityCoder
InfinityCoder

Leave a Reply

Cancel reply

Recent Tutorials InfinityQuest

  • Adding New Features to bash Using Loadable Built-ins in bash
    Adding New Features to bash Using Loadable …
    June 27, 2017 0
  • Getting to the Bottom of Things in bash
    Getting to the Bottom of Things in …
    June 27, 2017 0

Recent Comments

    Categories

    • Bash
    • PHP
    • Python
    • Uncategorized

    InfinityQuest - Programming Code Tutorials and Examples with Python, C++, Java, PHP, C#, JavaScript, Swift and more

    About Us

    Start learning your desired programming language with InfinityQuest.com.

    On our website you can access any tutorial that you want with video and code examples.

    We are very happy and honored that InfinityQuest.com has been listed as a recommended learning website for students.

    Popular Tags

    binary data python CIDR convert string into datetime python create xml from dict python dictionary into xml python how to create xml with dict in Python how to write binary data in Python IP Address read binary data python tutorial string as date object python string to datetime python

    Archives

    • June 2017
    • April 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    Copyright © 2019 InfinityQuest - Programming Code Tutorials and Examples with Python, C++, Java, PHP, C#, JavaScript, Swift and more
    Programming Tutorials | Sitemap