Python csv ignore newline in quotes
I get the impression that this is a common problem, I have a csv file with newlines within the fields. I am looking for a fix within Python--and within the csv module if possible. Here is an example file that I have created
Not all fields will be wrapped in quotes (although my usage is random in this example, the actual file should match quoting=csv.QUOTE_MINIMAL) The output should resemble
Instead I am getting
Please focus on the amount of rows and columns. Another concern is that in the thirds row, a quote was included when it should not have been. Here is my code so far: import csv
Thank you. I assume that you want to keep the newlines in the strings for some reason after you have loaded the csv files from disk. Also that this is done again in Python. My solution will require Python 3, although the principle could be applied to Python 2. The main trickThis is to replace the For my weird character, I will use the Icelandic thorn: Þ, but you can choose anything that should otherwise not appear in your text variables. Its name, as defined in the standardised Unicode specification is: LATIN SMALL LETTER THORN. You can use it in Python 3 a couple of ways:
That An alternative character that may serve as a good standard is Replacing \nNow we use this weird character to replace
The resulting dataframe looks like this, with newlines replaced:
Writing the results to diskNow we write either of those identical dataframes to disk. I set
What does it look like on disk?
Getting the original data back from diskRead the data back from file:
And we can replace the
And the final DataFrame:
If you want things back into your list of lists, then you can do this:
Which looks like this: |