CSV Table Cleaner

Somebody else's export is almost never square. One row has an extra field because an address had a comma in it and nobody quoted it, forty blank lines are riding along at the bottom, and one record got submitted twice. None of that shows up when you skim the file, and all of it shows up three steps later when a total comes out wrong.

rows
CSV Table Cleaner — Find Ragged Rows, Drop Blanks and Duplicates, Profile Every ColumnBuildFigure

What it will read

Comma-separated files, a range copied straight out of Excel or Sheets (which arrives tab-separated), and semicolon or pipe-delimited exports. Left on detect, it counts each candidate separator outside of quotes across the first eight lines and takes the winner. Standard RFC-style quoting is honoured: a field wrapped in double quotes may contain the delimiter, a newline, or a doubled "" standing in for a literal quote. Where it falls down is an unbalanced quote — one stray " swallows everything after it into a single field, so if the row count comes back far lower than the file looks, that is the first thing to check.

Why a row has the wrong number of fields

The expected width comes from the header when you have one, and from the most common row width when you do not. Anything else is ragged, and there are two flavours.

SymptomUsual causeWhat it does to the data
Too many fieldsA value contained the delimiter and was never quoted — addresses and free-text notes are the repeat offendersEvery column after the break is shifted one to the right
Too few fieldsTrailing empty values were omitted along with their separatorsUsually harmless; the missing cells are at the end
Wildly wrong on one rowAn embedded newline inside an unquoted cell split one record across two linesTwo broken rows where there should be one

Padding is offered because sometimes you need the load to go through today, but understand what it does: it fills short rows with empty strings and cuts long ones off at the expected width. On a row that was shifted by an unquoted comma, padding hides the symptom and keeps the wrong values in the wrong columns. Fix it at the source when you can.

Reading the column profile

The type label is a 90% threshold, not a schema. The useful signals are the ones that contradict what you expected. A column you believe is numeric coming back as mixed means something non-numeric is hiding in it — usually a - or N/A standing in for a blank, or a leading-zero code that a spreadsheet stored as text with padding. A distinct count equal to the row count means the column is effectively a key, which is worth knowing before you join on something else. A blank rate above about 40% usually means the field is optional in the source system rather than broken.

Deduplication and column selection

Two modes, and the difference matters. Whole-row matching compares every column in the original table, including ones you did not select, so two records that differ only in a column you dropped both survive. Selected-column matching compares only the columns you kept, which is how you collapse on an order ID or an email address and accept that the other fields may differ. In both cases the first occurrence is kept and later ones are discarded.

Columns are chosen by header name or by 1-based position, and they come out in the order you listed them — which makes the column box a reordering tool as much as a filter. Up to 3,000 rows and 80 columns are read; past that the extra is ignored and you are told so.

Questions people ask

Does any of this get uploaded?

No. Parsing, profiling and export all happen in JavaScript in this page, and nothing is sent anywhere or written to storage. Refreshing loses it, so copy the output before you navigate away. That said, if your organisation restricts where an export may be opened at all, the rule usually covers browser tools regardless of where the processing happens — check the policy rather than the network tab.

I pasted from Excel and everything landed in one column.

Copying a range of cells gives you tab-separated text, so leave the delimiter on detect or set it to tab. Copying from inside a cell in edit mode gives you the cell contents with no tabs at all, which is what a single-column result usually means. If a cell contained a line break, the paste wraps it in quotes; an unbalanced quote there will merge the rest of the file into one field.

Why is my amount column showing as mixed?

Something in it is not a plain number. The usual suspects are a currency symbol or unit attached to the value, a placeholder like a dash or N/A where a blank was meant, a leading-zero code, or a value stored as text with padding spaces around it. The counts in the profile row tell you how many of each kind are present; sorting that column in a spreadsheet usually surfaces the offenders in a few seconds.

Can I get the ragged rows on their own?

Not as an export, but the flagged table gives you the line numbers, which is what you need to find them in the original file. Setting the ragged-row option to drop and exporting gives you the clean subset; running it again with the option on flag gives you the list of what was excluded. Between the two you have both halves.

Related