CSV file cleanup
Local product data merge and validation report
Source files
| File | Encoding | Delimiter | Data rows |
|---|---|---|---|
| products_january.csv | UTF-8 | semicolon (;) | 10 |
| products_february.csv | CP1250 | comma (,) | 10 |
Results
Merged using these columns: sku, name, price, stock, description.
cleaned.csv contains all unique rows. review.csv lists records that need review; those records also remain in the merged file.
Only rows identical across every column after trimming whitespace were removed. Text values were preserved; only leading and trailing whitespace was trimmed.
Spreadsheet export safety
=, +, -, @, a tab, or a carriage return are prefixed with an apostrophe to reduce formula execution when a CSV is opened in a spreadsheet. Text beginning with a minus sign, including a number stored as text, is also prefixed.Encoding and numeric validation
Automatic encoding selection tries UTF-8, then CP1250. This is not universal detection: if you are unsure of a file's encoding, verify it at the source and specify it explicitly.
A price is valid when it contains ASCII digits, optionally followed by a decimal point or comma and one or two digits; the source text is unchanged. Stock must contain only non-negative ASCII digits; leading zeroes are allowed.
When opening a CSV in Excel, import the SKU column as text to preserve leading zeroes.