Validate CSV Field Boundaries in PCB Test Exports

A test export can look readable in a text editor while importing incorrectly into analysis software. Commas inside descriptions, quotes in operator notes, and embedded line breaks can shift fields when a file is split with simple text operations. Validate the actual tabular interpretation before trusting the results.

Python’s CSV documentation describes dialect settings for delimiters, quoting, and line handling. It also notes that automatic format detection is heuristic. A controlled production interface should define its format explicitly instead of depending only on successful guessing.

Specify the interchange contract

Record the delimiter, quote character, escaping convention, character encoding, header names, and treatment of blank fields. State whether notes may contain line breaks. The producer and consumer must agree on these details, particularly when files pass through spreadsheet applications before analysis.

Use a parser that understands the defined format. Counting commas or splitting each physical line does not correctly handle every quoted record. Preserve the source file so that a suspicious imported row can be compared with its original representation.

Build discriminating examples

Include distinctive neighboring values so that a column shift is easy to detect. For example, use a known serial identifier, a named test step, and a numeric reading whose expected field is unambiguous. Do not use a test file in which every cell contains the same placeholder.

Check the round trip

1. Export the representative records through the production path.
2. Import them with the intended downstream parser.
3. Compare every field with the original logical record.
4. Check logical record counts and all required column names.
5. Repeat after changing exporter or parser versions.

A physical line count may differ from a logical row count when quoted fields contain line breaks. Use the parsed record count for reconciliation. Reject or quarantine malformed records with a useful location and reason; silently dropping a bad line can remove a failed board from the dataset.

Keep format validation separate from numeric and unit validation. A correctly parsed field can still contain the wrong value, but reliable field boundaries are necessary before any later check has meaning. The release evidence should demonstrate that realistic notes and descriptions survive the same path used by actual production results.