Skip to content

Gracefully fail on errors while parsing data

Right now, an error while parsing the data files will result in an error which prevents all further processing. Case in point: One row had an invalid timestamp which caused the pipline to fail for ~12 hours with no data imported.

Instead of blocking further processing, move erroneous file to a seperate folder and continue w/ the rest of the files.