Turkey Tom: The Time I Screwed up a Cornucopia of Data

You ever shift a single row in a spreadsheet and not discover it until later? Yeah, happens sometimes, no big deal…until it is.

I’ll set the scene – its Thanksgiving week, I’m visiting family in Delaware, armed with two laptops so I can deploy a 6-month long build and data load side-by-side. The data I was uploading was seeding a few custom objects, Orders and Order Lines (which then created Assets) for an ERP integration we were turning on the Wednesday before Thanksgiving.

The project involved European & American teams, so communication is happening nearly around the clock for this go-live. We chose 10AM on the Wednesday before Thanksgiving for go-live since it would be close to end of Europe’s workday & the US team would be out-of-office.  Great plan because you know…


It’s Monday and the ERP consultant from Europe sends me the data to stage in Salesforce – we didn’t want to dump 100,000’s of records into the integration for the same reason I stay ahead of schedule on going up diaper sizes for my toddler, systems have their limits, I don’t need to find them.

So data set, looks great, let me clean up a few things, kick out some bad records, do some VLOOKUPS using the Account’s ERP External ID already in Salesforce to map everything correctly. I work on that in between doing some final documentation before my production push starts. Tuesday, I map my processes in Jitterbit (I <3 Jitterbit by the way, another post sometime) and let my data load while I continue with deployment efforts. No errors reported, sweet, good to go!

Fast forward – it’s Thanksgiving morning, 2:45AM (also known as 8:45AM Central European Time) and my phone keeps alerting. Rollover on my dad’s guest room’s incredibly uncomfortable mattress, check the phone to be greeted by a wave of emails that can be summarized as ‘wtf did you do wrong.’


 

Everything was off. Accounts had the wrong orders, order lines with the wrong orders, assets created for the wrong accounts and the integration was creating all new records instead of updating existing ones. What the hell did I do?

I get out of bed, start digging through my spreadsheets and after 3 hours of misery, as the sun is starting to break the horizon, I find my mistake. I had shifted the external ID on the order records by not including it in my filtered columns…causing a cascading effect that ruined all the data.

Well Shit. Bad enough the data is jacked…even worse that it’s my screw-up. It also exposed my lack of thoroughness during deployment, I did not do any spot checks on the data after uploading, I saw 0 errors and let it go.


Of course, this needs fixed today. Maybe not out of operational necessity, but I really could not have a 6-month project fall flat on its face out of the gate because of something I did.  So what if my family is already irritated with me for working so much this week, I’ve been up since 3AM, I have a migraine, and I have to work from dad’s cramped basement home office…your mistake, your pain, suck it up.

I start over from scratch and so there went the parade, dog show, and the Lions game. 14 hours later, finally manage to catch the back half of the Cowboys tilt and have some dinner.

This painful lesson has helped me slow down to test & verify my work more thoroughly.  Flying through a deployment to meet deadlines, get to your timeoff, etc. obviously caused shortcuts that made me miss my goals even worse.

Slow is smooth, smooth is fast.

 

Leave a Reply

Your email address will not be published. Required fields are marked *