Developer leaps data formats in a single bound

Migrating data to a new format should be simple, says Jon Udell

Recently I helped a friend categorise her Schedule C expenses. (Schedule C is where proft and loss from businesses is reported in US tax forms). All of her business income is in QuickBooks, but her expenses aren’t. I would have to reconstruct those from bank and credit card records. Although this friend has online accounts at both institutions, my Spidey sense was tingling: I knew there was going to be trouble.

As it turned out, the bank was a knock-over. It doesn’t export data in QuickBooks’ IIF (information interchange format) but does offer CSV (comma-separated variable). I had long ago written a CSV-to-IIF translator. So, with a minimum of fuss, I was able to suck the 2005 expenses into QuickBooks, where my friend could begin tagging them.

The credit card company’s defences, though, were more formidable. Its site had a CSV dumper, too, but when I asked for 2005 transactions, all I got back were fourth-quarter records. The 12 statements from 2005 are available as PDFs, but that wasn’t what I had in mind. The exchange went like this — Agent: “I’m sorry, sir, there’s nothing else we can do”. Me: “Or rather, nothing else you will do” Agent: “Would you like me to fax you those statements?” Me: Lots of grumbles.

Here’s a little secret I didn’t tell them. I have a superhero power that enables me to do battle with the evil that is data lock-in. I can’t leap tall buildings or crush lumps of coal into diamonds, but when I look at the barriers that divide one data format from another, they hardly seem to exist. For me, data transformation is almost an autonomic reflex, like breathing.

But PDF? Please, oh please, don’t make me dig the data out of those PDF files. I cajoled. I begged. I threatened. However, convincing organisations to make exceptions is one superhero power I don’t possess. So I found a PDF-to-Excel translator and went to work.

The results weren’t pretty. Entropy runs only one way, after all. It takes work to convert a less orderly system into a more orderly one. Naturally, the output had to be massaged.

As I ran through a series of regular expression search-and-replace operations in my programmer’s text editor, I was dimly aware of the fact that I was exercising a freak talent. What do normal people do? Transcribe the numbers by hand, I guess. Or, perhaps equally likely in the case of Schedule C, just invent them.

It doesn’t have to be this way. PayPal, for example, will happily disgorge all my transactions as far back as 2000. It even offers IIF, on the assumption that not everyone can easily convert to it from CSV.

I know what you’re thinking: that’s a security risk. And, you’re right, it is. But am I any safer with all my data sitting in PDFs?

I’m tempted to joke that if we regard PDF as a mode of encryption, my statement history is actually safer than a raw transaction history would be.

But, seriously, I should be able to encrypt my historical data in any format, so long as it’s not necessary to the operation of the service, and I’m willing to be responsible for the key.

If I don’t make that choice, though, let’s get real. If I want to turn my data into HTML, IIF or PDF for that matter, I will. If you can do those transformations for me, great.

But, first things first. Just give me my data when I ask for it. Not ink on paper, not a bitmapped image of ink on paper and not even a vector representation of ink on paper, just the data.

Join the newsletter!

Error: Please check your email address.

Tags datatechnologyformats

More about ExcelPayPal

Show Comments