A growing need to easily export information into internet-based dairy applications has led the country’s livestock services organisation to replace its ageing but heavily used national database.
Livestock Improvement applications resource manager Kerry Hubert says by using in-house staff to cut code and implement the new IBM DB2 database the project came in at two-thirds the cost of external estimates. He says the livestock services arm has “now got a future and a whole new way of doing things” as the internet becomes a more important part of the global dairy trade.
And by controlling the project rollout itself Livestock Improvement, a subsidiary of the Dairy Board, was able to keep the downtime for the national database to five days between Christmas and New Year – the one time of the year enquiries slow.
Livestock's database is available to farmers over the internet. It has the details of more than 16 million dairy animals, records 96% of cow locations and movements in the country, 90% of annual mating records and lists all dairy farmers, their farms and locations.
By last year the Supra database from software vendor Cincom occupied over 120GB of disk space and processed around 20 million online transactions a year and 30 million in batches. At peak times it handled more than 1200 database calls a second on a single processor.
Hubert says it had come to a point where Livestock’s technical team were spending too much time trying to build ways to extract the data and put it into other packages.
“We really needed an open database,” he says. IBM’s DB2 was chosen because of its ability to talk to relational programmes and query tools and support the Supra-related Mantis programming language. Once the decision was made to do it in-house last February, a team of up to 50 to 60 people at a time converted the Supra statements into equivalent raw SQL - which took some time because the raw conversion was at first inefficient – then applied it to 500 batch procedures and 650 Mantis programmes, and finally put it through a lengthy stress test phrase.
Hubert says disk space has not decreased much but Livestock does not consider this a downer: the data component itself is now much smaller and the extra space is taken up with valuable and easy to use indexes.
The new system is proving as fast if not faster than the old and developers are to spend the next months experimenting with mixes of transaction loads, he says.