The build versus buy dilemma

Daniel Mitchell, Chief Executive Officer •

Blog 8

The buy versus build dilemma in software is often said to be as old as commercial software itself. But the basics of the conflict are much older: build vs. buy is simply the current manifestation of the same tensions inherent in the broad historical trend towards the division of labour.

The division of labour is the separation of tasks in society or in an economy such that they are completed by specialist individuals or organisations, and has been credited as being a primary cause of economic growth across the centuries. Businesses take advantage of it all the time. If you run a supermarket delivering food to customers you don’t build your own vans, you buy them or hire them; if you run a restaurant you don’t make your own ovens, tables, cutlery, or dishes; if you are a builder you don’t make your own bricks, steel beams, or windows.

The same thing is widely accepted in some areas of software. Very few companies would consider creating their own operating system, making their own spreadsheet or word processing tools, or building a video conferencing solution. Broadly, with more mature and universally adopted technology, the division of labour is so well established it isn’t questioned. In a sense, the software as a service business model is predicated on the idea that the division of labour applies to software and its success suggests the market agrees.

Another way of thinking about the same trend is to say that one should build where the software embodies your company’s business value or competitive advantage and buy where it doesn’t. Sounds like an easy decision, right? Well, the fact this question is still around suggests that it’s not quite so black and white, and that there may be a considerable grey middle ground.

At Hivemind we’ve lived both sides of the debate. Our software originated as a bespoke tool created within the data science team at a hedge fund. For us, necessity was the mother of invention – there was no existing software on the market with the functionality we required to solve the challenge which faced us, so we built it. Now that we’ve spun out from the hedge fund and we sell that software under a SaaS model we’re now firm believers in buying when it comes to human-in-the-loop data preparation software!

Clearly there’s no universal right answer, but there are key themes to consider when making the decision: time, cost, functionality and risk.

If you can think your way through these sorts of questions with input from both the engineering side and the business side then you should be well on your way to making the right call.

One final thought worth considering is that build versus buy is a bit of a false dichotomy. In reality, the decision is often kicked down the road in favour of a third route: cobbling something together with pre-existing, non-specialised tools. This is perhaps particularly true when thinking about data preparation pipelines, which is our business here at Hivemind. It may seem tempting to avoid the internal politics of getting time from an engineering team or budget sign-off, and it’s a reasonable route to go down if you’re looking to build a quick prototype to prove business value within an internal proof-of-concept process, but don’t fool yourself that you’ve built a robust solution or something appropriate even for the medium term. To learn more about the dangers of this kind of approach, including a high profile example of high-stakes failure, see our blog post 'the dangers of using the wrong tool for the job.'


And as always, if you would like to discuss how Hivemind can help with your data preparation pipeline, get in touch!