Evaluating Software Technology For Your New Project

Ask Yourself: Should I Build or Buy?

Every new project needs to select a library of off-the-shelf software and services to deliver the required functionality in reasonable time. It never makes sense to build it all from scratch.

However, every 3rd party technologies you adopt introduces risk. Services might get shut down. Software might be buggy or unreliable. The technology might work great for 1,000 users but grind to a halt for 1,000,000. The technology may simply fall behind the rest of the industry and lock you into a hard to maintain solution.

How do you manage these risks and make sure you make good choices for your project?

Technical Assessment

The first step is to determine if the technology makes sense for the project based on its technical merits. It’s easy to want to adopt a cool, new technology. It’s better to adopt the technology because there are strong reasons why it is the right choice for the specific project you have in mind.

1. Does the technology really make sense for your current project?
2. Does it save development time?
3. Does it provide essential capabilities that would be difficult or impossible to develop on your own (within budget)?
4. Does the technology reduce the amount of code you have to write?
5. Does it improve the code quality, readability and organization of your project?
6. Are there viable alternatives?
7. Is the technology fully open source?
8. What language is it written in?
9. Hosted: If it is a hosted service, what would it take to switch to a self-hosted or alternative service once you’ve written substantial code (1-2 years out)?
10. Server-side: How flexible are the hosting options?
11. Can you run it on a serverless Lambda/Functions service?
12. Is there existing Docker support?
13. How extensible is it?
14. Can you add code to override most anything with custom behavior?
15. How easy is it to extend and customize?
16. What language(s) are supported for custom behavior?
17. Is there good support for instrumentation/logging/analytics?
18. Does it have good documentation?
19. How relevant is the technology within the global technology ecosystem?
20. Is it keeping pace with the rest of the industry?
21. Is the API well designed?

Scaling Assessment

Make sure your technology choices will scale with your projects’ needs. Start by asking yourself what your project might look like in 5 years if it is wildly successful. Then take a look at each technology you are using and make sure it is capable of meeting those future needs.

1. What fees are there? How do they scale from 10 users through 10,000,000 users and beyond?
2. Do you deeply understand how the technology scales under various loads?
3. Are there any thresholds beyond which the technology stops scaling well?
4. Does it scale horizontally?
5. Does it scale vertically?
6. Does it scale with application complexity? (e.g. 100,000 lines of code or 1,000,000 lines of code)
7. Does it scale with data quantity? (e.g. billions of records or more)
8. Does it scale with data size? (e.g. terrabyte DB or exabyte file-storage)
9. What is the largest known user of the technology? How large is their use case? How will is it working for them?

Risk Assessment

The hardest and most overlooked aspect of selecting 3rd party technology is assessing risks. How might the technology fail or let you down? How might the ongoing development effort for the technology fail to meet your specific needs? Will there even be an ongoing development effort?

Ultimately, risk is about tradeoffs. For example, if a technology is easy to swap out, risk goes way down. After all, if it’s easy to swap out, the cost is small if it doesn’t work out. Similarly, if a service provides a fully open-source solution you can self-host, risk again goes down. If the service ends up being too expensive, unreliable or underperforming, you can easily swap to the self-hosted option.

On the other hand, if you don’t have an easy alternative if the technology fails, you should do your due diligence to ensure you make the technology choice most likely to succeed in the long run. 

Use these questions to assess the risk of adopting 3rd party software and services:

1. How easy is it to swap out the technology for another?
2. How is it to export your data and import it into an alternative solution?
3. How confident are you that the technology will be well supported in 5 years?
4. How long has the technology been around?
5. How well maintained is the technology?
6. Is there evidence of regular releases and security patches?
7. How easy is it to train or hire developers to work on the technology?
8. What does Google Trends show for the technology? Is it up-trending or down-trending? How does it compare to competing technologies? (Note: Google Trends always needs at least one comparison to be meaningful.)
9. Who owns the technology? How likely are they to stick around in the long run? What might happen if they abandon the project or go out of business? Is there enough community to ensure ongoing development?
10. How much community support is there? Are there a lot of Q&A on StackOverflow? How does it compare to competing technologies?
11. How big is the ecosystem? How many related tools/packages/projects are there that integrate with the technology?
12. Who else is using this technology? Are they using it in a small or big way?

Selecting Your Technology Stack Isn’t Easy

Deciding which off-the-shelf technology to use, or if you want to build your own, is never an easy question. Ideally you’d like to look back in 5 years and feel like you made the best possible choice, but predicting the future is hard. First, start with the technical merits to make sure it really makes sense for the project. Second, make sure the technology will scale with your anticipated, best-case-scenario needs. Third, make sure you do your due diligence and select technologies that will be well supported and remain relevant over the lifetime of your project.

Finally, analyzing a single technology in a vacuum is largely meaningless. To have a meaningful result, you must compare it against 2-3 other top competing technologies to determine which one is truly best for your project.