Writings

How to Budget for Data Science and Data Engineering Work for 2021

By:
Ross Katz

Budgeting is easy for fixed costs and relatively simple for variable costs, as long as you know the drivers of the variable expense, such as number of products sold or parcels shipped.

For strategic investments in capabilities like Data Science and Data Engineering, budgeting can be more difficult, especially if you do not currently have full-time staff devoted to the function. If you have ideas about the type of outcomes you want to drive using data and a high-level understanding of the complexity involved in implementing those ideas. With those two things, you have everything you need to get a rough estimate of your data science and data engineering budget for 2021. Just use the framework below.

Prioritize Using Impact vs. Complexity  

We propose two metrics, ranked 1-5 with 1=low and 5=high, to frame your conversation about budget for data projects:

  1. Impact: On a scale of 1-5, how much incremental revenue or cost could we capture through data-driven improvements to our business (e.g. operational efficiency, marketing spend, customer targeting, portfolio optimization, etc.)?
  2. Complexity: On a scale of 1-5, how complex is the value chain between starting a data project and delivering value to the business? 

At this level, there is no need to consider the actual dollar amount of revenue or cost driven. All you want to consider is the relative impact and complexity of different data-driven initiatives your company could consider.

Create a spreadsheet that mirrors the below (Feel free to use this template):

Figure 1: Impact vs. Complexity Matrix


For questions you can use to assess impact and complexity, please use the appendix to this article.

Once you have your scores assigned for each project, you can plot the impact vs. complexity of each data project. Prioritize the projects in the bottom right corner, those with high impact and low complexity, followed by those in the top right, those with high impact and high complexity. At that point, you can start to dive deeper into the details of budgeting for those projects.

Figure 2: Data Project Impact vs. Complexity Matrix


After prioritization, take each of the projects you have decided to prioritize, and make a quick attempt to estimate the potential value of the project to your company. 

Quantify Project Value

To understand project value, you will need to do the important work of determining how your company will use the output of the project:

  • If you knew how profitable your products were, how would you use that information? (i.e. Cut unprofitable products and optimize marginal products) 
  • If you knew your lifetime value and customer acquisition cost, what would you do next?  (i.e. Increase marketing spend to accelerate growth to a rate more reflective of the value of each customer acquired)
  • If you knew which advertising channels were more and less successful, what would you do differently? (i.e. Reallocate funds or concentrate efforts on optimizing low-performing channels).

Explore Project Cost

Once you understand the project’s value, you can take the budgeting process in two directions:

  1. Experimentation-based: Agree to invest a small portion of the project’s value in validating that the project will produce the expected results. Engage an internal or external partner to build a proof-of-concept using that portion. 
  2. Assessment-based: Engage a partner, internal or external, to assess the cost of completing the project. Teams capable of delivering the value should be able to provide a range estimate for the time and resources needed to complete the project.

A good partner will also be able to help you understand the best way to structure a project to capture maximum value. Oftentimes, multiple value propositions from separate projects can be captured with a minimal increase in the scope of a single project. Automated reporting on ad spend and product profitability, for example, both involve a data pipeline, data warehouse, and business intelligence suite.

With projects prioritized and the costs and benefits of data projects estimated, you should have the confidence you need to secure a budget for the highest priority data projects you need to grow and optimize your business in 2021.

Find a Data Partner You Trust

The CorrDyn team can guide you through either the experimentation-based approach or the assessment-based approach. 

If your company wants to move quickly with the experimentation-based approach, we can leverage our experience to guide your project toward the highest ROI proof-of-concept that can be achieved in the shortest period of time. We excel at delivering proofs of concept early in each engagement to build trust in the ROI of the solution being developed.

If your company wants to use the assessment approach to understand the full scope of work to be completed, CorrDyn begins the engagement with an assessment to develop the project plan and help you estimate the cost of the solution proposed. If you are considering embarking on data projects in 2021, we can help you estimate the return and investment required for each project. 

We want our data projects to deliver value, and we are prepared to be your long-term technology partner. We will not begin an engagement without confidence that the project will deliver strong ROI for your business and can show you the path to success. 

Appendix: Surveys for Assessing Impact and Complexity 

Impact Assessment Survey

  1. On a monthly basis, how much additional revenue do we expect to drive from this change? 
  2. On a monthly basis, how much time would this project save our team members? How much does that time cost our business each month?
  3. Would this project reduce our fees to other service providers or channel partners?
  4. How many additional sales could we make from reallocation of resources?
  5. How much additional profit could we drive from portfolio optimization of products or channels?

Complexity Assessment Survey

  1. How many data sources need to be integrated to drive this data project?
  2. How well do our SaaS platforms and internal systems enable data to be retrieved?
  3. How much data cleaning will need to be done to prepare the data for this project?
  4. How complex is the data product to be developed? Does it require machine learning, or is it simply an automated reporting system?
  5. Does the output of this project need to be surfaced to our customers? 
  6. How big is the maintenance burden? Will the project require regular monitoring or updates to ensure it is accomplishing its goals?