Running an AI Project (Part 1 of 4)

Lets Herd Cats

The following scenario illustrates some of the ways in which project planning for artificial intelligence (AI) projects differs from traditional IT and business projects.  My goal in writing this is to give you an idea of what to expect with regards to the project planning process, and to set appropriate expectations regarding projects of this nature.  It is not a comprehensive example, and is not intended for use as a how-to guide.  

From the perspective of the project owner, projects generally go through the following phases:

  1. Pre-initiation (I have an idea!)
  2. IT Initiation (I need some techies…)
  3. Definition and Planning (Let’s be clear on what we’re doing here.)
  4. Launch and Execution of the Project (Let’s get this thing going, I have the money!)
  5. Performance and Controls on the Project (Are we making progress?)
  6. System Launch (It’s…Alive?)
    1. Launch business, make money
  7. Warranty Period (It’s not supposed to do that, fix it.)
  8. Project Close Out (I guess this is what I bought?)

Pre-Initiation

Before a project is even kicked off, a lot of work should have already been done:

1. The owner/financier came up with a creative (or not so creative) way to make or save money.

2. Someone did a baseline analysis on how that will be accomplished, and what the limitations of the proposal are (hopefully). 

Business Analysis

Let’s assume the business analysis was done and it looks something like this:

  1. The idea: “I’m going to automatically classify people’s pictures for them!”
    • Assumptions
      1. I guesstimate I can charge $1 per person in the US for this service = $350 million potential!
      2. I guesstimate that only 3% of people will opt into this service = $350M * .03 = $10.5 million potential
      3. I guesstimate that we can only reach 10% US population with advertising = 10.5M * .1 = $1.5 million potential
  2. All of our costs come out to $800k says the business analyst…(1.5M – 800k = 700k) let’s figure out how we get at that $700k potential profit we still see out there!
    • The limitation is that this opportunity will only be for six months and it will decay at 100k a month after that…so no pressure
  3. Let’s start a project to find our market (For the purposes of this article we’ll assume all of the marketing and other business resources are accounted for and ready to start at our whim. Hiring is a completely different issue we won’t address here.)
  4. Go/No-go decision.
    • Do I want to spend the money on a more technical discovery phase to see what it will take to build this thing?  If the Business Analyst says there is profit to be made, move forward.  If they are super confident, they may try to skip the next phase where you get a more general assessment by technical architects and cost/timeline estimates from those experts.  A common situation in projects is that the budget and timeline is set before anyone with a background in these types of projects is engaged
  5. With projects that have a budget and timeline that are set before getting estimates, it can go a few different ways:
    • IT people are engaged and they respond "It'll be tight but we can pull it off…" because if they don't, they don't get the work/contract
    • The budget they are given is too big... and they just spend it/bank it
    • The budget they are given isn't big enough... and they work their butts off, only to deliver a subpar product, if anything at all
  6. Deliverables:
    • Charter for the project
    • Business goals, loosely defined
    • Stakeholders defined, and their education level in AI is at least assessed if not addressed
    • General estimate at the highest level, assessed by someone who has worked on a similar AI project purely for charter purposes

IT Initiation

Next, develop the Project Charter in more detail to educate everyone involved on what is possible, and what is a stretch. The Charter is important for any project where you are going into a completely new realm: it sets expectations, helps prevent scope creep, and potentially saves a lot of headache as the project progresses and hits blockers, runs into issues, or exceeds expectations in ways that were not intended.

The Charter is a gauge that is set on the project before it even starts receiving resources (monetary or personnel). It’s a measuring stick for the project.  Most of the analysis and decision making in this phase is high level, has very broad numbers and generalizations, and tends to be overly optimistic. It is a necessary and sometimes fortune-saving phase of the project even if it is flawed.  If you can’t get through this step and justify a budget, you shouldn’t move on to phase 3, where you will do a much deeper dive into the analysis and planning and commit even more money. You can avoid complications if you do the initiation phase properly and receive honest feedback from all parties (How you ensure you have honest feedback is a separate issue beyond the scope of this discussion.  If compensation is in any way tied to the project moving forward, the answer will almost always be "Great idea!  Let’s do it!")

Questions You Need Answers To

What are the business objectives?

Answer: Build an AI system that:

  1. Forecasts (Finance, resource management, predicts
    1. What will the value of variable X be at Y time?
    2. Useful for making any type of decision
    3. Customer churn analysis/sales projections/pricing analysis/
  2. Simplifies
    1. Combines some processes for automation and decision making
  3. Provides insight
    1. Representation of data and information in new ways that make more sense, commonly referred to as “Business Intelligence”
  4. Solves non-linear problems
    1. Balancing problems, gaming theory, decision matrixes, open ended problem solving
  5. Classifies
    1. Recommendation systems, auto encoders

What is your technology stack?

  1. Are we limited by pre-existing systems that are in place that we’ll have to integrate with?
  2. Which stack fits best with our objective? R, python, Sas, C++
  3. Are there technical requirements for performance availability or data integrity?
    1. Should there be?
  4. What are your technical skill resources?
  5. Do we have the people with the skills necessary to achieve this?
    1. Can we get them?
    2. How much do they cost?
  6. Are we gambling on unfamiliar technologies and are we good with that?

What is your budget?

  1. How much money do we have to work with/are we willing to spend to achieve this?
  2. Is the Return on Investment of the business objective higher than the cost to implement?
    1. If not, is it a strategic imitative that will have a halo effect on other initiatives?
  3. Can the Halo effect be quantified?

What is our timeline for launch?

  1. How does that impact other initiatives, resources, costs, and the ROI?
  2. If we had it right now we would make 800k
  3. If we wait six months it isn’t worth anything.
  4. Has any of this or something similar been done before that we can leverage?

Go/no-go decision…do you still want to do this project after all of that, even with your estimates?

Deliverables for this Phase

You will have:

  1. A commitment on budget
  2. A timeline to target
  3. Scientific Wild Ass Guess (SWAG) estimates around the major milestones that are hopefully within 20% of reality
  4. Metrics for those milestones defined so you can measure progress through the definition and planning phases.

The charter will be something that all stakeholders are in agreement on.

At this stage, the “organization” (could be one guy) has gotten through both the business market analysis and past the go/no-go decision point.

If the decision is a YES it’s worth spending the money for an IT Initiation. This decision should be based on the SWAG estimates provided in phase 2, in comparison to the business case return that was provided in phase 1. Together, they can be used to calculate a return on investment (ROI) for the project. If the ROI is negative, I hope you aren’t doing it for the money – but sometimes you aren’t! E.g., Elon Musk and Space X, Most medical research, Academics projects, community development projects, Art…etc.

Next, I'll get into the Definition and Planning Phase, as well as the Execution of the project in phases 3 and 4.