Running an AI Project (Part 1 of 4)

Lets Herd Cats

This example and outline will focus specifically where AI projects differ from other IT projects and business projects. The audience intended for this is anyone who would be engaged on such a project either as a stakeholder, resource, driver, or consumer. PMs will find it lacking, technical people will find it irritatingly high level, but it may give them a view into some of the decisions occurring around them that they may have ignored int the past.  Hopefully this will help explain why corners are sometimes cut, and a general description of the steps to take while putting together a plan. It is not intended as a how to guide and is not comprehensive. Hopefully it will give you some idea of what you are in for, and set your expectations for a project of this nature appropriately.

All projects in general, go through these phases from the perspective of an Owner:

  1. Pre-initiation ( I have an idea.)

  2. IT Initiation (I need some techies.)

  3. Definition and Planning (Let us be clear what we are doing here.)

  4. Launch and Execution of the Project (Lets get this thing going, I have the money.)

  5. Performance and Controls on the Project (Are we making progress?)

  6. System Launch (It’s Alive?)

    1. Launch business, make money.

  7. Warranty Period (It’s not supposed to do that, fix it)

  8. Project Close Out (I guess this is what I bought.)

Pre-Initiation

Before a project is even kicked off, a lot of things have already been done.

1. The owner/financier came up with a creative, or not so creative way to make some money or save some money

2. Hopefully somebody did the baseline analysis on how that will be acomplished, and what the limitations of the proposal are.  

But lets assume the business analysis was done and it looks something like this:

i.e. “I’m going to classify peoples pictures for them!”

I guesstimate I can charge 1$ per person in the US for this service = 350 million dollar potential!

I guesstimate that only 3% of people will opt into this service = 10.5 Million dollar potential.

I guesstimate that we can reach some of the US population with advertising = 1.5M potential

All of our costs come out to 800k says the business analyst… So lets see how we get at that 700k potential we still see out there!

The limitation is that this opporunity will only be for six months and it will decay at 100k a month after that.  So no pressure.

Now lets start a project to get our market. For the purposes of this article we’ll assume all of the marketing and other business resources are accounted for and ready to start at our whim. Hiring is a completely different issue we won’t address here.

Go-No go decision. Do I want to spend the money on a more technical discovery phase to see what it will take to build this thing?  If the Business Analyst says there is profit to be made, then move forward.  If they are super confident, they may try to skip the next phase where you get a more general assessment by technical architects and cost/timeline estimates from those experts.  A common situation in projects is that the budget and timeline is set before anyone with an enginering or background in these types of projects is engaged.  

With projects that have a budget and timeline that are set before getting estimates it goes a few different ways. 

IT people are engaged, they respond "It'll be tight but we can pull it off" - because if they don't say that, they don't get the work/contract.

  • The budget they were given is too big... and they just spend it/put it in the bank
  • The budget they were given isn't big enough... and they work their butts off, deliver a sub par product if anything useable at all.

Deliverables: Charter for the project, Business goals loosely defined, stakeholders defined, and their education level in AI is at least assessed if not addressed. General estimate at the highest level assessed by someone who has worked on a similar AI project purely for charter purposes.

IT Initiation

Develop the Project Charter in more detail, educate everyone involved in what is possible, and what is a stretch. The Charter is important for any project where you are going into a completely new realm. It sets expectations, helps prevent scope creep, and potentially saves a lot of headache as the project progresses, hits blockers, runs into issues, or exceeds expectations in ways that were not intended.

The charter is a gauge that is set on the project before it even starts getting resources that are monetary or personel. It’s a measuring stick for the project.  Most of the analysis and decision making in this phase is high level, has very broad numbers and generalizations, and tends to be overly optimistic. It is a necessary and sometimes fortune saving phase of the project even if it is flawed. If you can’t get through these questions to get the justification for a budget for phase 3 you have no business moving on to phase 3 where you will do a much deeper dive into the analysis and planning and commit even more money. You can dodge a major bullet if you do the initiation phase properly and you have honest feedback from all parties.  How you enusre you have honest feedback is a whol other article that revolves around motivations and psychology and is beyond the scope of this.  If their compensation is in any way tied to the project moving forward, the answer will always be "Great idea!  Lets do it!"

Questions you need an answer to:

What are the business objectives? Build an AI system that …..

  1. Forecasts (Finance, resource management, predicts)
    1. What will the value of variable X be at Y Time type problem.

      1. Useful for making any type of decisions.

      2. Customer churn analysis/sales projections/Pricing analysis/

  2. Simplifies – Combines some processes for automation and decision making

  3. Provides insight ( Representation of data and information in new ways that make more sense, Commonly refered to as “Business Intelligence”)

  4. Solves non-linear problems (Balancing problems, Gaming theory, decision matrixes, Open ended problem solving)

  5. Classifies (recommendation systems, Auto encoders)

What is your technology stack?

  1. Are we limited by pre-existing systems that are in place that we’ll have to integrate with?

  2. Which stack fits best with our objective? R, python, Sas, C++

    1. Are there technical requirements for performance availability or data integrity?

    2. Should there be?

  3. What are your resources?

    1. Do we have the people with the skills necessary to achieve this?

    2. Can we get them?

    3. How much do they cost?

    4. Are we gambling on unknowns and are we good with that?

What is your budget?

  1. How much money do we have to work with/are willing to spend to achieve this?

  2. Is the Return on Investment of the business objective higher than the cost to implement?

  3. If not, is it a strategic imitative that will have a halo effect on other initiatives?

    1. Can the Halo effect be quantified?

What is our time line for launch?

  1. How does that impact other initiatives, resources, costs, and the ROI?

  2. If we had it right now we would be billionaires!

  3. If we wait six months it isn’t worth anything.

  4. Has any of this or something similar been done before that we can leverage?

Go – No Go decision…. You still want to do this project after all of that and you have your estimates?

Deliverables for this phase: You will have a commitment on budget and time line to target, Scientific Wild Ass Guess (SWAG) estimates around the major milestones that are hopefully within 20% of reality, metrics for those milestones defined so you can measure progress through the definition and planning phases.  The charter will be something that all stakeholders are in agreement on.

At this stage, the organization (Could be one guy) has gotten through both the business market analysis and passed that go-nogo decision point. YES It’s worth dropping the money for an IT Initiation. This decision should be based on the SWAG estimates provided in phase 2 in comparison to the Business case return that was provided in phase 1. Together, those things can be used to calculate a Return on Investment for the project. If this number is negative, I hope you aren’t doing it for the money. I’ll also add that sometimes you aren’t! i.e. Elon Musk and Space X, Most medical research, Academics projects, community development projects, Art.. etc.

In the next article, I'll get into the Definitiaon and Planning Phase as well as the Execution of the project in phases 3 and 4.

Feedback as always is appreciated.