Machine Learning Project Checklist

As a Data Scientist or Machine Learning Engineer working in a real world Machine Learning Project, you should go through this checklist to make sure that everything goes well throughout the whole pipeline.

machine learning project checklist

There are eight main steps:

1. Frame the problem
2. Get the data
3. Explore the data
4. Prepare the data
5. Model the data
6. Fine-tune the models
7. Present the solution
8. Launch the ML system

The very first step is where the (business) objective is defined and assumptions about the solution are made. The following questions need to be answered:

• What are the current solutions/workarounds (if any)?
• How the problem should be framed (supervised/unsupervised, online/offline, etc.)?
• How should performance be measured (metrics) ?
• What would be the minimum performance needed to reach the business objective?
• How the problem can be solved manually?

2. Get the data

In this step, the required data is gathered and splitted into training set and test set. The following steps must be considered:

1. List the type of data required and quantity
2. Find appropriate source to get data
3. Check how much space it will take
4. Check legal obligations, and get authorization if necessary
5. Get the data
6. Convert the data into required format
7. Ensure sensitive information is deleted or protected
8. Split data into training set and test set

3. Explore the data

The gathered data is explored to gain insights: study and understand more deeply about data. The following sub-steps should be considered in this step:

1. Make a copy of data to explore
2. Visualize the data
3. Study about attributes (name, data type, missing values)
4. Study the correlations between attributes
5. Document everything learned about data

4. Prepare the data

After knowing about the data, data need to be cleaned and pre-processed before training the models.

- Data Cleaning:

• Fix or remove outliers (binning & clipping)
• Fill in missing values or drop rows/columns (scrubbing)

- Feature engineering:

• Select appropriate features
• Discretize continuous features
• Feature Crossing: Create new feature by combining two or more features
• Feature scaling: standardize or normalize features

5. Model the data

Now, its time to train various models (algorithms) and list down the most promising models. The following sub-steps need to be considered:

1. Train many quick and dirty models from different categories (e.g., linear, naive Bayes, SVM, Random Forests, neural net, etc.) using standard parameters
2. Measure and compare their performance (use N-fold cross-validation)
3. Analyze the most significant variables for each algorithm
4. Analyze the types of errors the models make
5. Short-list the top three to five most promising models

6. Fine-tune the models

From the short-listed few models, we need to find the best performing model. The major tasks to be considered are:

- Fine tune the hyper-parameters using Cross Validation

• Grid Search / Random Search
• Bayesian Optimization

- Try Ensemble Methods (combining best models)

- Keep the best model after measuring performance on test set.

7. Present the solution

This is much less technical step. The proper documentation until now should be prepared and presented in the best way possible highlighting the big picture. Remember the following steps:

1. Document everything what is done until now
2. Create a nice presentation
3. Explain why this solution achieves the business objective
4. Don’t forget to present interesting points noticed along the way
5. Describe what worked and what did not (list assumptions & system’s limitations)
6. Ensure that key findings are communicated through beautiful visualizations or easy-to-remember statements (e.g., “the median income is the number-one predictor of housing prices”).

8. Launch the ML system

Finally, its time to launch and deploy the ML System in the desired platform. As it a software solution, the testing and maintenance also come under this step.The following things need to be done:

• Get your solution ready for production (plug in production data, write unit tests, etc.)
• Monitor the performance at regular intervals
• Retrain the models on a regular basis on fresh data

Extra Tips:

• Always work on copies of the data
• Never look on test set (use test set only for final model testing)
• Try to automate steps wherever possible on all of above steps
• Write functions for all data transformations (reuse)
• Feel free to adopt the checklist as per your needs

Originally published at https://exploreml.blogspot.com on April 28, 2019.