The Lifecycles of Data Science and MLOps

CRISP-DM for Data Science

I have used CRISP-DM — Cross Industry Standard Process for Data Mining — as a process model for data-science project execution for several years, and I can confirm that it works.

The process consists of six major steps. Data-science sub-tasks can be mapped to those steps as shown below.

Data Science Lifecycle

While studying for the Dell EMC Data Science Associate Exam, I learned that Dell also recommends a data-science lifecycle. Its six phases are named differently but serve similar functions: Discovery, Data Prep, Model Planning, Model Building, Communicate Results, and Operationalize.

The data-science lifecycle is iterative. You may move back and forth between phases as new information becomes available.

1. Business Understanding

• Determine business objectives and goals
• Assess the situation
• Produce a project plan

It might seem as though extensive documentation is required from the initial phase, and this is often cited as a weakness of CRISP-DM. In practice, a formal one-page business case can be enough. It should state both the business and machine-learning objectives, record relevant information from similar efforts, and be signed off by the right stakeholders.

This exercise helps prioritise the backlog and protects the team from scope creep.

2. Data Understanding

• Collect initial data
• Describe data
• Explore data
• Verify data quality

This is the “scary,” time-consuming, and crucial phase.

I would sum it up as ETL/ELT + EDA = ♥.

Acquiring data can be complex when it originates from both internal and external sources, appears in structured and unstructured formats, and must be assembled without dedicated data-engineering support. Without high-quality data, machine-learning projects become unreliable.

This part is often missing from online data-science courses and competitions, so data scientists should learn the essentials of ETL and ELT themselves.

Exploratory data analysis, or EDA, can be accelerated with tools such as ydata-profiling, Sweetviz, D-Tale, and AutoViz in Python; or DataExplorer, GGally, SmartEDA, and tableone in R.

Most teams use a combination of automated profiling and custom exploratory analysis.

3. Data Preparation

• Select data
• Clean data
• Construct data
• Integrate data
• Format data

This is another code-intensive phase. It requires knowing the dataset well enough to select modeling data, clean it, and perform feature engineering with confidence.

EDA reveals missing data, outliers, and highly correlated features. In the Data Preparation stage, however, you decide which imputation strategy to apply. You may return to Business Understanding when a new hypothesis emerges or when subject-matter experts need to confirm an assumption.

Feature engineering is central here. Until recently, it was largely a manual task: creating new variables from available features through data-wrangling techniques. Data scientists commonly use pandas in Python and dplyr in R. Tools such as Featuretools can speed up the process, but they should support — not replace — informed human judgment.

Feature Selection

• Feature importance: algorithms such as random forests and XGBoost can estimate which variables contributed most to predicting the target. Building an initial model and reviewing feature importance can help identify useful variables.
• Dimensionality reduction: techniques such as Principal Component Analysis, or PCA, reduce a large number of features into a smaller set using linear algebra.

By the end of this phase, much of the project foundation is in place. Then comes the part most people associate with machine learning.

4. Modeling

• Select modeling techniques
• Generate test design
• Build the model
• Assess the model

Once it is clear which algorithm to try first, begin by separating training and test data.

from sklearn.model_selection import train_test_split

Splitting a dataset into training and test sets is essential for evaluating model performance. The model is trained on one subset and assessed on previously unseen data.

Depending on model complexity, writing the initial code can be quick. In Python, scikit-learn is a common library for model building. In R, useful packages include caret, e1071, xgboost, and randomForest.

As you assess model quality, you may return to earlier phases and iterate. Modeling time is subjective: models can always be tuned further, but there is a point at which the additional gain in performance does not justify the additional time or complexity.

Accuracy alone is rarely enough. Choose metrics appropriate to the problem, such as precision, recall, F1 score, ROC-AUC, loss functions, thresholds, sensitivity, and specificity.

5. Evaluation

• Evaluate results
• Review the process
• Determine next steps

At this stage, decide which model to select using evidence such as ROC curves, the number and quality of features, operational constraints, and business feedback.

For more practical reading on evaluation, see Jeremy Jordan’s guide to evaluating machine-learning models.

6. Deployment

• Plan deployment
• Plan monitoring and maintenance
• Produce the final report
• Review the project

During deployment preparation, create a final report that documents whether the model met its objectives. Define how model stability and accuracy will be monitored, and establish the conditions that should trigger retraining.

Always communicate the outcome back to the business.

MLOps Lifecycle

You may have noticed that the word “automated” appears several times in this article. Does that mean machine learning can become 100% automated and data scientists will no longer be needed? I believe the opposite.

Automation can help junior data scientists ramp up faster and free experienced teams to focus on higher-value work. Useful tools include:

• DataRobot
• H2O AutoML
• Alteryx
• RapidMiner
• Dataiku DSS
• Amazon SageMaker
• Google Cloud AutoML
• Qlik AutoML
• Azure Machine Learning Studio

Senior data scientists also benefit from AutoML tools, particularly when they must scale ML delivery quickly. This is where MLOps becomes essential: applying DevOps practices to data-science and machine-learning workflows so that delivery becomes more reliable, automated, and reproducible.

Affiliate disclosure: I participate in the Amazon affiliate programme. This post may contain affiliate links from Amazon or other publishers I trust, at no extra cost to you. I may receive a small commission when you buy through these links. See the affiliate disclosure for details.

MLOps extends the data-science lifecycle into production. It helps teams build, deploy, monitor, and improve machine-learning systems in a reliable and repeatable way.

MLOps means scaling end-to-end data-science products in a reliable and automated way. Deploying a single data-science product is not always straightforward; deploying several each month without automation can be slow, error-prone, and dependent on a few individuals.

Prepare for Production

• Runtime environment
• Risk evaluation
• Quality assurance

If a model works on your machine, the next question is whether it will work in production. First validate it in a development environment, then prepare it for production. In an ideal setup, the data scientist exports the model and the data engineer deploys it. Real-world resource constraints can make that boundary less clean.

Agree and test the production format early. Once the conversion is proven, add it as a post-modeling pipeline step.

Another production-readiness consideration is data access: test internal and external data connections, credentials, and required configurations.

The risk-evaluation milestone addresses model risk. Assess and document it, log material risks, and ensure they are mitigated in time.

For quality assurance, borrow the advanced tools and practices used by software engineering wherever possible.

Watch the video below for an example of how unit tests can look for data scientists.

Development to Production

• Elastic scaling
• Containerization
• CI/CD pipelines

CI/CD refers to continuous integration and continuous delivery. In MLOps, after deployment, the team should push code, metadata, and documentation to a central repository and trigger an automated CI/CD pipeline.

A practical MLOps delivery pipeline can follow three stages:

1. Build the model
   1. Build model artifacts
   2. Send artifacts to long-term storage
   3. Run checks
   4. Generate fairness and explainability reports
2. Deploy to a test environment
   1. Run tests to validate ML and computational performance
   2. Validate manually
3. Deploy to a production environment
   1. Deploy the model as a canary release
   2. Fully deploy the model

Monitoring and Feedback Loop

• Logging and alerting
• Input-drift tracking
• Performance drift
• Online evaluation

Once a model is in production, performance may change. Performance-drift management defines how much degradation is acceptable before the model must be investigated, updated, or retrained.

Input-drift tracking is equally important. It helps identify changes in incoming data: schema mismatches, changes in missing-value rates, NaNs, infinities, Population Stability Index (PSI), Characteristic Stability Index (CSI), and other warning signals.

Monitoring should also account for system upgrades and environment changes, including GPU memory allocation, network traffic, and disk usage. You can read more here.

Logging and alerting help teams stay ahead of problems. For further reading, see this paper on machine-learning-specific risk factors and design patterns that should be avoided or refactored where possible.