Why analytics and ML live in different systems
Analysts calculate metrics in the warehouse, while models train on extracts — and these are two different worlds. Data for training is copied to a separate environment because the training environment cannot read the warehouse directly.
The copy lives its own life: it updates on a schedule, lags by a day or even a week, and no one can guarantee that it contains the same version of the mart as the report.
Then metric desynchronization begins. The report shows one number, the model relies on another, and at the planning meeting it turns out there is nothing to argue about — the sources are different.
Manual extracts add a human factor: someone forgot to reload a table, someone modified the schema, and training fails or, worse, quietly trains on stale data.
The cost of this gap is not just engineers' time spent on reloading. A model trained on a copy degrades imperceptibly: quality drops not at the moment of failure but gradually, and it can be noticed only through business metrics.
The analyst and the data scientist stop trusting each other, and decisions are made on numbers that no one can reconcile.
Snowflake ML eliminates the gap itself: model training and inference happen where the data resides, without intermediate copies or manual extracts. One version of the mart — one set of metrics for both the report and the model. Let's break down how this works and what changes in the team's workflow.
What a unified environment for models provides
When data and computation live in a single loop, the most expensive stage disappears — shuffling exports between the storage and a separate cluster for training.
Features and the target variable are assembled with a single query to the same tables where analytics runs, so preparing a dataset for a model shrinks from weeks to hours, and the discrepancy between "data for a report" and "data for a model" disappears as a category.
Reproducibility here is not a declaration but a consequence of how the environment is built.
The dataset version, the code version, and the run parameters are recorded together with the training result, so any experiment can be re-run and produce the same metric — without archaeology through someone else's notebooks.
When a model degrades, an engineer finds in minutes what changed: a new data slice arrived or the feature logic changed.
Manual operations move to where they belong — into scheduled routines. Feature rebuilding, training, and forecast export are triggered on a schedule and by events, not by a request in chat; the analyst stops being a dispatcher between teams. The freed-up time goes into testing hypotheses rather than repeating the same run.
A unified environment also changes how the team agrees on the result.
The data engineer, the analyst, and the model specialist work with the same objects and the same change history, so the argument "it computes differently on my side" is resolved by a reference to a specific calculation rather than another meeting.
We deploy such an environment within your infrastructure and hand it over to the team together with the routines — so that the effect holds after launch, not just in the first month.
Options for organizing computations
The choice of approach determines where queries and model training are physically executed: inside the warehouse, in a separate environment, or in a hybrid scheme.
In practice, the decision is made along three axes — data volume, latency requirements, and the mode of access to raw events. Below are the working options that teams realistically choose between at the start.
| Option | When it fits | Limitations |
|---|---|---|
| Computations in the warehouse | Data is already there, marts and training on historical slices are needed, latency of minutes is acceptable | Warehouse compute resources are shared with analytics; heavy jobs compete with reports |
| Separate environment | Custom libraries, non-standard models, strict control over the training environment are needed | Data has to be exported and synchronized; the risk of feature version divergence grows |
| Hybrid scheme | Training on large volumes in the warehouse, inference — next to the product | Harder to track feature provenance and experiment reproducibility |
What to check before choosing
The key criterion is where features are born. If they are assembled from the same tables as reporting, computations in the warehouse eliminate a whole class of discrepancies: one logic, one slice, no exports.
A separate environment is justified when specific libraries or workload isolation are needed — then feature synchronization is built as a separate regular process with a match check.
A hybrid scheme is a compromise for mature teams: heavy training stays where the data lives, while inference is moved closer to the user so that traffic is not routed through the warehouse.
We record the choice in a document with rationale, so that six months later it is clear why the environment is set up exactly this way.
How we implement Snowflake ML
Implementing Snowflake ML starts with an audit: we look at where the data currently lives, how the marts are structured, which roles already exist, and who has access to what.
At this stage, it's important to understand which models the business actually needs and which tasks can be solved with regular SQL — this saves weeks of work. Based on the audit results, we record a data map and a list of scenarios that will deliver a measurable effect.
Next, we design the architecture: we lay out the data across layers, determine where models are trained and where inference happens, and how the results are returned to reporting.
We set up roles and permissions right away according to the principle of least privilege — an analyst sees the marts but doesn't touch the training datasets, an engineer works with features but not with raw personal data. This removes most of the risks at the review stage.
- Data and scenario audit. We examine the sources, quality, volume, and update frequency. We record which models are needed and what result counts as success.
- Architecture design. We define the data layers, training and inference points, and the way results are delivered to reporting and operational systems.
- Setting up roles and access. We delineate permissions according to the principle of least privilege, and set up separate roles for analytics, engineering, and operations.
- Data and feature preparation. We build the marts, describe the features, and set up regular updates without manual runs.
- Model training and validation. We compare quality on a holdout set, check robustness to data drift, and select models for production launch.
- Validation in the customer's environment. We run the scenarios on real data, compare the results with current calculations, and eliminate discrepancies.
- Handover and team training. We hand over documentation, model update regulations, and a breakdown of typical failures so that the system can live without us.
Each stage ends with an artifact that can be verified: a data map, an architecture diagram, an access matrix, a model quality report.
This order makes it possible to launch scenarios one at a time without waiting for the entire platform to be ready, and at any step to see what is already working and what is still in progress.
Artifacts and documentation after implementation
After project handover, you are left not with "magic in the cloud" but with a working environment that your team can maintain and evolve without us.
We hand over everything needed for reproducible model recalculation, access control, and knowledge transfer to new engineers.
- Environment configurations. Parameter files for development, test, and production environments: roles, compute warehouses, recalculation task schedules, resource limits. An environment can be deployed from scratch using these files, without restoring settings manually.
- Data and model schemas. A schema of layers — from raw data to marts and feature tables, with a description of relationships and owners. Separately — a feature schema with versions: you can see which data each model was trained on and what changed between recalculations.
- Access policies. A matrix of roles and permissions: who sees raw data, who sees only aggregates, who triggers recalculation. Rules for masking sensitive fields and row-level access control, so that an analyst does not export more than they should.
- Recalculation regulations. A calendar for model training and updates, trigger conditions, quality thresholds at which a model goes for retraining or rolls back to a previous version. Plus the procedure for handling failures.
- Instructions for the team. Step-by-step guides: how to add a feature, how to retrain a model, how to read quality metrics, and where to look when a forecast diverges from actuals.
- Monitoring and alerts. Configured checks for data drift and model degradation, trigger thresholds, and a list of notification recipients.
- Decision log. What was chosen and why: data sources, feature logic, rejected hypotheses. This saves weeks during the next evolution of the solution.
All of this is in your repository and your cloud account, with access for your engineers. Describe your task — we will come back with an estimate and a plan, including the composition of the documentation to be handed over.
Case study: accelerated data preparation for models
The client — a product analyst in retail — was preparing data marts for demand forecasting models manually: exports from several sources, consolidation into tables, cleaning of gaps and duplicates.
One iteration took 2-3 days, and the discrepancy between the training and production samples surfaced only after the model was launched.
What we did
We moved data preparation into Snowflake ML: raw data stays in the warehouse, and transformations are described declaratively — the same code prepares the sample for both training and inference.
Heavy steps — joins, aggregations, feature computation — are performed where the data resides, without intermediate exports.
We added quality control at every step: checks for gaps, duplicates, and feature distribution drift. If a check fails, the next step does not start — the error is visible at the input, not at the model's output.
Metrics before and after
| Indicator | Before | After |
|---|---|---|
| Data preparation per iteration | 2-3 days | 3-4 hours |
| Manual operations in the process | 11 | 2 |
| Pipeline failures per month | 6-8 | 0-1 |
| Discrepancy between training and production samples | regularly | not reproducible |
Only acceptance of the result and approval of changes in feature logic remained manual — everything else runs on a schedule and on an event. The analyst returned to interpreting results instead of routine table assembly.
What this gave the business
The demand forecasting model is updated daily on fresh data, rather than once a week. Forecast accuracy increased because training and production now see the same features.
The team stopped spending time on investigating discrepancies and manual reconciliation — the freed-up hours went to new scenarios.
How Snowflake ML differs from exporting models
The key difference is where features are physically computed and the model is trained. When models are exported, data is first unloaded from the warehouse into an external environment, features are recomputed on that environment's side, and only then does training start.
In Snowflake ML, feature computation, model training, and model inference all happen where the data lives: features are computed once and in one place — for both training and inference — so the discrepancy between the training and production samples disappears as a class.
The second difference is how access is organized. With exporting, you have to expose extracts, copy sensitive fields into external storage, and separately synchronize permissions: who sees raw data, who sees features, who sees results.
In Snowflake ML, access is inherited from the warehouse's own roles: the model is trained on the same tables and views the team already has rights to, and no copies of data appear outside the perimeter.
The third is cost and maintenance complexity. Exporting adds a separate loop: scheduled data movement, its own training infrastructure, a separate layer for model inference, and synchronization of feature versions between them.
Snowflake ML removes these links — there is no shuttling of data back and forth, no two inconsistent implementations of the same feature, no separate loop that needs to be updated and secured.
In practice, this changes not only the architecture but also the pace of work.
When features, training, and inference live in one place, a new model version goes from idea to production without rebuilding data pipelines, and the result is reproduced on the same feature definitions as during training. Let's break down how this works in your data landscape and what it delivers.
Common technical concerns
Working with data in cloud storage raises a predictable set of questions: where the data physically resides, what happens with access, how the solution will behave under load, and who maintains it after launch. Below are answers to those that most often come up during the technical evaluation stage.
Where is the data stored and how is access segregated?
The data stays in your cloud storage account — we do not export it to our side. Access is granted through roles with the minimum necessary permissions: separately for reading data marts, separately for writing training results.
Keys and tokens are stored in a secure secrets vault, not in code. All access to data is recorded in an audit log, so it is clear who read a specific dataset and when.
Is this compatible with our current stack and data sources?
First we look at where the data comes from: relational databases, file storage, event streams, external services. Most sources are connected with standard connectors; for the rest, we build an exchange through an intermediate layer.
If part of the analytics already lives in another tool, we migrate it gradually — old reports keep working until the new ones produce the same numbers.
Will the solution withstand data volume growth and peak load?
Compute scales separately from storage: heavy training jobs run on isolated compute resources and do not compete with reports. As volume grows, processing nodes are added rather than the logic being rewritten.
Peak periods are handled through scheduling: training runs in low-activity windows, while interactive queries get priority.
What about maintenance and rollback after launch?
Models and transformations are versioned: every change is recorded, and the previous version remains available for rollback. Quality and response time metrics are displayed on a dashboard, so deviation from the norm is visible immediately.
Maintenance includes updating dependencies, handling incidents, and scheduled quality checks — according to a procedure, not after a failure occurs.
Describe your task and current data landscape — we will come back with an assessment and an integration plan.
Let's discuss your task and come back with a plan
For the estimate to be realistic rather than an "market average", we need to understand your current analytics picture: where the data lives, how it gets into Snowflake ML, which teams are already working with data marts, and what exactly should change in the business process after implementation.
Describe the task in free form — that is enough for us to ask targeted clarifying questions and come back with a work plan.
At the first meeting, we discuss not abstract "wishlist items" but specifics: data sources and volume, update frequency, requirements for feature quality, user roles, and the scenarios in which the model or report should provide an answer.
We separately talk through constraints — on access, on timelines, on the load on existing systems. The more precise the inputs, the fewer assumptions we build into the plan.
What we will prepare based on the review:
- a diagram of Snowflake ML integration with your data sources and data marts;
- a list of clarifying questions on access, roles, and permission separation;
- a draft work plan with stages, dependencies, and checkpoints;
- an estimate of effort and timelines for each stage, with no hidden assumptions;
- implementation options with the pros and limitations of each — so the choice is informed;
- a list of risks and ways to mitigate them at early stages;
- a proposal for a pilot scenario to validate hypotheses before full implementation.
Describe the task — what you already have, what you want to achieve, and by when. We will come back with clarifications, an estimate, and a work plan you can use to make a decision.
