Deep Learning: How we cut compute time 6x with Claude

Can you make a deep learning model run faster without changing the model itself? At Batch, the answer was yes. The fix wasn't simplifying the algorithm. It was pinpointing exactly where compute was being wasted. That approach cut the churn prediction score's compute time from 2.5 hours to 20 minutes, with Claude.

This work was led by Batch's machine learning team, with Claude as copilot at every step. It's generative AI put to concrete use, not abstract talk: speeding up the optimization of the models behind a CRM platform's predictive scores.

The churn score: accurate, but far too slow

The churn score predicts, for each customer, the probability and likely timing of churn. Under the hood, it's a recurrent neural network (RNN), a type of deep learning model built to analyze sequences. Here, that means a customer's history over time.

The model itself worked. Its predictions were accurate. The problem was speed: one training iteration took about 2.5 hours, even on a well-equipped local machine.

In practice, a data scientist could run 2 tests a day at best, once you factor in launch time, waiting, analyzing results, and adjusting parameters. A cycle that long rules out any serious exploratory work. It also drives up the delay and cost of onboarding a new client onto this score.

A resource problem, not a complexity problem

The real issue was underused compute resources. The model's operations stayed relatively simple, but the load was never distributed or optimized for the available hardware. Picture a powerful engine stuck in first gear: the power is there, it's just not being used.

That distinction changes the whole approach. The goal isn't to simplify the model. It's to make better use of the machine running it.

The method: build a measurement lab before optimizing

Instead of tweaking parameters at random, the team first built an isolated test environment: a fully instrumented sandbox on a dedicated machine.

The goal was specific: measure what a standard managed cloud environment never shows you. Without that visibility, there's no way to know if the slowdown comes from the model itself, from how data moves to the GPU (the processor built for deep learning computation), or from an imbalance between data prep time and actual compute time.

This setup step, often skipped, is what let the team pinpoint exactly where to act. Without a measurement lab, optimization is just guesswork.

What claude changed in the process

Once the measurement environment was in place, Claude got full access to the machine. That only worked because the environment was isolated and posed no risk to production. Four concrete contributions came out of this project:

• Structuring experiments. Claude logged every test in a research notebook: hypothesis, measurements, results, comparison with previous attempts. A once-artisanal series of tests turned into a traceable process.

• Automating reporting. Claude generated the report for each experiment, a time-consuming task that used to eat into the team's schedule.

• Cross-referencing code and system behavior. Claude analyzed the correlation between code execution and hardware behavior, helping isolate the real bottleneck in the compute chain.

• Speeding up hypothesis testing, not replacing expertise. Claude doesn't magically "know" which parameter to change. Batch's data scientists identified which levers to test. Claude's value is in the speed and depth of exploration, not in some intuition of its own.

Heads up: this point matters more than any other. A generative AI model speeds up exploration. It doesn't replace data science expertise. The two work together, not one instead of the other.

The result: from 2.5 hours to 20 minutes, an iteration cycle that finally works

The score, which originally took around 2.5 hours, now runs in a 20-minute compute window. The associated compute cost dropped in the same proportion.

Beyond the numbers, this project delivered three lasting benefits:

• A viable iteration cycle for the data science team, with multiple tests possible per day instead of 2

• Full traceability of experiments: hypotheses, measurements, and results for each test, instead of scattered trials

• A reusable method for future performance projects, even if the optimization itself doesn't transfer as-is to another model

What this means for CRM teams

A shorter iteration cycle lets Batch's teams build a more reliable churn prediction score. Faster compute also changes the game on the CRM side: predictions can be recalculated more often. The result: retention campaigns run with more precision, at the right moment.

The method documented here (isolated measurement lab, structured experimentation, AI as technical copilot) applies to other predictive models in Batch Predict too, from engagement scoring to content recommendation.

At Batch, generative AI is used as an internal engineering tool: to make the models behind the platform's predictive scores, including the churn score, more reliable and faster.