Building an AI agent with Code Interpreter: executing code in a sandbox

AI agent with code interpreter: where it breaks on real tasks

An AI agent with a code interpreter works differently from an ordinary chatbot: instead of answering “from memory,” the model writes code, runs it in an isolated environment, reads the output, and refines the calculation.

It is code execution in the agent that delivers what all this is for — numbers are calculated from your files rather than guessed. That is why such an agent first breaks where data stops being textbook-clean.

The first point of failure is schema mismatch. In a test set, the table has uniform columns; in a real export, fields are renamed, values are empty, delimiters differ, sheets have no header.

The agent generates code for the format it saw in the first iteration and fails on the first file that does not match.

The second is a loop: a failure is not recognized as a failure, the agent edits one line, gets the same result, and repeats until the step limit, burning time and tokens.

The third is run isolation: if the environment is not restricted, generated code reaches into the network, reads environment variables with keys, and writes outside the working folder.

And reproducibility of results: without pinning a data snapshot, a set of libraries, and a random seed, two runs of the same task produce different numbers, and the report has nothing to defend itself with before the person who will use it to make a decision.

Next, we break down how this is addressed.

What managed code execution in the agent provides

Managed code execution is when code generated by the agent goes not into the main process, but into a disposable isolated environment with a defined policy: available modules, network, process lifetime, memory and disk limits.

The agent remains the planner, while execution is a separate step with a verifiable input and output.

Repeatable runs rely on a pinned environment, the same input data, and the same limits, so incident analysis reduces to replaying the request rather than arguments about “it works locally for us.” The run trace is human-readable: code, logs, intermediate files, and charts are stored in a Jupyter-based notebook, and you can see which step produced each number.

Restricting agent permissions eliminates a whole class of failures: network access only by allowlist, writes only to the working folder, secrets do not enter the environment.

Incorrect code damages a disposable environment, not the production environment, and agent stability stops depending on the model’s accuracy.

• The same request and the same environment produce the same artifact — the result is verifiable before handoff to the customer.
• A failure in one step does not bring down the service: the environment is recreated from a clean state, and recovery takes minutes.
• Time, memory, and disk limits cut off looping before it consumes the shared environment.
• Network access by allowlist closes data leakage outward.

The result is predictable resource consumption and reproducibility instead of hoping for a lucky run: the run boundaries, its trace, and limits are described in advance.

Code interpreter operating modes: comparison of approaches

Before an AI agent with a code interpreter executes its first line of code, you need to decide where that code runs.

Three things depend on the mode: whether the agent can reach internal systems, what data will leave the perimeter, and who is responsible for operating the environment.

Local run: where it is appropriate

Local run is needed at the debugging stage: the engineer sees the output and repeats the scenario on their own data.

As a permanent mode, it is unsuitable — each person has their own set of libraries, the environment state is not reproducible, and a failure of one scenario takes down the workflow. There is no shared log, so you cannot use it to determine what exactly the agent executed.

Sandbox in the customer’s infrastructure

An isolated sandbox is deployed in your environment: code runs in a separate container with reduced privileges, the file system is read-only, and outbound connections are limited by an explicit allowlist.

Data does not leave the perimeter, and limits on CPU time and memory prevent one scenario from occupying the entire node. The cost is dedicated resources and joint operation of the environment.

We define environment restrictions before the start, not after the first incident — otherwise the code execution mode is chosen based on the fact of a failure.

How we implement the agent: stages of engineering work

We carry out the work in an order in which risks surface before the agent gets access to data and external systems. First we define the scenarios, then we build isolation, after that we configure permissions and limits, and only then do we run acceptance on real tasks.

1. Agent scenario audit — we analyze what tasks it solves, where it executes code, what data it reads, and where it writes. The result is a scenario map with a list of operations and points where human verification is needed.
2. Isolation scheme design — we determine in which environment the code runs, how often it is recreated, and what persists between runs. We check the scheme from two sides: what the agent can break and what can leak.
3. Permission and limit configuration — we grant only the necessary accesses: specific directories, an allowlist of addresses, restrictions on execution time and write volume. Anything outside the boundaries simply does not execute.
4. Data source connection — the agent receives the files, database, and external services it works with. Every connection goes through the same permission control.
5. Run on real tasks — we test scenarios together with expected failures: invalid input, unavailable service, exceeding a limit. We observe how the agent recovers and what gets into the log.
6. Agent acceptance — you verify the result on your tasks according to criteria agreed in advance. We separately define which failures we consider acceptable and which block launch.
7. Handover of the scheme and support regulations — we provide the architecture description, rules for updating scenarios, and the procedure for actions in case of failure.

What we hand over: configurations, policies, and regulations

We hand over not a “working prototype,” but a package your operations team can use to deploy the agent in its own environment and support it without us.

The code interpreter executes generated programs in an isolated environment — therefore everything related to permissions, restrictions, and observability must be documented in writing. The set of artifacts looks like this.

• Runtime environment configuration: description of the base image, pinned list of libraries, network access rules, mount points for working files.
• Access policies: roles and permissions, key lifetime, secret rotation procedure, separation between test and production environments.
• Resource limits: maximum time per run, memory and disk volume per session, number of concurrent sessions, behavior when quota is exhausted.
• Agent tool matrix: which calls are allowed automatically, which require operator confirmation, which are fully closed.
• Artifact storage scheme: where intermediate files are stored, how their expiration is configured, and how to delete data on request.
• Handover documentation: deployment instructions, dependency list, environment variable description, log and metric connection points.
• Set of checks: scenarios for typical requests, tests for limit exhaustion, quality control of final answers before rollout.
• Support regulations: incident response levels, image and library update windows, rollback procedure to the previous version.

We write the documents so that an engineer can reproduce the environment, change a limit, or revoke access without our participation. We separately specify which configuration areas are intended for independent editing and which are tied to agent logic and require approval.

Case study: agent stability before and after code isolation

The customer’s analytics platform delegated calculations to an AI agent: it wrote code on a user request and executed it directly in the API process.

While requests were simple, this worked; on a stream of several hundred sessions per day, one infinite loop or memory leak in someone else’s script would bring down the entire working process — along with other tasks and unfinished dialogues.

The share of failed tasks held at 18%, and analyzing each crash took the on-call engineer hours: from the logs, it was impossible to distinguish an agent error from a crash of the overall process.

We moved execution into a separate isolated environment on Docker: its own read-only file system, a temporary volume for artifacts, a ban on network calls, and strict CPU and memory limits per session.

The image with a pinned set of libraries is built in advance — the agent’s code does not pull dependencies from the internet or break the environment for its neighbors.

What changed in agent behavior

A session failure stopped being a general outage: a crashed calculation terminates by time limit, and the queue continues to work. The share of failed tasks decreased from 18% to 2%, and there were no “agent hung and does not respond” refusals for the quarter.

Repeated runs of the same request produce an identical result in 96% of cases — previously discrepancies were caught manually because the environment “drifted.”

Diagnostics also benefited separately: artifacts, run logs, and input data are stored in one directory — the incident is analyzed by facts, not by chat correspondence.

Recovery time after a calculation crash is seconds instead of manually restarting the process and rerunning the dialogue.

What to do if the agent already works without code isolation?

There is no need to stop the scenarios: execution moves into an isolated container in parallel, and the previous path remains operational until results are compared.

The difference is fundamental — a process sandbox restricts calls inside the interpreter but shares an address space with the agent, while container-level isolation sets a boundary for the file system, network, and user permissions.

Therefore, a failure in the first option brings down the entire service, while in the second it is extinguished together with the container.

Migrating an already deployed agent looks like this: in the Kubernetes cluster, a second execution environment is brought up alongside, routing is switched by a share of traffic via a flag, and responses are compared with the previous ones on one sample of tasks.

Discrepancies are analyzed before the full switch, and rollback is done with the same flag. Containers are reused between calls, so response time does not grow, and memory and process count limits are set separately for each run.

As a result, user code stops being considered trusted by default: even with a malicious script, only allowed files leave outward, and the environment boundary cuts off the rest.

Common technical doubts about execution isolation

Engineers ask these questions most often during review: where are the boundaries of the agent’s access to files, what remains in the environment after a session, and how the system behaves when generated code crashes. Below are short answers to each point, without generalities.

How is the environment cleaned between runs?

State lives exactly as long as the dialogue proceeds: the agent needs to see intermediate files and results of previous steps.

After the session ends, the container is deleted entirely together with the file system, package cache, and temporary data — the next run starts from a clean image. Only artifacts from an explicit list are transferred outward, so one user’s data does not intersect with another’s.

What happens if the code crashes or loops?

A limit is set on execution time, memory, and number of steps per task.

If exceeded, the process is terminated, and an error log is returned to the agent — it rewrites the code and repeats the step, but does not go into an infinite loop: looping is cut off by an iteration counter.

One failed run does not pull away the resources of the entire task, and the cause of the crash remains in the log for analysis.

Let’s discuss the task: we’ll design a scheme for your scenario

To make the scope estimate rely on facts rather than assumptions, for the first conversation it is enough to have a scenario description, an example of input data, and a list of systems the agent will exchange with.

If there are current artifacts — instruction formulations for the model, tests, failure logs — send them: they noticeably shorten the review.

The task discussion proceeds in steps: we go through the scenario in full, note what the agent does on its own and where a human is involved, how the correctness of the result is checked, and what we consider an error.

Based on the results, we assemble an implementation plan with the sequence of stages and points at which readiness is confirmed. Based on the discussion, the customer receives:

• solution scheme: where data comes from, in which isolated environment the code runs, where the result is returned;
• environment boundaries: available libraries, network and file system restrictions, process lifetime;
• list of checks and test scenarios, including failure branches and repeated runs;
• scope estimate by stages with launch order and control points;
• list of risks and solutions that remove them before development begins;
• operation regulations: logging, error tracking, procedure for updating agent rules.

Describe the task in free form — we will come back with an estimate, environment scheme, and plan for the first stage. If the scenario has not settled yet, we will review the options and propose the one that can be tested on real data faster.