Scale AI · Technical Core
Prepare Your AI Workload for Compute Access
Learn how data moves through its lifecycle, which infrastructure constraints shape an AI workload, how hosting choices differ and what working with high-performance computing (HPC) means. The goal is to prepare a credible request for future compute access—not to design or administer a production cluster.
Compute access is more than receiving a GPU
Useful access depends on three descriptions that must fit together. A powerful machine cannot compensate for an undefined workload, inaccessible data or an unreproducible software environment.
The task, job type, scale, frequency, target result and acceptable completion time.
Sources, preparation, permissions, volume, storage, transfer, outputs and retention.
Users, access controls, software, logging, quotas, support, ownership and cleanup.
What this module prepares you to do
Start with the workload, not the hardware
Translate the use case into work the infrastructure must perform before naming a cloud, cluster or accelerator.
What is a workload?
A workload is a bounded unit of computational work together with its input, output and operating requirements. “Train a model” is still too vague. Sizing begins with job type, data volume, frequency, acceptable duration, memory pressure, parallelism and the evidence the run must produce.
Compute is not one thing. A central processing unit (CPU) is flexible general-purpose compute. A graphics processing unit (GPU) performs many similar numerical operations in parallel and can accelerate suitable AI workloads. More GPUs do not guarantee a proportionally faster run: data loading, memory, software and communication can become the bottleneck.
| Workload shape | Primary concern | Typical first question |
|---|---|---|
| Interactive inference | Response latency, availability and predictable concurrency | How quickly must one user receive a result? |
| Batch inference | Throughput, scheduling and cost per processed item | Can the work run hourly, nightly or on demand? |
| Fine-tuning | GPU memory, dataset size, runtime and checkpoints | Which model, precision and training method will be used? |
| Large training or simulation | Parallelism, interconnect, storage throughput and reproducibility | Can the work use multiple nodes efficiently? |
Right-sizing principle
Request the smallest resource profile that can answer the next decision. Start with a short representative run, observe the bottleneck and revise the estimate before scaling the job.
Activity · Which description is compute-ready?
All three teams want GPU access. Select the description that gives an infrastructure specialist enough information to start a sizing conversation.
Trace the data lifecycle before moving data
Compute readiness begins at the source and continues after the job finishes.
One lifecycle, several gates
The data lifecycle describes how raw signals are collected, prepared, governed, made available to compute, transformed into results and eventually retained or removed. Every stage needs an owner and a minimum control. If a source definition, permission or quality rule is unclear, faster compute only produces unreliable results sooner.
Three controls travel through the entire flow
The boxes below are not extra stages. They are questions that connect several numbered stages. A compute result is trustworthy only when the team can follow these controls from the source to the retained output.
Stages 1–3 and 6
Quality
Define meaning and representative cases at the source, validate during ingestion, apply the quality gate during preparation and check whether compute results reveal new data problems.
Stages 1, 3, 4, 6 and 7
Lineage and version
Data lineage is the traceable record of where data came from, how it was transformed and versioned, where it moved and which outputs it produced. Link the source snapshot, preparation logic, labels, software and output so another authorised person can explain and reproduce the result.
All stages · especially 4, 5 and 7
Security and retention
Apply permitted use and least-privilege access throughout, protect the staged copy and decide which inputs, checkpoints, logs and outputs are returned, archived or deleted.
Optional deep dive · Storage close to compute
Large jobs are often limited by how quickly data reaches processors, not by processor speed alone. Repeatedly reading many small files across a slow connection can leave expensive GPUs idle even though the resource request looked sufficient on paper. This is why capacity—how much data fits—and throughput—how quickly it can be read or written—must be considered separately.
Authoritative storage and compute storage have different roles. The governed source remains the system of record. For a bounded run, the team may create an approved, versioned snapshot and stage that copy in storage designed for the compute environment. The staged copy is temporary working material, not a new uncontrolled master dataset.
File layout matters. Millions of very small files can create more overhead than a smaller number of well-organised files with the same total size. A representative transfer and loading test can reveal whether packaging, indexing or a different data format is needed before costly processors are reserved.
Integrity must be demonstrated. Record the snapshot identifier, file count and expected size; use checksums or an equivalent validation to show that the copy arrived unchanged. Failed, missing or quarantined records must be visible rather than silently omitted from the experiment.
Security follows the copy. Apply least-privilege access in the temporary environment, encrypt transfers where required, avoid unnecessary fields and define whether checkpoints or logs could reproduce sensitive information. Access to compute does not automatically grant access to every dataset.
Plan the exit before the transfer. Decide which metrics, checkpoints or logs return to the project workspace, who accepts them, when temporary data is deleted and what evidence proves cleanup. Include storage duration, transfer volume and cleanup effort in the cost and operating estimate.
Practical readiness check: before requesting the full run, transfer and load a representative sample. Record elapsed time, effective throughput, failed files, temporary storage used and the owner who approves the result. That small test turns a data-path assumption into sizing evidence.
Activity · Which action makes this data path compute-ready?
A research team wants to evaluate a model on case documents. Use of a redacted snapshot is approved, but no snapshot has been versioned, its transfer has not been checked and nobody owns deletion after the run. What should happen before requesting more compute?
Understand the infrastructure building blocks
Infrastructure is a delivery system: processors matter, but so do memory, storage, networking, software and operational controls.
CPU, GPU and accelerators
Execute the work. Suitability depends on the algorithm, supported software, precision and degree of parallelism.
Working space
Holds model parameters, activations and batches during a run. A job that does not fit cannot use the processor effectively.
Data and artefacts
Holds datasets, containers, checkpoints, logs and outputs. Capacity and throughput are different requirements.
Movement and coordination
Transfers data and connects nodes. Distributed jobs may depend on low-latency, high-bandwidth interconnects.
Reproducible execution
Drivers, frameworks, libraries, containers and versions must match the hardware and the tested workflow.
Safe shared use
Identity, permissions, scheduling, quotas, logging, monitoring, support and cost attribution make resources governable.
The bottleneck moves
A slow job does not automatically need more GPUs. It may be waiting for data, exceeding memory, using software that cannot parallelise, communicating too much between nodes or requesting resources it leaves idle.
Activity · Match each symptom to the first infrastructure question
Choose a hosting direction from constraints
Hosting determines where data and models run, who operates them and how much control, speed and responsibility the organisation accepts.
Fastest start
Provider operates the service. Strong for learning and variable use; assess data movement, service limits, dependency and unit cost.
Flexible capacity
Rent configurable compute and storage. Offers elasticity but still requires identity, security, cost control and operations.
More direct control
Organisation operates hardware or private infrastructure. Control comes with staffing, maintenance, utilisation and lifecycle responsibility.
Specialised batch compute
Access powerful shared resources through projects, quotas and a scheduler. Suits bounded parallel jobs more than always-on services.
| Decision dimension | Questions to answer |
|---|---|
| Data and sovereignty | Where may inputs, logs, checkpoints and outputs be stored or processed? |
| Workload shape | Interactive or batch? Continuous demand or occasional burst? One node or many? |
| Operations | Who patches, monitors, supports, secures and updates the environment? |
| Economics | What drives variable cost, idle capacity, transfer cost, licences and staff effort? |
| Optionality | Can data, containers, artefacts and evaluation evidence move to another environment? |
Hybrid is a design, not indecision
A hybrid approach can keep sensitive source data in an existing controlled environment, use temporary external compute for an approved snapshot and return only defined outputs. It earns its complexity only when a real constraint requires more than one environment.
Activity · Choose a hosting direction for a constrained workload
Scenario: sensitive source records must remain in the existing controlled environment. An approved, minimised snapshot may be transferred for occasional GPU-intensive batch training, and only checkpoints and evaluation outputs should return.
Understand how shared HPC changes the workflow
High-performance computing is powerful shared infrastructure. Users normally describe and submit bounded jobs rather than treating a machine like a permanently available laptop.
HPC in plain language
High-performance computing (HPC) combines powerful processors, large memory, fast storage and high-speed networking so demanding jobs can run efficiently. A cluster contains multiple computers, called nodes. A scheduler allocates shared resources according to job requests, priorities and availability.
A submitted job may wait in a queue before it runs. This is normal resource sharing, not a failure. A good request states the processors or GPUs, memory, expected runtime and software environment it needs. Over-requesting wastes scarce capacity and can increase queue time; under-requesting can make the job fail.
One computer in a cluster, with its own processors and memory.
A resource pool or policy lane for particular job types and limits.
The service that decides when and where submitted jobs run.
A non-interactive task defined in advance and executed when resources become available.
A saved intermediate state that can protect long runs from total restart.
How much useful speed is gained when more processors or nodes are added.
Activity · Which workload is a plausible HPC candidate?
Prepare a Compute Readiness Brief
Give an access or infrastructure team enough context to challenge the workload, recommend a starting resource profile and identify prerequisites.
A brief is a sizing hypothesis, not a purchase order
State what is known, label estimates and begin with a representative trial. The receiving team may recommend a smaller resource, different job shape, alternative hosting path or more data preparation before access.
Purpose and success
Decision supported, workload outcome, evaluation baseline and acceptable runtime.
Technical envelope
Job type, data volume, resources, software, storage, transfer and expected outputs.
Access and operation
Users, permissions, classification, owner, support, quota, logging, retention and cleanup.
Activity · Generate your Compute Readiness Brief
Use a fictional or sanitised workload. Do not enter dataset locations, credentials, personal data or sensitive system details.
Complete at least seven fields to generate your brief.
Scale AI technical summary
Six takeaways before requesting compute
Leadership questions
Which lifecycle gate can block the workload? What is the likely bottleneck? Why is this hosting direction proportionate? Can the team reproduce the environment? What will the first compute run prove, and who decides what happens next?