RunPod

Summary

Explore our guides and examples to deploy your AI/ML application on Runpod. Review setup and usage guidance in the Runpod documentation.

Release history

1. docs update Aug 26, 2026 · issue 008

   RunPod Serverless adds Batch Jobs API for async bulk inference with dedicated workers, isolated from standard /run traffic

   └──▷ TRY IT
   Create a batch with an initial set of embedding requests and get back a batch ID to track progress.
   copy

   $ curl -X POST https://api.runpod.io/v2/{endpoint_id}/batch \
     -H 'Authorization: Bearer {api_key}' \
     -H 'Content-Type: application/json' \
     -d '[{"input":{"text":"The quick brown fox"}},{"input":{"text":"Jumped over the lazy dog"}}]'

   Finalize a draft batch to lock it and start processing, then poll for completion using request counts.
   copy

   $ # Finalize
   curl -X POST https://api.runpod.io/v2/{endpoint_id}/batch/{batch_id}/finalize \
     -H 'Authorization: Bearer {api_key}'
   
   # Poll until requestCompleted + requestFailed == requestTotal
   curl https://api.runpod.io/v2/{endpoint_id}/batch/{batch_id} \
     -H 'Authorization: Bearer {api_key}'

   Page through completed and failed request results after a batch finishes, inspecting error messages on failed items.
   copy

   $ curl 'https://api.runpod.io/v2/{endpoint_id}/batch/{batch_id}/requests?offset=0&limit=50' \
     -H 'Authorization: Bearer {api_key}'

   • ›New POST /v2/{endpoint_id}/batch endpoint creates a batch job (optionally with initial requests) that runs asynchronously on dedicated workers, isolated from standard /run traffic.
   • ›New POST /v2/{endpoint_id}/batch/{id}/requests endpoint appends additional requests to a DRAFT batch incrementally; each call accepts up to 10 MiB of request body.
   • ›New GET /v2/{endpoint_id}/batch/{id} endpoint returns batch summary with progress fields requestTotal, requestInProgress, requestCompleted, and requestFailed.
   • ›New GET /v2/{endpoint_id}/batch/{id}/requests endpoint returns paginated child request results; supports offset and limit query parameters with a hasMore field for pagination.
   • ›Batch lifecycle moves through states DRAFT → FINALIZED → FAILED / CANCELLED; completion is inferred when requestCompleted + requestFailed equals requestTotal.
   +1 moreshow less

   • ›Console Inbox notifications are sent on terminal state transitions, including batch ID, endpoint name, status, and item counts.
2. docs update Aug 26, 2026 · issue 008

   RunPod Serverless adds batch jobs: submit large inference request collections as a single named batch, processed asynchronously.

   • ›New batch jobs capability for RunPod Serverless allows submitting large collections of inference requests as a single named batch for asynchronous processing.
3. launch-20260825-f17a307f Aug 25, 2026 · issue 009

   RunPod Flash lets you deploy GPU-accelerated Python functions to Serverless with @remote — no Dockerfile needed.

   └──▷ TRY IT
   Deploy a GPU Python function to RunPod Serverless without writing a Dockerfile — ideal for fast iteration on inference code.
   copy

   $ pip install runpod-flash

   Serve a Flash-decorated FastAPI app as a live GPU inference endpoint from your terminal.
   copy

   $ flash run

   • ›New runpod-flash Python SDK introduces the @endpoint decorator to deploy GPU-accelerated Python functions directly to RunPod Serverless, specifying GPU type, worker count, and pip dependencies inline — no Dockerfile or registry push required.
   • ›New flash run CLI command serves Flash-decorated endpoints locally and as production APIs, enabling GPU inference behind a FastAPI router with a single command.
   • ›Flash endpoints are full RunPod Serverless endpoints, providing autoscaling, cold-start management, and access to RunPod's full GPU fleet without additional configuration.
4. New ReleaseBatch Jobs (BETA) Aug 1, 2026 · issue 008

   RunPod launches Batch Jobs (BETA) for Serverless endpoints, adding a full async bulk-inference API via REST API v2.

   └──▷ TRY IT
   Submit a batch of embedding requests to a Serverless endpoint and poll until all requests are done.
   copy

   $ # 1. Create a batch with initial requests
   curl -s -X POST 'https://api.runpod.io/v2/{endpoint_id}/batch' \
     -H 'Authorization: Bearer {api_key}' \
     -H 'Content-Type: application/json' \
     -d '[{"input":{"text":"The quick brown fox"}},{"input":{"text":"Jumped over the lazy dog"}}]'
   
   # Response: {"id": "batch_01j9abc123", "status": "DRAFT"}
   
   # 2. Finalize the batch to start processing
   curl -s -X POST 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/finalize' \
     -H 'Authorization: Bearer {api_key}'
   
   # 3. Poll progress until requestCompleted + requestFailed == requestTotal
   curl -s 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123' \
     -H 'Authorization: Bearer {api_key}'

   Retrieve the first page of results for a completed batch, then page through remaining results.
   copy

   $ # Fetch first 50 results
   curl -s 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests?offset=0&limit=50' \
     -H 'Authorization: Bearer {api_key}'
   
   # Fetch next page
   curl -s 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests?offset=50&limit=50' \
     -H 'Authorization: Bearer {api_key}'

   Incrementally build a large batch in chunks under the 10 MiB per-call limit before finalizing.
   copy

   $ # Add a second chunk of requests to a DRAFT batch
   curl -s -X POST 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests' \
     -H 'Authorization: Bearer {api_key}' \
     -H 'Content-Type: application/json' \
     -d '{"requests":[{"input":{"text":"More text to embed"}},{"input":{"text":"Another piece of text"}},{"input":{"text":"And another one"}}]}'

   • ›Adds POST /v2/{endpoint_id}/batch to create a new batch (optionally with initial requests as a JSON array) against a Serverless endpoint.
   • ›Adds POST /v2/{endpoint_id}/batch/{id}/requests to append requests to a DRAFT batch; request body is limited to 10 MiB per call and can be called multiple times to build large batches incrementally.
   • ›Adds POST /v2/{endpoint_id}/batch/{id}/finalize to lock a batch and make it eligible for execution, transitioning status from DRAFT to FINALIZED.
   • ›Adds GET /v2/{endpoint_id}/batch/{id} to poll batch progress via requestTotal, requestInProgress, requestCompleted, and requestFailed counts; createdAt is a Unix epoch timestamp in milliseconds.
   • ›Adds GET /v2/{endpoint_id}/batch/{id}/requests to retrieve paginated child-request results including per-request status, output, error, startedAt, and completedAt; supports offset and limit query parameters with a hasMore field.
   +9 moreshow less

   • ›Adds POST /v2/{endpoint_id}/batch/{id}/cancel to cancel a batch; queued requests are cancelled immediately (not billed) while in-progress requests finish and are billed normally.
   • ›Adds DELETE /v2/{endpoint_id}/batch/{id}/requests/{requestId} to remove a single request from a DRAFT batch.
   • ›Adds PUT /v2/{endpoint_id}/batch/{id} to update batch attributes such as display name.
   • ›Adds GET /v2/{endpoint_id}/batch to list all batches for an endpoint, newest first.
   • ›Batch jobs run on dedicated workers isolated from standard /run traffic, so submitting a batch never delays interactive requests.
   • ›Batch limits: 10 active batches per endpoint, 5,000 requests per batch, 50,000 queued requests per endpoint (up to 1,000,000 daily for enterprise); limits are configurable for enterprise accounts.
   • ›Adds webhook event notifications when a batch reaches a terminal state (FAILED or CANCELLED), if a webhook subscription is configured for batch events.
   • ›Adds a Batch tab in the RunPod console for each endpoint, showing batch name, status, progress counts, per-request rows with timestamps and error messages, and links to child request detail views.
   • ›Adds AWS ECR Integration (BETA) allowing private container images to be pulled into Pods and Serverless endpoints via ECR delegation, without migrating registries or managing credentials.
   └──▷ BREAKING ON UPGRADE

   • !REST API v1 will be retired on November 15, 2026; existing integrations must migrate to REST API v2 at the new base URL https://api.runpod.io/v2 before that date.
   • !The GraphQL API will be retired in early 2027; new integrations should use REST API v2.
5. New ReleaseBatch Jobs (BETA) Aug 1, 2026 · issue 003

   RunPod launches Batch Jobs (BETA) with a full REST API for submitting, managing, and polling large async inference workloads on Serverless endpoints.

   └──▷ TRY IT
   Kick off a nightly embedding run by creating a batch with an initial set of inputs, then finalizing it so dedicated batch workers begin processing asynchronously.
   copy

   $ # 1. Create batch with initial requests
   curl -X POST https://api.runpod.io/v2/{endpoint_id}/batch \
     -H 'Authorization: Bearer {api_key}' \
     -H 'Content-Type: application/json' \
     -d '[{"input":{"text":"The quick brown fox"}},{"input":{"text":"Jumped over the lazy dog"}}]'
   
   # 2. Finalize to start processing
   curl -X POST https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/finalize \
     -H 'Authorization: Bearer {api_key}'

   Poll a running batch until all requests have finished, using the request counts rather than a COMPLETED status.
   copy

   $ curl -X GET https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123 \
     -H 'Authorization: Bearer {api_key}'
   # Batch is done when requestCompleted + requestFailed == requestTotal

   Page through completed results after a batch finishes, inspecting outputs and error messages for failed child requests.
   copy

   $ curl -X GET 'https://api.runpod.io/v2/{endpoint_id}/batch/batch_01j9abc123/requests?offset=0&limit=50' \
     -H 'Authorization: Bearer {api_key}'

   • ›New POST /v2/{endpoint_id}/batch endpoint creates a batch job, optionally seeding it with an initial array of requests (each matching the standard /run shape with an input field).
   • ›New POST /v2/{endpoint_id}/batch/{id}/requests endpoint appends additional requests to a DRAFT batch; accepts up to 10 MiB per call and can be called multiple times incrementally.
   • ›New POST /v2/{endpoint_id}/batch/{id}/finalize endpoint locks the batch and makes it eligible for execution, transitioning status to FINALIZED.
   • ›New GET /v2/{endpoint_id}/batch/{id} endpoint returns a batch summary including requestTotal, requestInProgress, requestCompleted, and requestFailed counts for progress polling.
   • ›New GET /v2/{endpoint_id}/batch/{id}/requests endpoint returns paginated child-request results with per-request status, output, error, startedAt, and completedAt fields; supports offset and limit query parameters, with a hasMore field indicating remaining pages.
   +8 moreshow less

   • ›New POST /v2/{endpoint_id}/batch/{id}/cancel endpoint cancels a batch; queued requests are cancelled and not billed, while in-progress requests complete normally.
   • ›New GET /v2/{endpoint_id}/batch endpoint lists all batches for an endpoint, newest first.
   • ›New PUT /v2/{endpoint_id}/batch/{id} endpoint updates batch attributes such as display name.
   • ›New DELETE /v2/{endpoint_id}/batch/{id}/requests/{requestId} endpoint removes a single request from a DRAFT batch.
   • ›Batch jobs run on dedicated workers isolated from standard /run traffic, so submitting a batch never delays interactive requests; limits are 10 active batches per endpoint, 5,000 requests per batch, and 50,000 queued requests per endpoint (up to 1,000,000 daily for enterprise).
   • ›Console Batch tab on each endpoint shows all batches with name, status, and progress counts, plus a detail view with per-request rows, timestamps, and error messages sorted failures-first.
   • ›Webhook and Console Inbox notifications fire once when a batch reaches a terminal state (FAILED or CANCELLED), including batch ID, endpoint name, status, and completed/failed/total counts.
   • ›Batch jobs are billed at the same rate as standard serverless requests; flex worker discounts apply for enterprise customers.
6. New ReleaseREST API v2 Aug 1, 2026 · issue 002

   RunPod REST API v2 is generally available, adding catalog endpoints, pod log streaming, and Serverless observability at a new base URL.

   └──▷ TRY IT
   Fetch the full OpenAPI spec to generate a typed client or validate requests against the v2 API.
   copy

   $ curl --request GET --url https://api.runpod.io/v2/openapi.json

   • ›New base URL https://api.runpod.io/v2 for all REST API v2 requests, with reorganized resource paths and standardized request/response shapes.
   • ›New GET https://api.runpod.io/v2/openapi.json endpoint returns the complete OpenAPI specification for client generation, request validation, and tooling integration.
   • ›New catalog endpoints let you browse available GPU types, CPU types, and data centers programmatically.
   • ›New pod log streaming capability available via REST API v2.
   • ›New Serverless observability features available via REST API v2.
   +2 moreshow less

   • ›New /runpod:migrate command (via the Runpod skills plugin in a coding agent) automatically migrates v1 or GraphQL integrations to v2.
   • ›New ECR Integration (BETA) lets you pull private container images from AWS ECR into Pods and Serverless endpoints without migrating registries or managing credentials.
   └──▷ BREAKING ON UPGRADE

   • !REST API v1 will be retired on November 15, 2026 — existing v1 integrations must be migrated to REST API v2 before that date.
   • !The GraphQL API will be retired in early 2027 — new integrations should use REST API v2.