Batches

Create and run large groups of requests asynchronously. Batches are only available for OpenAI models.

Create batch

post

Create and immediately start a batch from a file of requests that was uploaded beforehand.

Batches are only available for OpenAI models.

Authorizations

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Body

input_file_idstringRequired

The ID of an uploaded file that contains requests for the new batch.

Your input file must be formatted as a JSONL file, and must be uploaded with the purpose set to "batch". The file can contain up to 50,000 requests, and can be up to 200 MB in size.

Example: file-abc123

endpointstring · enumRequired

The endpoint to be used for all requests in the batch. Currently /v1/chat/completions, /v1/embeddings, and /v1/completions are supported.

Possible values:

completion_windowstring · enumRequired

The time frame within which the batch should be processed. Currently only "24h" is supported.

Default: 24hExample: 24hPossible values:

Responses

200

Batch retrieval response

application/json

A batch object representing a collection of requests processed together.

idstringRequired

The unique identifier of the batch.

Example: batch_abc123

objectstring · enumRequired

The object type, which is always "batch".

Example: batchPossible values:

endpointstring · enumRequired

The endpoint used for the batch requests.

Example: /v1/chat/completionsPossible values:

input_file_idstringRequired

The ID of the input file used for the batch.

Example: file-abc123

completion_windowstringRequired

The time frame within which the batch should be processed.

Example: 24h

statusstring · enumRequired

The current status of the batch.

Example: completedPossible values:

output_file_idstring · nullableOptional

The ID of the file containing the outputs of the batch requests. This is only available when the batch status is "completed".

Example: file-xyz789

error_file_idstring · nullableOptional

The ID of the file containing the errors of the batch requests. This is only available when there are errors in the batch.

Example: file-error123

created_atintegerRequired

The Unix timestamp (in seconds) for when the batch was created.

Example: 1714508400

in_progress_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch started processing.

Example: 1714508460

expires_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch will expire.

Example: 1714594800

finalizing_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch started finalizing.

Example: 1714510200

completed_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch was completed.

Example: 1714510260

failed_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch failed.

Example: 1714510300

expired_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch expired.

Example: 1714594800

cancelling_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch cancellation was initiated.

Example: 1714509000

cancelled_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch was cancelled.

Example: 1714509060

400

Invalid request data.

500

Server error.

post/v1/batches

POST /v1/batches HTTP/1.1
Host: api.nexos.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 91

{
  "input_file_id": "file-abc123",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h"
}

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": {
    "object": "list",
    "data": [
      {
        "code": "invalid_request",
        "message": "The request was malformed",
        "param": "messages",
        "line": 42
      }
    ]
  },
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "file-xyz789",
  "error_file_id": "file-error123",
  "created_at": 1714508400,
  "in_progress_at": 1714508460,
  "expires_at": 1714594800,
  "finalizing_at": 1714510200,
  "completed_at": 1714510260,
  "failed_at": 1714510300,
  "expired_at": 1714594800,
  "cancelling_at": 1714509000,
  "cancelled_at": 1714509060,
  "request_counts": {
    "total": 1000,
    "completed": 950,
    "failed": 50
  },
  "metadata": {
    "customer_id": "user-456",
    "batch_description": "nightly-eval"
  }
}

Retrieve batch

get

Look up a single batch by its ID, including its current status and metadata.

Authorizations

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Path parameters

batch_idstringRequired

The ID of the batch to retrieve

Example: batch_abc123

Query parameters

modelstringOptionalDeprecated

Responses

200

Batch retrieval response

application/json

A batch object representing a collection of requests processed together.

idstringRequired

The unique identifier of the batch.

Example: batch_abc123

objectstring · enumRequired

The object type, which is always "batch".

Example: batchPossible values:

endpointstring · enumRequired

The endpoint used for the batch requests.

Example: /v1/chat/completionsPossible values:

input_file_idstringRequired

The ID of the input file used for the batch.

Example: file-abc123

completion_windowstringRequired

The time frame within which the batch should be processed.

Example: 24h

statusstring · enumRequired

The current status of the batch.

Example: completedPossible values:

output_file_idstring · nullableOptional

The ID of the file containing the outputs of the batch requests. This is only available when the batch status is "completed".

Example: file-xyz789

error_file_idstring · nullableOptional

The ID of the file containing the errors of the batch requests. This is only available when there are errors in the batch.

Example: file-error123

created_atintegerRequired

The Unix timestamp (in seconds) for when the batch was created.

Example: 1714508400

in_progress_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch started processing.

Example: 1714508460

expires_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch will expire.

Example: 1714594800

finalizing_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch started finalizing.

Example: 1714510200

completed_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch was completed.

Example: 1714510260

failed_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch failed.

Example: 1714510300

expired_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch expired.

Example: 1714594800

cancelling_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch cancellation was initiated.

Example: 1714509000

cancelled_atinteger · nullableOptional

The Unix timestamp (in seconds) for when the batch was cancelled.

Example: 1714509060

400

Invalid batch ID format or malformed request

404

Batch not found

500

Internal server error

get/v1/batches/{batch_id}

GET /v1/batches/{batch_id} HTTP/1.1
Host: api.nexos.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": {
    "object": "list",
    "data": [
      {
        "code": "invalid_request",
        "message": "The request was malformed",
        "param": "messages",
        "line": 42
      }
    ]
  },
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "file-xyz789",
  "error_file_id": "file-error123",
  "created_at": 1714508400,
  "in_progress_at": 1714508460,
  "expires_at": 1714594800,
  "finalizing_at": 1714510200,
  "completed_at": 1714510260,
  "failed_at": 1714510300,
  "expired_at": 1714594800,
  "cancelling_at": 1714509000,
  "cancelled_at": 1714509060,
  "request_counts": {
    "total": 1000,
    "completed": 950,
    "failed": 50
  },
  "metadata": {
    "customer_id": "user-456",
    "batch_description": "nightly-eval"
  }
}