GCO Cluster Gateway (internal ALB) — API spec sheet¶

Gateway cluster-gateway · OpenAPI 3.1.0 · API version 1.0.0 · 44 endpoints · 12 schemas

The HTTP surface of the internal ALB that fronts a GCO region: the Kubernetes Gateway gco-system/gco-gateway (GatewayClass gco-aws-alb, implemented by gateway.k8s.aws/alb, the AWS Load Balancer Controller) with one HTTPS:443 listener. Its HTTPRoute gco-routes sends each request to a Service by longest path prefix; every operation below is one a backend Service serves and the route delivers to it, tagged with that Service and annotated with the winning rule (x-gco-route). The ALB is internal and answers only the API Gateway proxy Lambdas (over Global Accelerator or from the regional bridge's VPC), which sign every request with the HMAC envelope (x-gco-signature and companions) that the services verify on every path except /api/v1/health, /healthz, /metrics, /readyz. The ALB terminates TLS with a deployment-local certificate and re-encrypts to a TLS proxy sidecar on each pod; the service documents describe the same operations as their applications see them.

• Machine-readable document: docs/openapi/cluster-gateway.json — the document scripts/generate_cluster_gateway_openapi.py composes from the HTTPRoute manifest and the service documents; this sheet is rendered from it
• Interactive console (Swagger UI): https://aws-solutions-library-samples.github.io/global-capacity-orchestrator-on-aws/swagger/cluster-gateway/
• Catalogue index: README.md · interaction diagram

Servers¶

• https://{alb_hostname}

The region's internal ALB, registered in SSM as /<project>/alb-hostname-<region> by ga-registration and kept current by the health monitor; reachable from inside the VPC and from Global Accelerator only. Clients present SNI backend.<project>.gco.internal.

Routing¶

• Gateway: gco-system/gco-gateway (GatewayClass gco-aws-alb, controller gateway.k8s.aws/alb)
• Listener: https HTTPS:443; load balancer scheme internal
• HTTPRoute: gco-routes
• Paths served without the HMAC envelope: /api/v1/health, /healthz, /metrics, /readyz

HTTPRoute rules¶

In document order; the controller awards precedence by path specificity (longest prefix wins), so / is the catch-all.

Load balancer¶

{
  "listenerConfigurations": [
    {
      "defaultCertificate": "{{BACKEND_TLS_CERTIFICATE_ARN}}",
      "protocolPort": "HTTPS:443",
      "sslPolicy": "ELBSecurityPolicy-TLS13-1-2-2021-06"
    }
  ],
  "loadBalancerAttributes": [
    {
      "key": "idle_timeout.timeout_seconds",
      "value": "300"
    }
  ],
  "scheme": "internal",
  "tags": {
    "gco.aws/gateway": "gco-system/gco-gateway"
  }
}

Target group¶

{
  "healthCheck": {
    "healthCheckInterval": 15,
    "healthCheckPath": "/healthz",
    "healthCheckProtocol": "HTTPS",
    "healthCheckTimeout": 5,
    "healthyThresholdCount": 2,
    "unhealthyThresholdCount": 2
  },
  "protocol": "HTTPS",
  "targetGroupAttributes": [
    {
      "key": "deregistration_delay.timeout_seconds",
      "value": "900"
    }
  ],
  "targetType": "ip"
}

Paths a Service serves but never receives¶

A prefix cannot be split between Services, so these implementations are reachable only by addressing the Service inside the cluster.

Services not on the ALB¶

• cost-monitor — reached through manifest-processor (/api/v1/cost*). Confined to manifest-processor traffic by a NetworkPolicy and runs no authentication middleware of its own; the manifest processor's /api/v1/cost/* routes are its authenticated front, reaching it at https://cost-monitor.gco-system.svc.cluster.local:8443 and verifying its certificate against the cluster's internal CA.

Downstream of the backends¶

• inference-proxy → gco-inference/, -canary or -proxy Services. Resolves /inference/<endpoint>/... by endpoint name from the inference endpoint store, requires the endpoint to be running in this region, allows only its serving and health paths, and streams https://<service>.gco-inference.svc.cluster.local:8443/<path> back, verifying the model pod's TLS sidecar against the cluster's internal CA.

Security schemes¶

gcoHmac¶

• Type: apiKey — header x-gco-signature

Request-bound HMAC envelope minted by the API Gateway proxy Lambdas (x-gco-signature-version, x-gco-timestamp, x-gco-nonce, x-gco-signature): the signature covers the method, exact path and query, body digest, timestamp and nonce under the shared signing key from Secrets Manager. Verified by gco/services/auth_middleware.py; a failure is a 403. Original caller identity is authenticated upstream by API Gateway IAM.

Endpoints¶

Endpoint details¶

GET /¶

Root endpoint with basic service information and API overview.

• Operation ID: root__get
• Tags: manifest-processor, Info
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

GET /api/v1/cost/reports¶

List this region's most recent cost report objects in S3.

• Operation ID: list_cost_reports_api_v1_cost_reports_get
• Tags: manifest-processor, Cost
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

POST /api/v1/cost/reports¶

Generate an ad-hoc cost report for the trailing window.

• Operation ID: generate_cost_report_api_v1_cost_reports_post
• Tags: manifest-processor, Cost
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Request body (required): application/json: CostReportRequest

Responses

GET /api/v1/cost/status¶

Cost monitoring status for this region, including OpenCost health.

• Operation ID: get_cost_status_api_v1_cost_status_get
• Tags: manifest-processor, Cost
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

GET /api/v1/health¶

Primary health check endpoint for ALB health checks Returns 200 if cluster is healthy, 503 if unhealthy

• Operation ID: health_check_api_v1_health_get
• Tags: health-monitor
• Security: none (unauthenticated)
• Delivered by rule: /api/v1/health → Service health-monitor:443 in gco-system

Responses

GET /api/v1/jobs¶

List Kubernetes Jobs with pagination and filtering.

• Operation ID: list_jobs_api_v1_jobs_get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

DELETE /api/v1/jobs¶

Bulk delete jobs based on filters.

• Operation ID: bulk_delete_jobs_api_v1_jobs_delete
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Request body (required): application/json: BulkDeleteRequest

Responses

POST /api/v1/jobs/from-template/{name}¶

Create a job from a template with parameter substitution.

• Operation ID: create_job_from_template_api_v1_jobs_from_template__name__post
• Tags: manifest-processor, Templates
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Request body (required): application/json: JobFromTemplateRequest

Responses

GET /api/v1/jobs/{namespace}/{name}¶

Get details of a specific Job, including where its pods were scheduled.

• Operation ID: get_job_api_v1_jobs__namespace___name__get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

DELETE /api/v1/jobs/{namespace}/{name}¶

Delete a Job, optionally requiring its immutable Kubernetes UID.

• Operation ID: delete_job_api_v1_jobs__namespace___name__delete
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/jobs/{namespace}/{name}/events¶

Get events related to a Job.

• Operation ID: get_job_events_api_v1_jobs__namespace___name__events_get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/jobs/{namespace}/{name}/logs¶

Get logs from a Job's pods.

• Operation ID: get_job_logs_api_v1_jobs__namespace___name__logs_get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/jobs/{namespace}/{name}/metrics¶

Get resource usage metrics for a Job's pods.

• Operation ID: get_job_metrics_api_v1_jobs__namespace___name__metrics_get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/jobs/{namespace}/{name}/pods¶

Get pods belonging to a Job, with the hardware each one landed on.

• Operation ID: get_job_pods_api_v1_jobs__namespace___name__pods_get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/jobs/{namespace}/{name}/pods/{pod_name}/logs¶

Get logs from a specific pod belonging to a Job.

• Operation ID: get_pod_logs_api_v1_jobs__namespace___name__pods__pod_name__logs_get
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

POST /api/v1/jobs/{namespace}/{name}/retry¶

Retry a failed job by creating a new job from its spec.

• Operation ID: retry_job_api_v1_jobs__namespace___name__retry_post
• Tags: manifest-processor, Jobs
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

POST /api/v1/manifests¶

Submit Kubernetes manifests for processing.

• Operation ID: submit_manifests_api_v1_manifests_post
• Tags: manifest-processor, Manifests
• Security: gcoHmac
• Delivered by rule: /api/v1/manifests → Service manifest-processor:443 in gco-system

Request body (required): application/json: ManifestSubmissionAPIRequest

Responses

POST /api/v1/manifests/validate¶

Validate manifests without applying them.

• Operation ID: validate_manifests_api_v1_manifests_validate_post
• Tags: manifest-processor, Manifests
• Security: gcoHmac
• Delivered by rule: /api/v1/manifests → Service manifest-processor:443 in gco-system

Request body (required): application/json: ManifestSubmissionAPIRequest

Responses

GET /api/v1/manifests/{namespace}/{name}¶

Get the status of a specific resource.

• Operation ID: get_resource_status_api_v1_manifests__namespace___name__get
• Tags: manifest-processor, Manifests
• Security: gcoHmac
• Delivered by rule: /api/v1/manifests → Service manifest-processor:443 in gco-system

Parameters

Responses

DELETE /api/v1/manifests/{namespace}/{name}¶

Delete a specific resource from the cluster.

• Operation ID: delete_resource_api_v1_manifests__namespace___name__delete
• Tags: manifest-processor, Manifests
• Security: gcoHmac
• Delivered by rule: /api/v1/manifests → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/metrics¶

Detailed metrics endpoint with resource utilization information

• Operation ID: get_metrics_api_v1_metrics_get
• Tags: health-monitor
• Security: gcoHmac
• Delivered by rule: /api/v1/metrics → Service health-monitor:443 in gco-system

Responses

GET /api/v1/policy¶

The validation policy this region actually enforces, as deployed.

Answers "will this cluster admit the job I am about to pay to run?" before submission, so a policy conflict surfaces at plan time instead of after a region has been provisioned and billed.

Reads the live ManifestProcessor instance rather than any config file. A local cdk.json is the input to a deploy, not the state of one: the cluster may have been deployed from a different checkout, and CDK augments trusted_registries with the project's own ECR hostnames at synth time, so the effective allowlist is strictly larger than the configured one.

Three layers govern admission and all three are reported:

1. policy — the front-door checks the manifest processor and the SQS queue processor both apply (they read the same env vars, so neither submission path is a bypass).
2. cluster_enforcement.limit_ranges — per-container ceilings.
3. cluster_enforcement.resource_quotas — namespace aggregate ceilings.

A manifest must clear all three. Layers 2 and 3 are read live from the Kubernetes API and degrade to status="unavailable" rather than failing the whole response.

• Operation ID: get_job_validation_policy_api_v1_policy_get
• Tags: manifest-processor, Health
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

GET /api/v1/queue/jobs¶

List one bounded page of jobs with optional filters.

• Operation ID: list_queued_jobs_api_v1_queue_jobs_get
• Tags: manifest-processor, Job Queue
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

POST /api/v1/queue/jobs¶

Submit one validated batch/v1 Job exactly once.

• Operation ID: submit_job_to_queue_api_v1_queue_jobs_post
• Tags: manifest-processor, Job Queue
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Request body (required): application/json: QueuedJobRequest

Responses

GET /api/v1/queue/jobs/{job_id}¶

Get details of a specific queued job.

• Operation ID: get_queued_job_api_v1_queue_jobs__job_id__get
• Tags: manifest-processor, Job Queue
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

DELETE /api/v1/queue/jobs/{job_id}¶

Cancel a job only while it remains unclaimed in the queue.

• Operation ID: cancel_queued_job_api_v1_queue_jobs__job_id__delete
• Tags: manifest-processor, Job Queue
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

POST /api/v1/queue/poll¶

Run one immediate queue-worker pass for this region.

The manifest API also runs this same processor continuously when the deployment enables CENTRAL_QUEUE_WORKER_ENABLED. This endpoint remains useful for an authenticated operator-triggered pass and diagnostics.

• Operation ID: poll_and_process_jobs_api_v1_queue_poll_post
• Tags: manifest-processor, Job Queue
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/queue/stats¶

Get job queue statistics by region and status.

• Operation ID: get_queue_stats_api_v1_queue_stats_get
• Tags: manifest-processor, Job Queue
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

GET /api/v1/status¶

Service status endpoint with detailed information.

• Operation ID: get_service_status_api_v1_status_get
• Tags: manifest-processor, Health
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

GET /api/v1/templates¶

List all job templates.

• Operation ID: list_templates_api_v1_templates_get
• Tags: manifest-processor, Templates
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

POST /api/v1/templates¶

Create a new job template.

• Operation ID: create_template_api_v1_templates_post
• Tags: manifest-processor, Templates
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Request body (required): application/json: JobTemplateRequest

Responses

GET /api/v1/templates/{name}¶

Get a specific job template.

• Operation ID: get_template_api_v1_templates__name__get
• Tags: manifest-processor, Templates
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

DELETE /api/v1/templates/{name}¶

Delete a job template.

• Operation ID: delete_template_api_v1_templates__name__delete
• Tags: manifest-processor, Templates
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /api/v1/webhooks¶

List all registered webhooks.

• Operation ID: list_webhooks_api_v1_webhooks_get
• Tags: manifest-processor, Webhooks
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

POST /api/v1/webhooks¶

Register a new webhook for job events.

• Operation ID: create_webhook_api_v1_webhooks_post
• Tags: manifest-processor, Webhooks
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Request body (required): application/json: WebhookRequest

Responses

DELETE /api/v1/webhooks/{webhook_id}¶

Delete a webhook.

• Operation ID: delete_webhook_api_v1_webhooks__webhook_id__delete
• Tags: manifest-processor, Webhooks
• Security: gcoHmac
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Parameters

Responses

GET /healthz¶

Kubernetes-style health check endpoint Simple endpoint that returns 200 if the service is running

• Operation ID: kubernetes_health_check_healthz_get
• Tags: health-monitor
• Security: none (unauthenticated)
• Delivered by rule: /healthz → Service health-monitor:443 in gco-system

Responses

GET /inference/{endpoint_name}¶

Proxy an endpoint-root request after platform authentication.

• Operation ID: proxy_inference_root_inference__endpoint_name__get
• Tags: inference-proxy, Inference
• Security: gcoHmac
• Delivered by rule: /inference → Service inference-proxy:443 in gco-system

Parameters

Responses

POST /inference/{endpoint_name}¶

Proxy an endpoint-root request after platform authentication.

• Operation ID: proxy_inference_root_inference__endpoint_name__post
• Tags: inference-proxy, Inference
• Security: gcoHmac
• Delivered by rule: /inference → Service inference-proxy:443 in gco-system

Parameters

Responses

HEAD /inference/{endpoint_name}¶

Proxy an endpoint-root request after platform authentication.

• Operation ID: proxy_inference_root_inference__endpoint_name__head
• Tags: inference-proxy, Inference
• Security: gcoHmac
• Delivered by rule: /inference → Service inference-proxy:443 in gco-system

Parameters

Responses

GET /inference/{endpoint_name}/{upstream_path}¶

Proxy an endpoint sub-path after platform authentication.

• Operation ID: proxy_inference_path_inference__endpoint_name___upstream_path__get
• Tags: inference-proxy, Inference
• Security: gcoHmac
• Delivered by rule: /inference → Service inference-proxy:443 in gco-system

Parameters

Responses

POST /inference/{endpoint_name}/{upstream_path}¶

Proxy an endpoint sub-path after platform authentication.

• Operation ID: proxy_inference_path_inference__endpoint_name___upstream_path__post
• Tags: inference-proxy, Inference
• Security: gcoHmac
• Delivered by rule: /inference → Service inference-proxy:443 in gco-system

Parameters

Responses

HEAD /inference/{endpoint_name}/{upstream_path}¶

Proxy an endpoint sub-path after platform authentication.

• Operation ID: proxy_inference_path_inference__endpoint_name___upstream_path__head
• Tags: inference-proxy, Inference
• Security: gcoHmac
• Delivered by rule: /inference → Service inference-proxy:443 in gco-system

Parameters

Responses

GET /readyz¶

Kubernetes readiness includes the enabled queue worker task.

• Operation ID: kubernetes_readiness_check_readyz_get
• Tags: manifest-processor, Health
• Security: none (unauthenticated)
• Delivered by rule: / → Service manifest-processor:443 in gco-system

Responses

Schemas¶

AdhocReportRequest¶

Request body for POST /internal/reports.

BulkDeleteRequest¶

Example:

{
  "dry_run": false,
  "namespace": "gco-jobs",
  "older_than_days": 7,
  "status": "completed"
}

CostReportRequest¶

Request body for POST /api/v1/cost/reports.

HTTPValidationError¶

JobFromTemplateRequest¶

Example:

{
  "name": "my-training-job",
  "namespace": "gco-jobs",
  "parameters": {
    "image": "my-custom-image:v1"
  }
}

JobStatus¶

• Type: enum: "pending", "running", "completed", "succeeded", "failed"

JobTemplateRequest¶

Example:

{
  "description": "Template for GPU training jobs",
  "manifest": {
    "apiVersion": "batch/v1",
    "kind": "Job",
    "metadata": {
      "name": "{{name}}"
    }
  },
  "name": "gpu-training-template",
  "parameters": {
    "image": "pytorch/pytorch:latest"
  }
}

ManifestSubmissionAPIRequest¶

API model for manifest submission requests.

Example:

{
  "dry_run": false,
  "manifests": [
    {
      "apiVersion": "batch/v1",
      "kind": "Job",
      "metadata": {
        "name": "example"
      }
    }
  ],
  "namespace": "gco-jobs"
}

QueuedJobRequest¶

Example:

{
  "manifest": {
    "apiVersion": "batch/v1",
    "kind": "Job",
    "metadata": {
      "name": "my-training-job"
    }
  },
  "max_spot_price": 0.5,
  "namespace": "gco-jobs",
  "priority": 10,
  "spot_instance_type": "g5.xlarge",
  "target_region": "us-east-1"
}

ValidationError¶

WebhookEvent¶

• Type: enum: "job.completed", "job.failed", "job.started"

WebhookRequest¶

Example:

{
  "events": [
    "job.completed",
    "job.failed"
  ],
  "namespace": "gco-jobs",
  "url": "https://example.com/webhook"
}