Part 5 of 15 • 🟡 Intermediate • ⏱️ 120 minutes

Non-Blocking Async Execution in watsonx Orchestrate: Building Background Job Workers for Heavy Tasks

Preventing gateway timeouts by dispatching long-running tasks into asynchronous background workers with job polling.

---

Step 1: Write Tool Logic (async_tools.py)

from ibm_watsonx_orchestrate import tool
import threading
import uuid
import time
from datetime import datetime

# In-memory job registry (use Redis/Postgres in production clusters)
JOBS = {}

@tool
def start_background_job(job_name: str, duration_seconds: int = 15) -> str:
    """
    Start a long-running background task asynchronously without blocking the chat.
    """
    job_id = str(uuid.uuid4())[:8]
    JOBS[job_id] = {
        'id': job_id,
        'name': job_name,
        'status': 'RUNNING',
        'started_at': datetime.now().strftime("%H:%M:%S"),
        'progress': '0%'
    }

    def worker():
        for pct in [25, 50, 75, 100]:
            time.sleep(duration_seconds / 4)
            JOBS[job_id]['progress'] = f"{pct}%"
        JOBS[job_id]['status'] = 'COMPLETED'
        JOBS[job_id]['completed_at'] = datetime.now().strftime("%H:%M:%S")
        JOBS[job_id]['result'] = f"Success: Dataset '{job_name}' processed (14,200 rows indexed)."

    thread = threading.Thread(target=worker, daemon=True)
    thread.start()

    return f"🚀 Background job '{job_name}' dispatched! Job ID: {job_id}. You can continue chatting while it runs."

@tool
def check_job_status(job_id: str) -> str:
    """Check the status and output of a running background job."""
    job = JOBS.get(job_id)
    if not job:
        return f"❌ Job ID '{job_id}' not found."
    
    if job['status'] == 'COMPLETED':
        return f"✅ Job {job_id} [{job['name']}] COMPLETED at {job['completed_at']}: {job['result']}"
    else:
        return f"⏳ Job {job_id} [{job['name']}] is STILL RUNNING (Progress: {job['progress']}). Started at {job['started_at']}."

Step 2: Dependencies (requirements.txt)

ibm-watsonx-orchestrate>=2.9.0

Step 3: Agent Manifest (agent.yaml)

spec_version: v1
kind: native
name: async_job_manager
title: "Async Job Dispatcher"
description: "Dispatches and monitors asynchronous long-running batch operations."
model: ibm/granite-3-8b-instruct
instructions: |
  You are a background operations assistant. When a user asks to run an extensive task,
  dispatch it using `start_background_job` and give them their Job ID.
  When they query status, use `check_job_status`.
tools:
  - start_background_job
  - check_job_status

Step 4: Live Execution

orchestrate chat ask -n async_job_manager "Start data aggregation for Q3 Sales"

╭─ 🤖 async_job_manager ─────────────────────────────────────────────────╮
│                                                                        │
│  🚀 Background job 'Q3 Sales' dispatched! Job ID: 4f8a19de.             │
│  Feel free to ask other questions while the pipeline executes!         │
╰────────────────────────────────────────────────────────────────────────╯

---

💡 Pro Tips & Architecture Insights

---