Pipeline v2 Recipes

RAPS Pipeline v2 introduces retry logic, parallel execution, conditional steps, for-each iteration, and expression evaluation. These features let you build robust, production-grade automation pipelines that handle failures gracefully, process files in bulk, and branch based on runtime conditions.

Feature Overview

Retry

Retry automatically re-executes a failed step up to a configurable number of attempts. This is essential for handling transient network errors, API rate limits, and temporary service unavailability.

Basic Retry

# raps-pipeline.yaml
name: upload-with-retry
steps:
  - name: upload-model
    action: object_upload
    params:
      bucket: project-models
      file: building.rvt
    retry:
      max_attempts: 3
      delay: "5s"

Exponential Backoff

For API rate limiting, use exponential backoff to progressively increase the delay between retries:

steps:
  - name: translate-model
    action: translate_start
    params:
      urn: "${{ steps.upload.outputs.urn }}"
      format: svf2
    retry:
      max_attempts: 5
      delay: "2s"
      backoff: "exponential"

With exponential backoff, the delays are 2s, 4s, 8s, 16s, 32s for the five attempts.

Retry with Timeout

Combine retry with a per-step timeout to cap total execution time:

steps:
  - name: translate-and-wait
    action: translate_start
    params:
      urn: "${{ steps.upload.outputs.urn }}"
      format: svf2
      wait: true
    timeout: "10m"
    retry:
      max_attempts: 3
      delay: "10s"
      backoff: "exponential"

If a single attempt exceeds 10 minutes, it is terminated and the next retry begins (subject to the max_attempts limit).

Run It

# Execute with retry logging visible
raps pipeline run raps-pipeline.yaml --verbose

# Dry run to validate retry configuration
raps pipeline run raps-pipeline.yaml --dry-run

Timeout

Set a maximum duration for any step. If the step does not complete within the timeout window, it is cancelled and marked as failed (which may trigger a retry if configured).

Per-Step Timeouts

steps:
  - name: quick-upload
    action: object_upload
    params:
      bucket: fast-bucket
      file: small-model.dwg
    timeout: "2m"

  - name: long-translation
    action: translate_start
    params:
      urn: "${{ steps.quick-upload.outputs.urn }}"
      format: svf2
      wait: true
    timeout: "30m"

Timeout Format

Always include the unit suffix. Bare numbers like "30" are not valid.

Parallel Execution

Run multiple independent steps concurrently to reduce total pipeline duration. Parallel steps start at the same time and the pipeline waits for all of them to complete before proceeding.

Basic Parallel

steps:
  - name: create-bucket
    action: bucket_create
    params:
      key: multi-discipline
      policy: persistent

  - name: upload-all
    parallel:
      - name: upload-arch
        action: object_upload
        params:
          bucket: multi-discipline
          file: architectural.rvt
      - name: upload-struct
        action: object_upload
        params:
          bucket: multi-discipline
          file: structural.rvt
      - name: upload-mep
        action: object_upload
        params:
          bucket: multi-discipline
          file: mep.rvt

Parallel Flow

Parallel Translations with Retry

Combine parallel execution with retry for resilient batch processing:

steps:
  - name: translate-all
    parallel:
      - name: translate-arch
        action: translate_start
        params:
          urn: "${{ steps.upload-arch.outputs.urn }}"
          format: svf2
          wait: true
        timeout: "15m"
        retry:
          max_attempts: 2
          delay: "10s"
      - name: translate-struct
        action: translate_start
        params:
          urn: "${{ steps.upload-struct.outputs.urn }}"
          format: svf2
          wait: true
        timeout: "15m"
        retry:
          max_attempts: 2
          delay: "10s"
      - name: translate-mep
        action: translate_start
        params:
          urn: "${{ steps.upload-mep.outputs.urn }}"
          format: svf2
          wait: true
        timeout: "15m"
        retry:
          max_attempts: 2
          delay: "10s"

Run It

# Parallel steps show interleaved output
raps pipeline run raps-pipeline.yaml --verbose

# Check which parallel step is slowest
raps pipeline run raps-pipeline.yaml --timing

Conditionals

Execute steps only when a condition evaluates to true. Conditions can reference environment variables, step outputs, and step statuses.

Branch-Based Conditions

Deploy only on the main branch:

steps:
  - name: upload-model
    action: object_upload
    params:
      bucket: staging-models
      file: building.rvt

  - name: promote-to-production
    action: object_copy
    params:
      source_bucket: staging-models
      target_bucket: production-models
      key: building.rvt
    when: "env.BRANCH == 'main'"

Status-Based Conditions

Run a cleanup step only after a successful upload:

steps:
  - name: upload
    action: object_upload
    params:
      bucket: project-models
      file: building.rvt

  - name: translate
    action: translate_start
    params:
      urn: "${{ steps.upload.outputs.urn }}"
      format: svf2
    when: "steps.upload.status == 'success'"

  - name: notify-failure
    action: webhook_send
    params:
      url: "${{ env.SLACK_WEBHOOK }}"
      message: "Upload failed for building.rvt"
    when: "steps.upload.status == 'failed'"

Conditional Flow

Output-Based Conditions

Use outputs from a previous step to decide what to do next:

steps:
  - name: check-bucket
    action: bucket_details
    params:
      key: project-models

  - name: create-bucket
    action: bucket_create
    params:
      key: project-models
      policy: persistent
    when: "steps.check-bucket.status == 'failed'"

For-Each

Iterate over a list of values to execute a step multiple times. The for_each field accepts a comma-separated string or a reference to an environment variable containing one.

Iterate Over Files

env:
  FILES: "arch.rvt,struct.rvt,mep.rvt,site.dwg"

steps:
  - name: upload-files
    action: object_upload
    params:
      bucket: project-models
      file: "${{ each.value }}"
    for_each: "env.FILES"
    retry:
      max_attempts: 2
      delay: "3s"

This expands to four upload operations, one per file. Each iteration has access to each.value (the current item) and each.index (the zero-based position).

For-Each with Translation

Upload and translate a dynamic list of models:

env:
  MODELS: "building-a.rvt,building-b.rvt,building-c.rvt"

steps:
  - name: upload-models
    action: object_upload
    params:
      bucket: batch-project
      file: "./models/${{ each.value }}"
    for_each: "env.MODELS"

  - name: translate-models
    action: translate_start
    params:
      urn: "${{ steps.upload-models.outputs[each.index].urn }}"
      format: svf2
      wait: true
    for_each: "env.MODELS"
    timeout: "20m"
    retry:
      max_attempts: 2
      delay: "5s"
      backoff: "exponential"

For-Each with Conditionals

Process files but only translate Revit formats:

env:
  FILES: "model.rvt,drawing.dwg,spec.pdf,layout.rvt"

steps:
  - name: upload-all
    action: object_upload
    params:
      bucket: mixed-assets
      file: "${{ each.value }}"
    for_each: "env.FILES"

  - name: translate-revit
    action: translate_start
    params:
      urn: "${{ steps.upload-all.outputs[each.index].urn }}"
      format: svf2
    for_each: "env.FILES"
    when: "each.value ends_with '.rvt'"

Run It

# For-each iterations show per-item progress
raps pipeline run raps-pipeline.yaml --verbose

# Override the file list at runtime
raps pipeline run raps-pipeline.yaml --set FILES="new-a.rvt,new-b.rvt"

Expressions

Expressions use the ${{ }} syntax to reference environment variables, step outputs, and built-in functions within pipeline YAML files.

Environment Variables

env:
  BUCKET_PREFIX: "myproject"
  REGION: "US"

steps:
  - name: create-bucket
    action: bucket_create
    params:
      key: "${{ env.BUCKET_PREFIX }}-models"
      policy: persistent
      region: "${{ env.REGION }}"

Step Outputs

Reference outputs from previous steps:

steps:
  - name: upload
    action: object_upload
    params:
      bucket: project-models
      file: building.rvt

  - name: translate
    action: translate_start
    params:
      urn: "${{ steps.upload.outputs.urn }}"
      format: svf2

  - name: get-metadata
    action: derivative_metadata
    params:
      urn: "${{ steps.upload.outputs.urn }}"
    when: "steps.translate.status == 'success'"

Built-In Variables

Full Example: Production Pipeline

A complete pipeline that combines all v2 features into a real-world multi-model processing workflow:

name: multi-model-processing
env:
  BUCKET: "prod-models-2026"
  MODELS: "arch.rvt,struct.rvt,mep-hvac.rvt,mep-elec.rvt"
  SLACK_WEBHOOK: "${{ env.SLACK_WEBHOOK_URL }}"

steps:
  # Step 1: Ensure the bucket exists
  - name: ensure-bucket
    action: bucket_create
    params:
      key: "${{ env.BUCKET }}"
      policy: persistent
      region: US
    retry:
      max_attempts: 2
      delay: "3s"

  # Step 2: Upload all models in parallel via for-each
  - name: upload-models
    action: object_upload
    params:
      bucket: "${{ env.BUCKET }}"
      file: "./models/${{ each.value }}"
    for_each: "env.MODELS"
    retry:
      max_attempts: 3
      delay: "5s"
      backoff: "exponential"
    timeout: "5m"

  # Step 3: Translate all models in parallel
  - name: translate-models
    parallel:
      - name: translate-arch
        action: translate_start
        params:
          urn: "${{ steps.upload-models.outputs[0].urn }}"
          format: svf2
          wait: true
        timeout: "20m"
        retry:
          max_attempts: 2
          delay: "10s"
      - name: translate-struct
        action: translate_start
        params:
          urn: "${{ steps.upload-models.outputs[1].urn }}"
          format: svf2
          wait: true
        timeout: "20m"
        retry:
          max_attempts: 2
          delay: "10s"
      - name: translate-mep-hvac
        action: translate_start
        params:
          urn: "${{ steps.upload-models.outputs[2].urn }}"
          format: svf2
          wait: true
        timeout: "20m"
        retry:
          max_attempts: 2
          delay: "10s"
      - name: translate-mep-elec
        action: translate_start
        params:
          urn: "${{ steps.upload-models.outputs[3].urn }}"
          format: svf2
          wait: true
        timeout: "20m"
        retry:
          max_attempts: 2
          delay: "10s"

  # Step 4: Verify all translations (conditional)
  - name: verify-translations
    action: translate_manifest
    params:
      urn: "${{ steps.upload-models.outputs[each.index].urn }}"
    for_each: "env.MODELS"
    when: "steps.translate-models.status == 'success'"

  # Step 5: Notify on failure
  - name: notify-failure
    action: webhook_send
    params:
      url: "${{ env.SLACK_WEBHOOK }}"
      message: "Pipeline failed at step: ${{ pipeline.name }}"
    when: "steps.translate-models.status == 'failed'"

Run the Full Pipeline

# Production run
raps pipeline run multi-model-pipeline.yaml

# Dry run first
raps pipeline run multi-model-pipeline.yaml --dry-run

# Override models list
raps pipeline run multi-model-pipeline.yaml \
  --set MODELS="custom-a.rvt,custom-b.rvt"

# With verbose logging to see retry attempts and parallel progress
raps pipeline run multi-model-pipeline.yaml --verbose

Troubleshooting

Retry exhaustion

If all retry attempts fail, the step is marked as failed and subsequent conditional steps that depend on its success are skipped. Check the verbose output to see individual attempt errors:

raps pipeline run raps-pipeline.yaml --verbose 2>&1 | grep "retry"

Parallel step failures

By default, if one parallel step fails, the other parallel steps continue to completion. The overall parallel group is marked as failed if any member fails. To fail fast (cancel siblings on first failure), add fail_fast: true:

  - name: upload-all
    parallel:
      fail_fast: true
      steps:
        - name: upload-a
          action: object_upload
          params:
            bucket: project
            file: a.rvt
        - name: upload-b
          action: object_upload
          params:
            bucket: project
            file: b.rvt

Expression syntax errors

Expressions must be wrapped in double quotes in YAML. A common mistake is forgetting the quotes:

# Wrong - YAML will parse this as a mapping
urn: ${{ steps.upload.outputs.urn }}

# Correct
urn: "${{ steps.upload.outputs.urn }}"

For-each with empty list

If the for_each value resolves to an empty string, the step is skipped entirely and marked as skipped. This is not an error.

Next Steps

• MCP & AI Workflows — Use AI assistants to run pipelines interactively
• Security & Authentication — Token management for pipeline execution
• CI/CD with GitHub Actions — Run pipelines in GitHub Actions
• CI/CD with GitLab CI — Run pipelines in GitLab CI