Introduction

SparkWren is a lightweight data processing and workflow automation platform that lets you connect apps, transform data, and automate repetitive tasks — all without managing infrastructure.

Whether you prefer a visual builder or code-first approach, SparkWren gives you the tools to build reliable automations that scale from prototype to production.

Quick Start

Get up and running in under 5 minutes. Install the CLI, create your first workflow, and deploy it to the cloud.

npm install -g @sparkwren/cli

Initialize a new project:

sparkwren init my-workflow

This creates a project with a sample workflow file, configuration, and test setup.

Core Concepts

SparkWren workflows are built from three core primitives:

Triggers — Events that start your workflow (webhooks, schedules, events, polling).

Steps — Individual actions within a workflow (API calls, transformations, conditions).

Connections — Authenticated links to external services (OAuth, API keys, tokens).

Workflow Definition

Workflows can be defined visually in the UI or as code using our TypeScript/Python SDKs:

import { workflow, trigger } from '@sparkwren/sdk';

Each workflow has a unique name, a trigger, and an ordered list of steps. Steps can reference data from previous steps using template expressions like {{stepName.output.field}}.

Error Handling

SparkWren provides built-in retry logic, dead-letter queues, and error notifications. Configure retry policies per-step or globally:

retries: { maxAttempts: 3, backoff: 'exponential' }

Failed executions are logged with full context for debugging, and you can replay them with a single click.

Deployment

SparkWren supports multiple deployment strategies to fit your team's workflow:

CLI Deploy — Push workflows directly from your terminal with sparkwren deploy.

Git Integration — Connect your repository and auto-deploy on push to main.

CI/CD Pipelines — Use our GitHub Actions, GitLab CI, or Jenkins plugins for automated deployments.

Environment Variables

Manage secrets and configuration across environments:

sparkwren env set API_KEY=sk_live_xxx --env production

Environment variables are encrypted at rest and injected at runtime. They're never exposed in logs or execution history.

Monitoring & Observability

Every workflow execution is tracked with detailed metrics:

Execution logs — Step-by-step output with timing information.

Metrics dashboard — Success rates, latency percentiles, and throughput graphs.

Alerting — Configure alerts for failures, slow executions, or custom conditions.

Integrate with your existing observability stack via OpenTelemetry, Datadog, or custom webhooks.