AI-native scientific workflows.

Ginkgo runs scientific workflows in plain Python. Each task is a function. Ginkgo links the tasks into a dynamic graph, running independent tasks in parallel and skipping work that's been done before. Built from the ground up to be written, executed and inspected by AI agents.

Read the docs
curl -LsSf https://raw.githubusercontent.com/sanjaynagi/ginkgo/main/install.sh | sh

How a workflow grows

Ginkgo plans the work. Calling tasks builds a graph. Ginkgo checks the planned graph before anything runs.

Tasks run as soon as their inputs are ready. Independent tasks run in parallel once their inputs exist.

The graph is dynamic and can change as it runs. A task can look at its inputs and add new tasks. The work then follows the data.

Outputs are cached by content, and queryable. Run the workflow again and only the changed tasks run. A SQL ledger stores history and provenance.

Why Ginkgo

Plain Python

You write workflows as ordinary Python functions, with no separate workflow language to learn.

Dynamic workflows

A task can look at its inputs and add new tasks to the task graph. The workflow takes the shape the data needs.

Built for agents

Ginkgo provides tooling to allow AI agents to run, monitor and inspect workflows on their own.

Read more

Plain Python, from the first task to the last

You don't need to learn a workflow language. Tasks are decorated functions, and a flow is a function that calls them. 

from ginkgo import file, flow, shell, task

samples = ["S01", "S02", "S03"]


@task(kind="shell", env="seqkit")
def clean(sample: str, assembly: file) -> file:
    out = f"results/{sample}.fasta"
    cmd = f"seqkit seq -m 200 {assembly} -o {out}"
    return shell(cmd=cmd, output=out)


@task()
def count_contigs(assemblies: list[file]) -> dict[str, int]:
    return {str(a): open(a).read().count(">") for a in assemblies}


@flow
def main():
    paths = [f"data/{s}.fasta" for s in samples]
    cleaned = clean().map(sample=samples, assembly=paths)
    return count_contigs(assemblies=cleaned)
  • Calling a task inside a flow returns a node in the graph, not a result. Ginkgo builds and checks the planned graph before anything runs.
  • .map() runs the task once for each sample, and the runs go in parallel.
  • env="seqkit" runs the command in its own Pixi environment, which Ginkgo installs the first time it's needed.
  • Run the flow again and a task reruns only if its code, inputs or environment changed.

What Ginkgo takes care of

Caching by content

Ginkgo saves each result under the task's code, inputs and environment. Change one task and it runs again. Tasks after it run again only if its output changed.

An environment per task

Shell tasks run in their own Pixi environment or container image, named on the task.

Provenance you can query

Ginkgo records every run. You can ask which task made a file or why a task ran again, or query the record with SQL.

Your laptop, then a cluster

Workflows run locally by default. With an executor plugin, the same flow can send tasks to Kubernetes or GCP Batch.

Watch a bacterial GWAS run

The example is a bacterial genome-wide association study (GWAS) on 60 assemblies. The video shows the first run, the cache, a quality-control change, provenance, resource use, and recovery from a broken file.