T — A Language for Tabular Data and Human–LLM Collaboration

T is an experimental programming language for declarative, functional manipulation of tabular data. Inspired by R’s tidyverse and OCaml’s semantic rigor, T is designed to make data analysis explicit, inspectable, and pipeline-oriented.

Unlike traditional scripting languages, T is built from the ground up to support human–LLM collaborative programming, where humans specify intent and constraints, and language tools (including LLMs) generate localized, mechanical code.

Status: Pre-alpha. Actively designed and implemented.

Documentation

• Language Overview — types, syntax, functions, and standard library
• Data Manipulation Examples — practical examples with core data verbs
• Pipeline Tutorial — step-by-step guide to T's pipeline model

Design Goals

• Data analysis as explicit pipelines
• First-class tabular data (DataFrame-centric)
• Expression-oriented, functional style
• Explicit semantics (no hidden rules, no implicit NA propagation)
• Minimal OCaml core, extensible via packages
• Deterministic execution and inspectable errors
• REPL-first exploratory workflow
• LLM-friendly structure and tooling

LLM-Native by Design

T treats large language models as first-class collaborators, not magic code generators. The language and tooling are designed to make LLM-generated code:

• Local rather than global
• Constrained rather than free-form
• Inspectable rather than opaque
• Correctable rather than brittle

Humans define intent, assumptions, and invariants. LLMs generate localized code. T enforces semantics and correctness.

Intent Blocks

T supports intent blocks: structured comments that encode analytical goals, assumptions, and checks in a machine-readable way.

-- intent:
-- goal: "Estimate approval as a function of age and income"
-- assumptions:
--   - age is approximately linear
--   - missing income is non-random
-- checks:
--   - no negative income
--   - at least 100 observations per group
  

Intent blocks are preserved by tooling, version-controlled with code, and used as stable regeneration boundaries for LLM-assisted workflows.

Pipelines

Pipelines are T’s core execution model. Each pipeline is a DAG of named nodes with explicit dependencies, cacheable results, and inspectable outputs.

pipeline analysis {
  raw = { read_csv("data.csv") }

  cleaned = {
    raw |> filter(age > 18)
        |> mutate(income_k = income / 1000)
  }

  model = {
    cleaned |> lm(approval ~ age + income_k)
  }
}
  

Pipelines enable local reasoning, reproducibility, and safe regeneration of individual steps without rewriting entire scripts.

Language Features

• R-style lambdas: \(x) x + 1
• Pipe operator: |>
• Python-style comprehensions: [x * x for x in xs if x > 2]
• Python-style dictionaries: {name: "Alice", age: 30}
• Errors as values, not exceptions
• Explicit, typed missing values

Numerical Backend

T’s numerical stack is layered:

• Tabular layer: Apache Arrow for columnar data and interoperability
• Compute layer: Owl for linear algebra, optimization, and statistics
• Fallback layer: Selective C bindings (e.g. LAPACK, GSL) when needed

This approach prioritizes fast development, explicit semantics, and safe defaults, while leaving room for future performance upgrades.

Standard Packages

• core: functional utilities
• stats: statistical primitives
• colcraft: DataFrame operations

Packages are part of the standard library and loaded by default. Each function lives in its own file.

Alpha Roadmap

The alpha version of T targets a complete, end-to-end workflow:

• Stable core syntax and interpreter
• Arrow-backed DataFrames
• DAG-based pipelines with caching
• Core data verbs (select, filter, group_by, summarize)
• Basic statistics and modeling
• Intent blocks and tooling hooks
• REPL and CLI

Performance tuning, GPUs, and distributed execution are explicitly out of scope for alpha.

Project Structure

.
├── flake.nix
├── ast.ml
├── parser.ml
├── lexer.ml
├── eval.ml
├── repl.ml
├── pipeline.ml
├── dataframe.ml
└── packages/
    ├── core/
    ├── stats/
    └── colcraft/
  

Building

nix develop
t repl
  

Contributing

Contributions focus on clarity, explicit semantics, and small, reviewable changes. Packages live in-repo during early development.

License: EUPL v1.2.