# Builds

## Builds

A build refers to the process of running a transformation and producing an output dataset or artifacts.

### Regular Build

A build can be triggered by selecting one or multiple transformations in the lineage page and clicking on the `Build` button.

<figure><img src="/files/urTf66zXuHQ3w9QlJCqM" alt=""><figcaption></figcaption></figure>

> ***Note*** When multiple transformations are selected, the build system resolved dependencies and chooses the most efficient build sequence.

### Upstream Build

When there are many upstream transformations, it can sometimes be beneficial to make use of the Upstream Build functionality, which, given a selected transformation, recursively crawls all upstream dependencies and builds them before building the current transformation.

### Draft Build

To enable rapid iteration during development, it is undesirable to have to commit every time we want to test our changes and build our dataset with new code.

The Draft Build serves as a short-lived build to test out our code and see the resulting dataset quickly without having to commit our changes.&#x20;

The Draft Build can be invoked by clicking the `Draft Build` button in the code tab. It will use the latest uncommitted code to create a temporary build.

{% hint style="danger" %}
There is only one Draft Build slot per transform, so everything, including data, artifacts, and logs, gets completely overwritten on every subsequent Draft Build.
{% endhint %}

When running a Draft Build on a Transform that has upstream dependencies, the scheduling engine will first attempt to pull a Draft Build dataset from upstream if available before falling back to a regularly built dataset. This behavior enables testing of new code on dependent datasets without having to commit the changes.

<figure><picture><source srcset="/files/JnVznlFcPwzpF6llHZpT" media="(prefers-color-scheme: dark)"><img src="/files/Khr4z97JUOhJ3ONYbHXN" alt="Draft Build Indicators"></picture><figcaption><p>Draft builds are denoted by a yellow outline and a flask Icon</p></figcaption></figure>

### Incremental Build

A regular build always deletes the current dataset and replaces it with the freshly built dataset. In some cases, however, it is desirable to retain old data, for example, when historisation is required. This use case is supported and can be achieved with the following setup:

DataSpace internally stores the transforms as folders, code as files, and the produced datasets as parquet files. To perform an incremental build, the existing Parquet file from the transform can be manually read and concatenated to the new calculations.

```python
import polars as pl
from datetime import datetime
import os

def transform():
    lf = pl.LazyFrame({'date': [datetime.now()]})
    
    old_path = os.environ['PREVIOUS_DATASET_PATH']
    if os.path.isfile(old_path):
        lf_old = pl.scan_parquet(old_path)
        lf = pl.concat([lf, lf_old], how="diagonal_relaxed")        
    else:
        print("No prior data found.")

    return lf
```

It often makes sense to run such an incremental build periodically. Please refer to Build Schedules to learn more.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.dataspace.ch/platform/builds.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
