Dafne: Documentation import

2026-08-06T13:36:48Z

Documentation import

New page

[[File:monitor_reprocess.png|thumb|right|Reprocess definition dialog]]
Reprocessing runs a selected set of tasks over a range of processing dates. It is the standard response to a corrective load: source data for a past period has been restated, and the tables derived from it have to be rebuilt for every affected day.

Reprocessing creates a '''processing instance for every day in the selected interval'''. It does not modify the original processings for those dates.

== When to use it ==
Use reprocessing when:
* data has been corrected retroactively and the dependent tables must be rebuilt over a period
* a defect in a transformation has been fixed and the affected days have to be recalculated
* a subset of tasks — not a whole scenario — has to run again across several dates

Use [[Controlling a processing#Reopen|Reopen]] instead when the correction concerns a single finished processing and should stay attached to it.

== The Reprocess definition dialog ==
The dialog has a selection area at the top and two task lists below.

{| class="wikitable"
|+ Fields
! Field !! Description
|-
| '''Scenario''' || The scenario the tasks belong to. Type to search; pick one from the list.
|-
| '''Group''' || Optional. Narrows the available task list to one group.
|-
| '''Date range''' || Start and end date. A processing instance is generated for '''every day''' in this interval.
|-
| '''Iteration''' || A code distinguishing this run from the regular one for the same dates. Maximum 20 characters.
|}

The '''Available tasks''' list shows the tasks matching the scenario and group selected, and can be filtered further by name. Click a task to move it to '''Selected tasks'''; click an entry there to remove it.

'''Run''' becomes available once the selection is valid. '''Cancel''' discards it.

== The iteration code ==
The iteration code matters more than it appears. A processing is identified by its scenario, date and iteration, and '''a processing cannot be created for a scenario and date that already has the same iteration code'''. Give the reprocessing run a distinctive code — something that identifies the reason, such as the ticket reference — rather than reusing a code already present.

Iteration codes are compared as '''text''', not as numbers or dates. Where the order of iterations matters, pad them so that string ordering matches the intended order: use ''09'' and ''10'' rather than ''9'' and ''10''. See [[Processing]].

== Selecting the right tasks ==
The dialog runs exactly the tasks selected. It does not automatically pull in the tasks that depend on them.

Before running:
* select the task that rebuilds the corrected data, then use the '''Forward''' tab in the [[Monitor console]] on that task to see what depends on it
* include every downstream task whose output would otherwise be inconsistent with the corrected data
* check the [[Parallelism|multiday mode]] of the tasks involved — this determines whether the days can be processed concurrently or must run in order

Selecting too few tasks leaves the warehouse internally inconsistent, which is harder to detect than an outright failure.

== How the days are executed ==
Each day in the interval becomes its own processing, and the days are subject to the normal rules:
* the scenario's '''Max parallel''' setting limits how many of them can be active at once
* each task's '''multiday mode''' determines whether it may run for several dates concurrently, must run in date order, or must wait for previous days to complete

A reprocessing run over a long interval with tasks set to ''Fully processed'' will run essentially in sequence, and will take correspondingly longer. Understanding this before starting avoids the impression that the run has stalled. See [[Parallelism]].

== Monitoring the run ==
The generated processings appear in the Active processings panel like any others, identified by their iteration code. They can be paused, resumed and have tasks added individually; see [[Controlling a processing]].

For a long interval, use a scenario or iteration filter to separate the reprocessing run from the regular daily run, which will continue alongside it.

== Before starting a large reprocessing ==
* Confirm the corrected source data is actually in place. Reprocessing against uncorrected data wastes the run.
* Consider the load. A reprocessing run competes with the regular processing for the same connections and resource limits. Where the interval is long, either run it outside the main processing window, or lower its priority so it yields to the daily run. See [[Prioritization]] and [[Resource management]].
* Decide the iteration code in advance and record it, so the run can be identified later.

== See also ==
* [[Recovery after an outage]]
* [[Parallelism]]
* [[Processing]]

Reprocessing data - Revision history

Dafne: Documentation import