Planning

Planning Overview

Planning is the process of preparing and setting up processing tasks based on a scenario definition. This activity involves several key steps to ensure tasks are scheduled and executed correctly.

Steps in the Planning Process

• Waiting for a Planning Event - A planning event is triggered based on the scenario planning mode setting. This event can be one of the following types
  • Time-based
  • File-based
  • Other types (as defined in Scenario)
• Retrieving Processing Parameter Values - User-defined parameters (refer to Parameter) are evaluated. Specifically, parameters whose values are defined by database queries will be evaluated. Only those parameters that are used by the scenario tasks will be evaluated. Parameters that are not part of the scenario will not be considered during this step.
• Scheduling Tasks for Execution - All valid tasks (see Task for task validity) are scheduled to be executed as part of the resulting Processing. These tasks are queued and await execution. No tasks will begin until the planning activity is complete.
• Notifying Operators - If the "Start scenario" operator is selected, a notification will be sent to the designated operators, informing them that the scenario has been triggered.

Generic constraints

Generic constraints as scenario constraints valid for all planning modes

• Scenario Validity Dates: - The scenario's validity period is taken into account. If the scenario has defined validity dates, planning will only proceed within that period.
• Time constraint:
  • If defined "Not before" field, planning will be delayed to match start time condition
  • If defined "Not after" field, planning will not start if the time is after the time set; will wait till midnight

Time-Based Planning

Daily scenario

In scenarios set to Daily Planning Mode, time-based events act as the primary driver for planning. This mode uses the following conditions to trigger planning events:

• Completion of Previous Instances: - The current planning event will only trigger if all previous instances (dates) of the scenario have completed.
• Date evaluation:
  • If the scenario schedule base is set to "Previous run", it will attempt to trigger the next instance for the day following the last completed one. This mode does not allow "holes" in processing dates. For example if the processing is paused for a week, then the next planning will occur for the day following, e.g. 6 days ago.
  • If the scenario schedule base is set to "Calendar/realtime", it will attempt to trigger for the current day. This mode allows date "holes" in processing, meaning if the processing is paused for a week, the planning is based on the Calendar day leaving a 6-day "hole" in processed dates.
• Holiday Considerations: - Planning will take into account holidays based on internal calendar information. (See Calendar for more details.) and the "Workdays only" flag
• Lag Settings: - The Lag setting defines a delay between the current processing and its execution. For example, a Lag setting of 1 means that processing scheduled for 2nd January 2025 will start no sooner than 3rd January 2025.
• Generic constraints - see the previous sections where the generic constraints are described

File-Based Planning

File based

In scenarios set to File-based Planning Mode, planning is started when a file matching a specified mask is stored at a specified location (folder). On scenario, the following fields must be set:

• Base directory: - Directory, where the files are being checked. It is a local directory or a directory mounted to the local filesystem (local relatively to the Dafne Executor component).
• File name regexp: - Regular expression to match filename in the defined location. File name must contain the date of processing for planning. For this purpose regular expression named groups are used. Group names are "year", "month" and "day", optionally "iteration". For example to match a file with name data_20240101_i1.xml the following expression can be used: data_(?<year>20[0-9]{2})(?<month>[01][0-9])(?<day>[0-3][0-9])_(?<iteration>i[0-9]{0,12}).xml

Manual Planning

In manual planning mode, executor is not waiting for any automated event. Planning process is initiated by user either clicking in GUI or calling API method.

SQL-Based Planning

Daily scenario

SQL-based planning is the most advanced and the most flexible way of triggering. It allows users to start planning based on custom condition with the use of SQL language, processing metadata and additional information from DB server. The following fields are relevant for SQL-based planning:

• Generic constraints - see the previous sections where the generic constraints are described
• Connection - Choose a connection to be used to run the query. Allowed only database-type providers
• Check type - defines if dates must be planned in a specific order:
  • None: No checking is done
  • Date: Planning is done if the proposed date is higher than any other processing dates of the same scenario; in other words it is not possible to initiate the planning a date that is older than already started in the past
  • Iteration: Checks ascending order at date and iteration level; allows to start only newer iterations than already started; allows planning if the date is higher than any other date or date is the same as the maximal started in the past and at the same time iteration name is higher (by string comparison) than the highest iteration within the same scenario and date
• Polling frequency - Defines period in minutes to wait between retries, retry runs only in case the Max parallel condition is met (see next item)
• Max parallel - Maximum number of scenario instances (processings) that can be active at once. For example when Max parallel=2, scenario can run in for 2 different dates at the moment
• Query - SQL query with the following expected structure of the result; only the first row is processed, additional rows are ignored
  • date - first column contains date as a native DB date type (or convertable), varchar type in format YYYY-MM-DD or numeric format YYYYMMDD; this result is mandatory for scenario planning to start
  • iteration - optional varchar column; used to distinguish standard daily runs from special runs or to allow intra-day runs

SQL can contain special placeholders (see Placeholders) formatted as %<parameter name>% or user defined parameters (with default values only) in the format %u:<parameter name>% where <parameter name> is the name of a system-level or user defined parameter (see Parameters)