Using Conditional Runtime Parameters

Let’s say you are creating a block and want to make certain parameters depend on others - where the requirement or value of one parameter depends on another parameter’s value. Conditional parameters are meant for this specific use case. Here’s how you can use them.

Understanding Conditional Parameters

The very first question we need to answer is how to define conditions for your parameters. For the following examples, let’s consider a data processing workflow where parameters like data_source, processing_type, filter_condition, and aggregation_method depend on each other.

  • Parameters can have Basic conditions (depending on a single parameter) or Composite conditions (depending on multiple parameters with AND/OR logic).

  • Each condition specifies which parameter it depends on, what operator to use for comparison, and what value to compare against.

  • The parameter validation function validates all conditions to ensure they are logically consistent and valid.

Implementing Conditional Parameters

Basic Condition

For straightforward dependencies, use a simple condition format:

# basic_conditions.yaml

parameters
  run
    name: filter_condition
    type: string
    required: true
    conditions:
        - dependsOn: data_source
          operator: equals
          value: database

In this case, filter_condition is only required when value of data_source equals “database”.

Composite Condition

For more complex scenarios, you can use logical operators to combine multiple conditions:

# composite_conditions.yaml

parameters
  run
    name: aggregation_method
    type: string
    required: true
    conditions:
        - conditionType: OR
          conditions:
            - dependsOn: processing_type
              operator: equals
              value: batch
            - dependsOn: filter_condition
              operator: equals
              value: time_series

Here, aggregation_method is required when either processing_type equals “batch” OR filter_condition equals “time_series”.

Complete Example

Here is a full example showing different types of conditions in a data processing workflow:

# workflow_conditions.yaml

parameters
    run:
      - name: data_source
        type: string
        required: true
      - name: processing_type
        type: string
        required: true
      - name: filter_condition
        type: string
        required: true
        conditions:
          - dependsOn: data_source
            operator: equals
            value: database
      - name: aggregation_method
        type: string
        required: true
        conditions:
          - conditionType: OR
            conditions:
              - dependsOn: processing_type
                operator: equals
                value: batch
              - dependsOn: filter_condition
                operator: equals
                value: time_series
      - name: custom_transformation
        type: string
        required: false
        conditions:
          - conditionType: AND
            conditions:
              - dependsOn: processing_type
                operator: equals
                value: batch
              - dependsOn: filter_condition
                operator: equals
                value: time_series

This example shows a data processing workflow where:

  • filter_condition is required when data comes from a database.

  • aggregation_method is needed for batch processing or time series data.

  • custom_transformation is optional but becomes relevant for batch processing or time series data.

Example with App Parameters for App Deployment

This example demonstrates how to use app parameters and block parameters together with conditions during app deployment:

Spec Parameters

# workflow_conditions.yaml

appParameters
    run:
      - name: data_source
        type: string
        required: true
      - name: processing_type
        type: string
        required: true
      - name: filter_condition
        type: string
        required: true
        conditions:
          - dependsOn: data_source
            operator: equals
            value: database

parameters
    run:
      - name: data_source
        type: string
        required: false
      - name: processing_type
        type: string
        required: false
      - name: filter_condition
        type: string
        required: false
        conditions:
          - dependsOn: data_source
            operator: equals
            value: database
      - name: aggregation_method
        type: string
        required: true
        conditions:
          - conditionType: OR
            conditions:
              - dependsOn: processing_type
                operator: equals
                value: batch
              - dependsOn: filter_condition
                operator: equals
                value: time_series
      - name: custom_transformation
        type: string
        required: true
        conditions:
          - conditionType: AND
            conditions:
              - dependsOn: processing_type
                operator: equals
                value: batch
              - dependsOn: filter_condition
                operator: equals
                value: time_series

Deployment Parameters

When deploying the app, app parameters and block parameters are provided as follows:

 appParameters:
  run:
    data_source: "database"
    processing_type: "batch"
    filter_condition: "date"

parameters:
  block-1:
    run:
      aggregation_method: "SUM"
      custom_transformation: "Aggregator transformation"

Parameter Evaluation

Let’s analyze how the conditions are evaluated in this example:

  1. App Level Parameters

  • data_source: Set to "database"

  • processing_type: Set to "batch"

  • filter_condition: Set to "date" (condition met because value of data_source is "database")

  1. Block Level Parameters

  • aggregation_method: "SUM"

    • Condition is met because:

      • processing_type equals "batch" (satisfies OR condition)

      • The value is accepted regardless of filter_condition

  • custom_transformation: "Aggregator transformation"

    • Condition is NOT met because:

      • Although processing_type equals "batch"

      • filter_condition equals "date", not "time_series"

      • AND condition requires both conditions to be true

    • This parameter will be excluded from the final run parameters

Alternative Scenario

If the appParameters in the deployment had specified as follows:

appParameters:
    run:
        data_source: "database"
        processing_type: "batch"
        filter_condition: "time_series" # Changed value

Then:

  • aggregation_method would still be accepted (satisfies OR condition)

  • custom_transformation would now be accepted because:

    • processing_type equals "batch"

    • filter_condition equals "time_series"

    • Both conditions in the AND clause are satisfied

Keys Points for app deployment

  • App parameters are evaluated first and inherited by blocks.

  • Conditions are evaluated based on the final combined parameter values.

  • Parameters that don’t meet their conditions are excluded from the final configuration.

Validation Rules

When implementing conditional parameters, the system enforces the following validation rules:

Parameter Existence

  • All referenced parameters must exist within the specification.

Operator Validity

  • Currently only 2 operators are supported: equals and notEquals for boolean, string and numeric parameter type.

Type Matching

  • The Condition values must match their respective parameter’s data types.

  • Example: For boolean parameters, use true, not "true".

Logical Consistency

  • Contradictory conditions are not allowed.

  • A parameter condition cannot depend on two different values of the same parameter at the same time.

  • Example: custom_transformation should not depend on processing_type = “batch” and processing_type = “stream” simultaneously.

Circular Dependencies

  • Parameters can’t create dependency cycles.

  • Example: if processing_type depends on filter_condition and filter_condition depends on processing_type then this condition is invalid.

Combined Conditions and Dependency Failures

In some cases, dependencies can cascade. For example:

  • filter_condition depends on data_source being “database”.

  • If the condition for filter_condition fails, any parameter depending on it (e.g., aggregation_method, custom_transformation) will also fail. This cascading failure ensures logical consistency and prevents invalid workflows.

Best Practices

  • Use simple array format for straightforward conditions.

  • Use nested logical format only when complex logic is required. The max depth of conditions for a parameter should be 5.

  • Keep condition chains as short as possible.

  • Account for the evaluation order of parameters in your application logic.