Using `Images`, `Workflows` and `Webapps` via CLI

To create images, workflows or webapps via the Command Line Interface (CLI), we can define a YAML file or pass the required options through command line arguments. This YAML file includes the payload, metadata, and configuration of the image. By utilizing jinja syntax, we have the ability to pass parameter value as {{value}}, with the actual value being specified in a separate YAML file.

Tenants

The tenants resource provides required funtions/commands to Manage tenant settings and quota.
The tenants resource currently offers only one operation namely list-instance-options. The command helps in getting the allowed instance type options for a given tenant and entity-type. Here’s how the command can be used:
- To obtain a list of all available instances along with their corresponding instanceTypeId for workflows, we can use the following command:
```
peak tenants list-instance-options --entity-type workflow
```
- To obtain a list of all available instances along with their corresponding instanceTypeId for webapps, we can use the following command:
```
peak tenants list-instance-options --entity-type webapp
```
- Allowed values for the entity-type option are api-deployment, data-bridge, webapp, workflow, workspace.

Defining Parameters in Resource Creation

While creating resources, we can define parameters in a separate file and then use them in the actual file. We can refer the parameters by enclosing the parameter names in {{ }} and specifying the file path after the -v flag in the command. We can also add parameters by adding the desired key-value pair after the -p flag. If we specify both -v and -p, the parameters defined on the command line will overwrite the parameters defined in the file.

Consider the following parameters file:

# params.yaml

git_token: "random_git_token"
npm_token: "random_npm_token"

Consider the simple yaml file where we want to use the parameters defined in the above file:

# image.yaml

body:
    name: upload-image
    version: 0.0.1-test
    description: Creating a new upload type image
    type: workflow
    buildDetails:
        source: upload
        useCache: false
        buildArguments:
            - name: git_token
              value: {{ git_token }}
            - name: npm_token
              value: {{ npm_token }}
        context: "."
artifact:
    path: "."
    ignore_files:
        - ".gitignore"
        - ".dockerignore"

We can use the following command to create an image:

peak images create -v path/to/params.yaml path/to/image.yaml

In the above example, the {{ git_token }} and {{ npm_token }} will be replaced by the values defined in the params.yaml file.

We can also add parameters by adding the desired key-value pair after the -p flag. If we specify both -v and -p, the parameters defined on the command line will overwrite the parameters defined in the file.

peak images create path/to/image.yaml -p git_token=git_token_value -p npm_token=npm_token_value

Similar approach can be used for creating other resources as well.

Using a combination of command line options and yaml

For creating resources such as images and webapps, we can use a combination of command line options and the yaml. In this case the command line options will take precedence over the yaml values.

Consider the following example for creating a webapp:

# webapp_params.yaml

imageId: 2
versionId: 2

# webapp.yaml
body:
    title: Updated webapp
    description: This is an updated webapp
    imageDetails:
        imageId: {{ imageId }}
        versionId: {{ versionId }}

peak webapps update -v path/to/webapp_params.yaml <webapp-id> path/to/webapp.yaml --image-id 3

In the above example, the imageId will be overwritten by the command line options and the final webapp body will look like:

{
    "title": "Updated webapp",
    "description": "This is an updated webapp",
    "imageDetails": {
        "imageId": 3,
        "versionId": 2
    }
}

Creating Resource Dependencies with Artifacts

If we want to create a resource such as image, but we don’t want it to be source directly from the source code repository, then we will need to pack all the files we might need to build the image and pack them into an zip archive which is called Artifact.

More information for Artifact can be found in Artifact and Compression Doc.

Providing Instance Type and Storage in Workflow

When defining a workflow step, we have the option to set the instance type and storage by including them under the resources key in the following YAML format:

resources:
    instanceTypeId: 23,
    storage: 20GB,

To obtain a list of all available instances along with their corresponding instanceTypeId, we can use the following command:

peak tenants list-instance-options --entity-type workflow

Adding the resources section is optional. If we don’t specify it for a particular step, the default values will be used. We can retrieve the default values through the following command:

peak workflows list-default-resources

Providing Instance Type in Webapp

When creating a webapp, we can set the instance type by including it under the resources key in the following YAML format:

resources:
    instanceTypeId: 1

To obtain a list of all available instances along with their corresponding instanceTypeId, we can use the following command:

peak tenants list-instance-options --entity-type webapp

Session Stickiness in Webapp

By default, session stickiness is disabled (false). You can activate session stickiness by setting the sessionStickiness parameter to true.

Session stickiness ensures that each user’s requests are consistently directed to a particular server. This feature is especially valuable for stateful applications like web applications that rely on server-stored session data.

Note that employing session stickiness may lead to unpredictable behavior and is not recommended if you plan to scale your application.

Images

Creating an image

We can create an image by providing payload containing its configuration. If version is not provided in the request body, the version will be set to 0.0.1.

buildDetails is optional and is set to the following when it is not provided:

{
    "source": "upload"
}

Within buildDetails if source is not provided, it is set to upload by default.

# image_params.yaml

token: my_token
param1: my_parameter1
param2: my_parameter2

# upload_image.yaml

body:
  name: upload-image
  version: 0.0.1-test
  description: Creating a new upload type image
  type: workflow
  buildDetails:
    source: upload
    useCache: false
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    context: .
artifact:
  path: "."
  ignore_files:
    - ".gitignore"
    - ".dockerignore"

# github_image.yaml

body:
  name: github-image
  version: 0.0.1-test
  description: Creating a new github type image
  type: workflow
  buildDetails:
    source: github
    repository: https://example.com
    branch: main
    token: token
    dockerfilePath: Dockerfile
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    useCache: true

# dockerfile_image.yaml

body:
  name: dockerfile-image
  version: "0.0.1-test"
  description: Creating a new dockerfile type image
  type: workflow
  buildDetails:
    source: dockerfile
    useCache: false
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    dockerfile: "FROM nginx"

We can use the following command to create an image:

peak images create path/to/create_image.yaml -v path/to/image_params.yaml

We can create an image directly from the command line by specifying all the required parameters:

peak images create --name example-image --version 0.0.1-test --type workflow --description "Creating a new image" --source github --repository https://example.com --branch main --build-arguments arg1=value1 --build-arguments arg2=value2 --secrets secret_1,secret_2

We can also combine the YAML template and command line arguments, where the command line arguments will take precedence over the YAML file. Create the YAML template file with the following content:

# image.yaml

body:
    name: my-image
    version: 0.0.1-test
    type: workflow
    description: Creating a new image
    buildDetails:
        source: github
        repository: https://example.com
        branch: main
        useCache: true

peak images create path/to/image.yaml --name my-awesome-image

In this case, the final image body will look like:

{
    "name": "my-awesome-image",
    "version": "0.0.1-test",
    "type": "workflow",
    "description": "Creating a new image",
    "buildDetails": {
        "source": "github",
        "repository": "https://example.com",
        "branch": "main",
        "useCache": true
    }
}

Creating an image version

Once an image is created, we can create its subsequent version by providing the image id and payload containing its configuration. If version is not provided in the request body, the version will be automatically incremented.

buildDetails is optional and is set to the following when it is not provided:

{
    "source": "upload"
}

Within buildDetails if source is not provided, it is set to upload by default.

# image_params.yaml

token: my_token
param1: my_parameter1
param2: my_parameter2

# upload_image_version.yaml

body:
  version: 0.0.2-test-upload
  description: Creating a new image version
  buildDetails:
    source: upload
    context: "."
    dockerfilePath: Dockerfile
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    useCache: true
artifact:
  path: "."
  ignore_files:
    - ".gitignore"
    - ".dockerignore"

# github_image_version.yaml

body:
  version: 0.0.2-test-github
  description: Creating a new image version
  buildDetails:
    source: github
    repository: https://example.com
    branch: main
    token: token
    dockerfilePath: Dockerfile
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    useCache: true

# dockerfile_image_version.yaml

body:
  version: 0.0.2-test-dockerfile
  description: Creating a new image version
  buildDetails:
    source: dockerfile
    useCache: false
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    dockerfile: "FROM nginx"

We can use the following command to create an image version:

peak images create-version -v path/to/image_params.yaml <image-id> path/to/create_image_version.yaml

We can create an image version directly from the command line by specifying all the required parameters:

peak images create-version 123 --version 0.0.2-test --description "Creating a new image version" --source github --repository https://example.com --branch main --build-arguments arg1=value1 --build-arguments arg2=value2 --secrets secret_1,secret_2

Similar to creating an image, we can also combine the YAML template and command line options to create the image version, with the command line options taking precedence.

Updating an image version

An image version can be modified if it was not build successfully and is in not-ready state. We can update an image version by providing the image id, version id and payload containing its configuration. buildDetails and source are optional. If not provided, the details from existing version will be used.

# upload_image_version.yaml

body:
  description: Updating the image version
  buildDetails:
    source: upload
    context: "."
    dockerfilePath: Dockerfile
    buildArguments:
      - name: PARAM1
        value: param1
    useCache: true
artifact:
  path: "."
  ignore_files:
    - ".gitignore"
    - ".dockerignore"

# github_image_version.yaml

body:
  description: Updating the image version
  buildDetails:
    source: github
    repository: https://example.com
    branch: main
    token: token
    dockerfilePath: Dockerfile
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    useCache: true

# dockerfile_image_version.yaml

body:
  description: Updating the image version
  buildDetails:
    source: dockerfile
    useCache: false
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    dockerfile: "FROM nginx"

We can use the following command to update an image version:

peak images update-version --image-id -v path/to/image_params.yaml <image-id> --version-id <version-id> path/to/update_version.yaml

We can update an image version directly from the command line by specifying all the required parameters:

peak images update-version --image-id 123 --version-id 456 --description "Updating the image version" --source github --repository https://example.com --branch main --build-arguments arg1=value1 --build-arguments arg2=value2 --secrets secret_1,secret_2

Similar to creating an image, we can also combine the YAML template and command line options to update the version, with the command line options taking precedence.

Using the `create-or-update` operation

The operation creates a new resource if it doesn’t exist and updates the resource in case it exists. The search is based on resource name and version fields.

Consider the following example for creating a new image version:

# upload_image_version.yaml

body:
  name: test-image
  version: 0.0.1-test-upload
  description: Creating a new image version
  buildDetails:
    source: upload
    context: "."
    dockerfilePath: Dockerfile
    buildArguments:
      - name: PARAM1
        value: param1
      - name: PARAM2
        value: param2
    useCache: true
artifact:
  path: "."
  ignore_files:
    - ".gitignore"
    - ".dockerignore"

We can use the following command to create or update a image version:

peak images create-or-update -v path/to/image_params.yaml path/to/create_or_update_image.yaml

We can create or update a image version directly from the command line by specifying all the required parameters:

peak images create-or-update --name test-image --version 0.0.2-test --description "Creating a new image version" --source github --repository https://example.com --branch main --build-arguments arg1=value1 --build-arguments arg2=value2 --secrets secret_1,secret_2

Similar to creating an image, we can also combine the YAML template and command line options to create the image version, with the command line options taking precedence.

Deleting Images and Versions

At some point you might want to clean up a few images and image versions that you have on your tenant. The CLI provides you with a few options to easily perform the cleanup.

If you want to delete a specific version, you can use the delete-version command available in the images resource.
```
peak images delete-version --image-id=1 --version-id=1
```
You might have a lot of versions inside an image, that you want to delete. No worries, we have delete-versions command that allows you to delete multiple versions in an image with a single command.
```
peak images delete-versions --image-id=1 --version-ids=1,2,3
```
Note: All the versions will first go into deleting state and will then be asynchronously deleted. If the deletion of any version fails because of some issue, it will go into delete_failed state.
If none of the versions in an image are needed, and you want to delete the complete image, you can use the delete command in the images resource
```
peak images delete 1
```
Note: All the versions associated with the given image will first go into deleting state and will then be asynchronously deleted. If the deletion of any version fails because of some issue, it will go into delete_failed state.

Retrieving Image Build Logs

To obtain build logs for a specific image build, you’ll need to provide the image ID and build ID. There are three possible ways to get the image build logs.

Using Next Token

To fetch build logs, use the following command:

peak images build-logs --image-id <image_id> --build-id <build_id>

This command returns a set of logs along with a nextToken that can be used to fetch subsequent logs. The response looks like following:

{
    "buildStatus": "building",
    "finishTime": "2023-06-01T19:00:00.000Z",
    "logs": [
        {
            "ingestionTime": "1697089188119",
            "message": "Building the Docker image...",
            "timestamp": "1697089188119"
        }
    ],
    "nextToken": "f/37241814580157116960215105647056337845181039459911335941/s",
    "startTime": "2023-06-01T18:00:00.000Z"
}

To retrieve the next set of logs, use the nextToken received in the response as follows:

peak images get-build-logs --image-id <image_id> --build-id <build_id> --next-token <next_token>

Polling the logs

To continuously poll the logs for the image build, you can use the --follow flag. This command will keep polling the logs until the build is completed or all logs are fetched.

peak images get-build-logs --image-id <image_id> --build-id <build_id> --follow

Save the logs locally to a file

Image build logs can also be saved locally to a file using the --save option. You can also pass in a --file-name option which should include the file name and the path to used when saving the logs. If the --file-name option isn’t passed, the logs would be saved to a file named image_build_logs_<image_id>_<build_id>.log.

peak images get-build-logs --image-id <image_id> --build-id <build_id> --save --file-name <file_name>

Note The --save option would take the highest prioroty when both --follow and --save options are passed.

Workflows

Note: Workflows with only standard steps are supported.

Creating a workflow

We can create a workflow by providing payload containing its configuration.

# workflow.yaml

body:
  name: new-workflow
  triggers:
    - cron: "0 0 * * *"
  watchers:
    - user: abc@peak.ai
      events:
        success: false
        fail: true
    - webhook:
        name: info
        url: "https://abc.com/post"
        payload: '{ "pingback-url": "https:/workflow/123" }'
      events:
        success: false
        fail: true
        runtimeExceeded: 10
  retryOptions:
    duration: 5
    exitCodes: [1, 2]
    exponentialBackoff: true
    numberOfRetries: 3
  tags:
    - name: foo
    - name: bar
  steps:
    stepName:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: "python script.py"
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true
      parameters:
        env:
          key: value
        secrets:
          - secret-1
          - secret-2
      repository:
        branch: main
        token: random_token
        url: "https://github.com/org/repo"

We can use the following command to create a workflow:

peak workflows create path/to/create_workflow.yaml -v path/to/workflow_params.yaml

Updating a workflow

We can update a workflow by providing payload containing its configuration.

# workflow.yaml

body:
  name: updated-workflow
  triggers:
    - webhook: true
      webhookPolicy: "generate"
  tags:
    - name: CLI
  steps:
    step1:
      command: echo hello world
      type: standard
      repository:
        branch: main
        token: random_token
        url: "https://github.com/org/repo"
      imageId: 100
      imageVersionId: 100
      resources:
        instanceTypeId: 21
        storage: 10GB
    step2:
      command: echo world
      type: standard
      imageId: 200
      imageVersionId: 200
      resources:
        instanceTypeId: 21
        storage: 40GB

We can use the following command to update a workflow:

peak workflows update  -v path/to/workflow_params.yaml <workflow-id> path/to/update_workflow.yaml

Using the `create-or-update` operation

We can create or update a workflow by providing payload containing its configuration. The search operation will be based on name.

# workflow.yaml

body:
  name: my-new-workflow
  triggers:
    - webhook: true
      webhookPolicy: "generate"
  tags:
    - name: CLI
  steps:
    step1:
      command: echo hello world
      type: standard
      repository:
        branch: main
        token: random_token
        url: https://example.com
      imageId: 100
      imageVersionId: 100
      resources:
        instanceTypeId: 21
        storage: 10GB
    step2:
      command: echo world
      type: standard
      imageId: 200
      imageVersionId: 200
      resources:
        instanceTypeId: 21
        storage: 40GB

We can use the following command to create or update a workflow:

peak workflows create-or-update  -v path/to/workflow_params.yaml path/to/create_or_update_workflow.yaml

Partial Workflow Update using `patch` workflow

Consider an existing workflow with step details:

"name": "existing-workflow",
"triggers": [],
"steps": {
    "step1": {
        "imageId": 1,
        "imageVersionId": 1,
        "command": "python test1.py",
        "resources": {
            "instanceTypeId": 21,
            "storage": "10GB",
        }
    },
    "step2": {
        "imageId": 2,
        "imageVersionId": 2,
        "command": "python test2.py",
         "resources": {
            "instanceTypeId": 22,
            "storage": "10GB",
        }
    },
}

We can partially update a workflow in the following ways.

Command Line Arguments Approach

By passing the parameters to be updated as command line arguments. We can provide step names --step-names, which would update the steps with the provided parameters. In case step names are not provided, all the steps of the workflow would be updated.

peak workflows patch <workflow_id> --name updated-workflow --image-id 100 --image-version-id 100 --step-names step1

This would update imageId and imageVersionId of step1 and step2 of the workflow.

"name": "updated-workflow",
"steps": {
    "step1": {
        "imageId": 100,
        "imageVersionId": 100,
        "command": "python test1.py",
        "resources": {
            "instanceTypeId": 21,
            "storage": "10GB",
        },
    },
    "step2": {
        "imageId": 2,
        "imageVersionId": 2,
        "command": "python test2.py",
        "resources": {
            "instanceTypeId": 22,
            "storage": "10GB",
        },
    },
}

YAML File Approach

We can update parameters via a YAML file:

# patch-workflow.yaml

body:
  name: "updated-workflow"
  triggers:
    - {}
  steps:
    step1:
      imageId: 100
      imageVersionId: 100
    step3:
      imageId: 400
      imageVersionId: 400
      command: "python test4.py"
      resources:
        instanceTypeId: 24
      repository:
        url: "https://github.com/org/repo"
        branch: main
        token: random_token

Use the following command to update the workflow:

peak workflows patch <workflow_id> path/to/patch_workflow.yaml

The updated workflow would look like following:

"name": "updated-workflow",
"triggers": [],
"steps": {
    "step1": {
        "imageId": 100,
        "imageVersionId": 100,
        "command": "python test1.py",
        "resources": {
            "instanceTypeId": 21,
            "storage": "10GB",
        },
    },
    "step2": {
        "imageId": 2,
        "imageVersionId": 2,
        "command": "python test2.py",
        "resources": {
            "instanceTypeId": 22,
            "storage": "10GB",
        },
    },
    "step3": {
        "imageId": 400,
        "imageVersionId": 400,
        "command": "python test4.py",
        "resources": {
            "instanceTypeId": 24,
            "storage": "10GB",
        },
        "repository": {
            "branch": "main",
            "token": "github",
            "url": "https://github.com/org/repo",
        },
    },
}

Through the above approach, specific keys of step1 are updated, while step2 remains unchanged, and step3 is added as a new step.

We can also combine the YAML file and command line options, with command line options taking precedence.

Note that the keys and values in the above examples are for illustrative purposes.

Setting up advanced workflow configurations

You can customize workflows using options such as Auto retry, Skippable steps, Output Parameters, and Conditional step execution by including the relevant payload in the workflow creation or update process.

More information about these options can be found in the Peak Platform Knowledge Base.

Auto retry configuration

Auto retry is a powerful feature that automates the retrying of failed steps within a workflow. This configuration can be applied at either the workflow level or the individual step level, with the latter taking precedence over the former.

To implement auto retry at the workflow level, use the following payload example:

# workflow-auto-retry.yaml

body:
  name: new-workflow
  triggers:
    - cron: "0 0 * * *"
  retryOptions:
    duration: 5
    exitCodes: [1, 2]
    exponentialBackoff: true
    numberOfRetries: 3
  steps:
    step-1:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: echo hello
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true
      runConfiguration:
        retryOptions:
          duration: 5
          exitCodes: [1, 137, 143]
          exponentialBackoff: true
          numberOfRetries: 3
    step-2:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: echo world
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true

In the provided example, auto retry is configured at the workflow level, affecting all steps within the workflow.

Auto retry configuration set at the step level will override the workflow-level settings.

Skippable steps configuration

Configure individual workflow steps or their dependencies to be skipped during execution by providing the necessary payload during workflow creation or update.

Use the payload example below to create a workflow with skippable steps configuration:

# workflow-auto-retry.yaml

body:
  name: new-workflow
  triggers:
    - cron: "0 0 * * *"
  steps:
    step-1:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: echo hello
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true
      runConfiguration:
        skipConfiguration:
          skip: true
          skipDAG: true
    step-2:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: echo world
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: ["step-1"]
      stepTimeout: 30
      clearImageCache: true

In this example, step-1 is configured to be skippable, and the skipDAG property is set to true, skipping all steps dependent on step-1.

If skipDAG is true for a parent step, all children steps will be skipped, regardless of their individual configuration.

Input parameters configuration

Input parameters enable you to pass dynamic values as environment variables, which your scripts can utilize at runtime. These values can be used to control the behavior of the step or to provide input to the step.

Input parameters can be employed in two ways:

Using Key-Value pairs

In this method, input parameters are provided as key-value pairs. The key denotes the name of the environment variable, while the corresponding value sets the variable’s runtime value.

Using Secrets or External Credentials

Alternatively, input parameters can be passed as a list of predefined secrets or external credentials, enhancing security and usability.

To create a workflow with input parameters, you can refer the following payload example:

# workflow_input_parameters.yaml

body:
  name: new-workflow
  triggers:
    - cron: "0 0 * * *"
  steps:
    step-1:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: "echo hello"
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true
      parameters:
        env:
          key: value
        secrets:
          - secret-1
          - secret-2

Output parameters configuration

Output parameters allows you to capture and share the output generated by a parent step in a workflow. This captured output can then be used as input or reference in subsequent steps within the same workflow.

Output parameters can be defined by providing a name and path to the file that contains the data that should be exported. The file path specified in the output parameter should exist in the step, else an error will be thrown when the step is run. Once a parent has declared an output parameter, the child steps can use it as an input parameter.

To create a workflow with output parameters, use the following payload example:

# workflow_output_parameters.yaml

body:
  name: new-workflow
  triggers:
    - cron: "0 0 * * *"
  steps:
    step-1:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: "python script1.py > output.txt"
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true
      repository:
        branch: main
        token: random_token
        url: "https://github.com/org/repo"
      outputParameters:
        scriptOutput: "output.txt"

In the provided example, an output parameter named scriptOutput is defined for step-1, and this parameter can be utilized in any children steps.

Execution condition configuration

Configure individual workflow steps to execute based on the output parameters or status of parent steps. This feature allows you to control the flow of your workflow dynamically.

To create a workflow with execution condition configuration, use the following payload example:

# workflow-execution-parameters.yaml

body:
  name: new-workflow
  triggers:
    - cron: "0 0 * * *"
  steps:
    step-1:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: "python script1.py > output.txt"
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: []
      stepTimeout: 30
      clearImageCache: true
      repository:
        branch: main
        token: random_token
        url: "https://github.com/org/repo"
      outputParameters:
        scriptOutput: "output.txt"
    step-2:
      type: standard
      imageId: 100
      imageVersionId: 100
      command: "python script2.py"
      resources:
        instanceTypeId: 21
        storage: 10GB
      parents: ["step-1"]
      stepTimeout: 30
      clearImageCache: true
      repository:
        branch: main
        token: random_token
        url: "https://github.com/org/repo"
      executionParameters:
        conditional:
          - condition: "equals"
            paramName: "scriptOutput"
            stepName: "step-1"
            value: "output-value"
        parentStatus:
          - condition: "one-of"
            parents:
              - step-1
            status:
              - success

In this example, step-2 is configured to execute only if the output parameter scriptOutput of step-1 is equal to output-value and if the status of step-1 is success.

Executing a workflow

After creating a workflow, the next step is to execute it using the provided workflow ID. Dynamic parameters, defined as environment variables, can be passed to the steps during execution to enhance flexibility.

# execute_workflow.yaml

body:
  params:
    global:
      param1: value1
    stepWise:
      step1:
        param2: value2
      step2:
        param3: value3

Use the following command to execute the workflow:

peak workflows execute <workflow-id> path/to/execute_workflow.yaml

Partial Workflow Execution

If you don’t want to execute the entire workflow, you can selectively run specific steps or their dependencies. Provide the steps to be executed in the YAML configuration file, as shown below:

# execute_workflow.yaml

body:
  params:
    global:
      param1: value1
    stepWise:
      step1:
        param2: value2
      step2:
        param3: value3
  stepsToRun:
    - stepName: step1
      runDAG: true
    - stepName: step2

In this example, step1 and step2 are specified for execution. Note that runDAG is set to true for step1, which triggers the execution of all steps dependent on step1.

Retrieving Workflow Execution Details

When you run a workflow, you might want to get the details about the run - which step ran, which one failed, which one succeeded, etc. To get this detail, we have a command. Just run the following command to get the details about the execution

peak workflows get-execution-details --workflow-id=<workflow-id> --execution-id=<execution-id>

This returns the following details about the execution

{
    "executedAt": "2023-10-25T11:39:24.365Z",
    "finishedAt": "2023-10-25T11:39:37.342Z",
    "runId": "a0ac50ea-23ee-4256-bdd9-6a01a9bbe4f4",
    "status": "Failed",
    "steps": [
        {
            "childern": ["step-1"],
            "finishedAt": "2023-10-25T11:39:28.000Z",
            "name": "stepName",
            "startedAt": "2023-10-25T11:39:24.000Z",
            "status": "Failed",
            "stepType": "standard",
            "command": "echo Done!",
            "image": "<image-url>",
            "output": {
                "exitCode": "1",
                "message": "Error (exit code 1)"
            },
            "resources": {
                "instanceTypeId": "21",
                "storage": "40 GB"
            }
        },
        {
            "childern": [],
            "finishedAt": "2023-10-25T11:38:08.000Z",
            "name": "step-1",
            "startedAt": "2023-10-25T11:37:54.000Z",
            "status": "Failed",
            "stepType": "sqlQuery",
            "output": {
                "exitCode": "0"
            }
        }
    ]
}

As clear from the above example, the details returned for a Standard Step differs from the details for other types of steps (SQL Query in the above case). Steps other than Standard miss a few keys like - image, resources, and command.

Retrieving Workflow Step Execution Logs

To obtain execution logs for a specific step within a workflow, you’ll need to provide the workflow ID, execution ID, and the name of the step.

Using Next Token

To fetch execution logs, use the following command:

peak workflows get-execution-logs --workflow-id <workflow_id> --execution-id <execution_id> --step-name <step_name>

This command returns a set of logs along with a nextToken that can be used to fetch subsequent logs. The response looks like following:

{
    "logs": [
        {
            "timestamp": "1697089188119",
            "message": "log message 1"
        },
        {
            "timestamp": "1697089188119",
            "message": "log message 2"
        }
    ],
    "nextToken": "next_token",
    "stepRunStatus": "Success"
}

To retrieve the next set of logs, use the nextToken received in the response as follows:

peak workflows get-execution-logs --workflow-id <workflow_id> --execution-id <execution_id> --step-name <step_name> --next-token <next_token>

Polling the logs

To continuously poll the logs for a workflow step, you can use the --follow flag. This command will keep polling the logs until the execution is completed or all logs are fetched.

peak workflows get-execution-logs --workflow-id <workflow_id> --execution-id <execution_id> --step-name <step_name> --follow

Save the logs locally to a file

Workflow step execution logs can also be saved locally to a file using the --save option. We can also pass in a --file-name option which should include the file name and the path to used when saving the logs. If the --file-name option isn’t passed, the logs would be saved to a file named workflow_execution_logs_<workflow_id>_<execution_id>_<step_name>.log.

peak workflows get-execution-logs --workflow-id <workflow_id> --execution-id <execution_id> --step-name <step_name> --save --file-name <file_name>

Note The --save option would take the highest prioroty when both --follow and --save options are passed.

Webapps

Note: Only generic (EKS-based) webapps are supported with CLI.

Creating a webapp

We can create a webapp by providing payload like following.

# webapp.yaml

body:
  name: my-webapp
  title: New webapp
  description: This is a new webapp
  imageDetails:
    imageId: 100
    versionId: 100
  resources:
    instanceTypeId: 1
  sessionStickiness: true

We can use the following command to create a webapp:

peak webapps create path/to/create_webapp.yaml -v path/to/webapp_params.yaml

Alternatively, we can also use the create command without having the need to provide a yaml. This is possible using the command line options or optionally combining them with yaml template, with the command line options taking precedence.

peak webapps create --name my-webapp --description some-description --title my-title --image-id <image-id> --version-id <version-id> --instance-type-id 1 --session-stickiness

Above command will generate the following json body:

{
    "name": "my-webapp",
    "title": "my-title",
    "description": "some-description",
    "imageDetails": {
        "imageId": "<image-id>",
        "versionId": "<version-id>"
    },
    "resources": {
        "instanceTypeId": 1
    },
    "sessionStickiness": true
}

Updating a webapp

When updating the webapp, it will trigger a redeployment only under specific conditions. Redeployment is triggered if you make changes to any of the following parameters: imageId, versionId, instanceTypeId or sessionStickiness. However, only modifying the title or description will not trigger a redeployment.

We can update the webapp by providing webapp id and payload like following.

# webapp.yaml

body:
  title: Updated webapp
  description: This is an updated webapp
  imageDetails:
    imageId: 200
    versionId: 200
  resources:
    instanceTypeId: 1
  sessionStickiness: true

We can use the following command to update a webapp:

peak webapps update -v path/to/webapp_params.yaml <webapp-id> path/to/update_webapp.yaml

Alternatively, we can also use the update command without having the need to provide a yaml. This is possible using the command line options or optionally combining them with yaml template, with the command line options taking precedence.

peak webapps update <webapp-id> --description some-description --title my-title --image-id <image-id> --version-id <version-id> --instance-type-id 1 --session-stickiness

Above command will generate the following json body:

{
    "title": "my-title",
    "description": "some-description",
    "imageDetails": {
        "imageId": "<image-id>",
        "versionId": "<version-id>"
    },
    "resources": {
        "instanceTypeId": 1
    },
    "sessionStickiness": true
}

Using the `create-or-update` operation

The operation creates a new resource if it doesn’t exist and updates the resource in case it exists. The search is based on resource name field.

If the webapp is being updated, it will trigger a redeployment only under specific conditions. Redeployment is triggered if you make changes to any of the following parameters: imageId, versionId, instanceTypeId or sessionStickiness. However, only modifying the title or description will not trigger a redeployment.

Consider the following example for creating a webapp:

# webapp.yaml

body:
  title: New webapp
  description: This is an new  webapp
  imageDetails:
    imageId: 100
    versionId: 100
  resources:
    instanceTypeId: 1
  sessionStickiness: true

peak webapps create-or-update -v path/to/webapp_params.yaml <webapp-id> path/to/create_or_update_webapp.yaml --image-id 3 --name my-webapp

Now we could update the webapp created with above command as follows:

# webapp_params.yaml

imageId: 2
versionId: 2

# webapp.yaml
body:
    title: Updated webapp
    description: This is an updated webapp
    imageDetails:
        imageId: {{ imageId }}
        versionId: {{ versionId }}
    resources:
        instanceTypeId: 1
    sessionStickiness: true

peak webapps create-or-update -v path/to/webapp_params.yaml <webapp-id> path/to/webapp.yaml --image-id 3 --name my-webapp --instance-type-id 1 --session-stickiness

In the above example, the imageId will be overwritten by the command line options and the final webapp body will look like:

{
    "title": "Updated webapp",
    "description": "This is an updated webapp",
    "imageDetails": {
        "imageId": 3,
        "versionId": 2
    },
    "resources": {
        "instanceTypeId": 1
    },
    "sessionStickiness": true
}

Using Images, Workflows and Webapps via CLI

Tenants

Defining Parameters in Resource Creation

Using a combination of command line options and yaml

Creating Resource Dependencies with Artifacts

Providing Instance Type and Storage in Workflow

Providing Instance Type in Webapp

Session Stickiness in Webapp

Images

Creating an image

Creating an image version

Updating an image version

Using the create-or-update operation

Deleting Images and Versions

Retrieving Image Build Logs

Workflows

Creating a workflow

Updating a workflow

Using the create-or-update operation

Partial Workflow Update using patch workflow

Setting up advanced workflow configurations

Auto retry configuration

Skippable steps configuration

Input parameters configuration

Output parameters configuration

Execution condition configuration

Executing a workflow

Partial Workflow Execution

Retrieving Workflow Execution Details

Retrieving Workflow Step Execution Logs

Webapps

Creating a webapp

Updating a webapp

Using the create-or-update operation

Using `Images`, `Workflows` and `Webapps` via CLI

Using the `create-or-update` operation

Using the `create-or-update` operation

Partial Workflow Update using `patch` workflow

Using the `create-or-update` operation