Creating Peak Resources via SDK
To start creating Images, Workflows, Service and Web Apps through SDK, we first need to instantiate their respective clients.
The Reference Documentation for these resources can be found here and the usage examples can be found here.
API Documentation is linked with every function in the Reference Documentation which would help in understanding the schema of the payload.
Creating Resource Dependencies with Artifacts
If we want to create a resource such as image, but we don’t want it to be source directly from the code repository, we can package the required files for image creation into a zip archive known as an Artifact.
More information for Artifact can be found in Artifact and Compression Doc.
Providing Instance Type and Storage in Workflow
When defining a workflow step, we have the option to set the instance type and storage by including them in the resources key in the following format:
"resources": {
"instanceTypeId": 23,
"storage": "20GB"
}
A list of all available instances along with their corresponding instanceTypeId can be obtained using the list-instance-options function.
Adding the resources section is optional. If we don’t specify it for a particular step, the default values will be used. We can retrieve the default values by using get_default_resources function.
Providing Instance Type in Services
When creating a web-app, we have the option to set the instance type by including them in the resources key in the following format:
"resources": {
"instanceTypeId": 1,
}
Session Stickiness in Services (Not required for API type services)
By default, session stickiness is disabled (false). You can activate session stickiness by setting the sessionStickiness parameter to true.
Session stickiness ensures that each user’s requests are consistently directed to a particular server. This feature is especially valuable for stateful applications like web applications that rely on server-stored session data.
Note that employing session stickiness may lead to unpredictable behavior and is not recommended if you plan to scale your application.
Scale To Zero in Service Resource
This is only applicable for Service Resource of kind web-app.
By default, this option is disabled (false). You can activate it by setting the scaleToZero parameter to true.
Setting Scale to zero ensures that the resources hosting your app are scaled down when idle over a period of time. The resources will scale back up automatically on next launch.
This option is not available for shiny web app types and also does not work if sessionStickiness is set to true.
Images
Creating an image
We can create an image by using create_image function. We need to provide the payload containing the configuration of the image. If version is not provided in the request body, 0.0.1 would be used as the default version.
image_body = {
"version": "0.0.1",
"name": "image-sdk-101",
"description": "Hello from SDK",
"type": "workflow",
"buildDetails": {
"source": "upload",
"useCache": False,
"buildArguments": [
{
"name": "HELLO",
"value": "world",
},
],
"context": ".",
},
}
image = image_client.create_image(
body=image_body,
artifact={"path": "../peak", "ignore_files": [".dockerignore"]},
)
This returns the imageId and versionId of the newly created image which can be used to create a workflow or a web-app.
{
"buildId": 1,
"imageId": 1,
"versionId": 1
}
buildDetails and source are optional. By default, source will be set to upload. If buildDetails is not provided, it will be set to following:
{
"source": "upload"
}
Creating an image version
Once an image is created, we can create its subsequent version by using create_version function. We need to provide the image id and payload containing the configuration of the image version. If version is not provided in the request body, the version will be automatically incremented. If source is not provided in the build details, it will be set to upload.
version_body = {
"version": "0.0.2",
"buildDetails": {
"source": "upload",
"useCache": False,
"buildArguments": [
{
"name": "HELLO",
"value": "again",
},
],
"context": ".",
},
}
image_version = image_client.create_version(
image_id=1,
body=version_body,
artifact={"path": "../peak", "ignore_files": [".dockerignore"]},
)
This returns the imageId and versionId of the newly created image version which can be used to create a workflow or a web-app.
{
"buildId": 2,
"imageId": 1,
"versionId": 2
}
buildDetails and source are optional. By default, source will be set to upload. If buildDetails is not provided, it will be set to following:
{
"source": "upload"
}
Workflows
Creating a workflow
With the help of above created image versions, we can create a workflow by using create_workflow function. We need to provide the payload containing the configuration of the workflow.
Triggers can be one of the following: Time based, Webhook based or Manual.
// For Time Based Trigger, cron expression is required
"triggers": [
{
"cron": "0 0 * * *",
},
]
// For Webhook Based Trigger, we need to provide webhook key which is a boolean
"triggers": [
{
"webhook": true,
},
]
// For Manual Trigger, we can either skip this key or provide an empty list
"triggers": []
workflow_body = {
"name": "new-workflow",
"triggers": [
{
"cron": "0 0 * * *",
},
],
"watchers": [
{
"user": "abc@peak.ai",
"events": {
"success": False,
"fail": True,
},
},
{
"webhook": {
"name": "info",
"url": "https://abc.com/post",
"payload": '{ "pingback-url": "https:/workflow/123" }',
},
"events": {
"success": False,
"fail": True,
"runtimeExceeded": 10,
},
},
{
"email": {
"name": "email-watcher-1",
"recipients": {
"to": ["user1@peak.ai", "user2@peak.ai"],
},
},
"events": {
"success": False,
"fail": True,
"runtimeExceeded": 10,
},
},
],
"tags": [
{
"name": "foo",
},
{
"name": "bar",
},
],
"steps": {
"stepName": {
"type": "standard",
"imageId": 1,
"imageVersionId": 1,
"command": "python script.py",
"resources": {
"instanceTypeId": 21,
"storage": "10GB",
},
"parents": [],
"stepTimeout": 30,
"clearImageCache": True,
"parameters": {
"env": {
"key": "value",
},
"inherit": {},
"secrets": [
"secret-1",
"secret-2",
],
},
"repository": {
"branch": "main",
"token": "github",
"url": "https://github.com/org/repo",
},
},
"stepName2": {
"type": "http",
"method": "get",
"url": "https://peak.ai",
"auth": {
"type": "oauth",
"clientId": "client-id",
"clientSecret": "client-secret",
"authUrl": "https://get-access-token",
},
"payload": "{}",
"headers": {
"absolute": {
"x-auth-tenant": "some-tenant",
},
"secrets": {
"x-auth-token": "token",
},
},
},
"stepName3": {
"type": "sql",
"sqlQueryPath": "path/to/sql",
"parameters": {
"env": {
"key": "value",
},
"inherit": {
"key": "value",
},
},
},
"stepName4": {
"type": "export",
"parents": ["stepName3"],
"schema": "schema_name",
"table": "table_name",
"sortBy": "column_name",
"sortOrder": "asc",
"compression": False,
},
},
}
workflow = workflow_client.create_workflow(workflow_body)
It returns the workflowId of the newly created workflow which can be used to execute the workflow.
{
"id": 1
}
Executing a workflow
Once a workflow is created, we can run or execute it by providing workflow id. We can provide dynamic parameters that needs to be passed as environment variables to steps. We can execute the workflow by using execute_workflow function.
# Optional parameters
body = {
"params": {
"global": {
"key": "value",
},
"stepWise": {
"step1": {
"key1": "value1",
},
},
},
}
workflow_client.execute_workflow(workflow_id=1, body=body)
It returns the executionId of the newly executed workflow.
{
"executionId": "d6116a56-6b1d-41b4-a599-fb949f08863f"
}
Web Apps
Note: The functionalities provided through the
Webappfeature are deprecated and will eventually be removed. We recommend using theServicefeature which offers a more comprehensive and flexible solution for managing web applications and API deployments.
Note: Only generic (EKS-based) Web Apps are supported with SDK.
Creating a Web App
With the help of previously created image versions, we can create a web-app by using create_webapp function. We need to provide the payload containing the configuration of the web-app.
# webapp.yaml.j2
body = {
"name": "webapp101",
"title": "Hello from SDK",
"description": "Hi there",
"imageDetails": {"imageId": 1, "versionId": 1},
"resources": {
"instanceTypeId": 1,
},
"scaleToZero": True,
"sessionStickiness": False,
}
webapp = webapp_client.create_webapp(body)
It returns the id of the newly created web-app.
{
"id": "84a41de7-d67f-4aa0-aebe-83c1474f0eaf"
}
Services
Creating a service
With the help of previously created image versions, we can create a service by using create_service function. We need to provide the payload containing the configuration of the service.
# service.yaml.j2
body = {
"name": "service101",
"title": "Hello from SDK",
"description": "Hi there",
"serviceType": "web-app",
"imageDetails": {"imageId": 1, "versionId": 1},
"resources": {
"instanceTypeId": 1,
},
"parameters": {
"env": {
"arg1": "value1",
"arg2": "value2",
},
"inherit": {},
"secrets": ["secret-1", "secret"],
},
"entrypoint": "python\nscript.py",
"healthCheckURL": "/health",
"minInstances": 1,
"scaleToZero": True,
"sessionStickiness": False, # Not required for API type services
}
service = service_client.create_service(body)
It returns the id of the newly created service.
{
"id": "84a41de7-d67f-4aa0-aebe-83c1474f0eaf"
}
Launching Web App or Shiny type Service
The application URL can be obtained by using describe_service function for the service.
service = service_client.describe_service(
service_id="84a41de7-d67f-4aa0-aebe-83c1474f0eaf"
)
It returns the service details along with the application URL which can be used to launch web-app or shiny type services.
Testing API type Service
We can test an API type service to verify it’s health and if it is working.
We can do so by using test_service function. We need to provide the service id, http method, path and payload to be sent in the request.
response = service_client.test_service(
service_id="84a41de7-d67f-4aa0-aebe-83c1474f0eaf",
http_method="post",
path="/health",
payload={"key": "value"},
)
It returns the response from the service endpoint.
{
"reqEndTime": "2024-01-01T00:00:05.000Z",
"reqStartTime": "2024-01-01T00:00:00.000Z",
"responseBody": "Hello from the service!",
"responseSize": "25 B",
"responseStatus": 200
}