From eba13181ee467487195df7065b4477564ccbb920 Mon Sep 17 00:00:00 2001 From: stevenmeek Date: Sun, 7 Sep 2025 21:15:39 +0100 Subject: [PATCH 1/6] docs: Add page for kestra integration. --- sources/platform/integrations/index.mdx | 6 + .../workflows-and-notifications/kestra.md | 108 ++++++++++++++++++ static/img/platform/integrations/kestra.svg | 17 +++ 3 files changed, 131 insertions(+) create mode 100644 sources/platform/integrations/workflows-and-notifications/kestra.md create mode 100644 static/img/platform/integrations/kestra.svg diff --git a/sources/platform/integrations/index.mdx b/sources/platform/integrations/index.mdx index ef2bd39738..59c121d1af 100644 --- a/sources/platform/integrations/index.mdx +++ b/sources/platform/integrations/index.mdx @@ -134,6 +134,12 @@ If you use one of the main integration platforms, Apify's support is here for yo imageUrl="/img/platform/integrations/n8n.svg" smallImage /> + + + + + + + + + + + + + + + + + From 8c99c25ab191c9bb55be25b024eab5f26f9870bf Mon Sep 17 00:00:00 2001 From: stevenmeek Date: Wed, 10 Sep 2025 21:12:11 +0100 Subject: [PATCH 2/6] docs: Fix grammar and spelling mistakes for Kestra integration docs. --- .../workflows-and-notifications/kestra.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/sources/platform/integrations/workflows-and-notifications/kestra.md b/sources/platform/integrations/workflows-and-notifications/kestra.md index 1e32102ae0..471b7c4cf3 100644 --- a/sources/platform/integrations/workflows-and-notifications/kestra.md +++ b/sources/platform/integrations/workflows-and-notifications/kestra.md @@ -1,26 +1,26 @@ --- title: Kestra integration -description: Connect Apify with Kestra to orchestrate workflows — run Flows, extract structured data, and react to Actor or task events in real time. +description: Connect Apify with Kestra to orchestrate workflows — run flows, extract structured data, and react to Actor or task events in real time. sidebar_label: Kestra sidebar_position: 7 slug: /integrations/kestra --- -**Connect Apify with Kestra to orchestrate workflows — run Flows, extract structured data, and react to Actor or task events in real time.** +**Connect Apify with Kestra to orchestrate workflows — run flows, extract structured data, and react to Actor or task events in real time.** --- [Kestra](https://kestra.io/) is an open-source, event-driven orchestration platform that unifies workflows for all engineers. -With the [Apify plugin for Kestra](https://github.com/apify/kestra-plugin-apify), you can seamlessly connect Apify Actors and storage to your workflows. +With the [Apify plugin for Kestra](https://github.com/kestra-io/plugin-kestra), you can seamlessly connect Apify Actors and storage to your workflows. Run scrapers, extract structured data — all defined declaratively in YAML and orchestrated directly from the UI. -In this guide, you'll learn how to set up authentication and incorporate it into a Kestra flows. +In this guide, you'll learn how to set up and incorporate authentication into a Kestra flows. ## Prerequisites Before you begin, make sure you have: - An [Apify account](https://console.apify.com/) -- An [Kestra instance](https://kestra.io/docs/getting-started/quickstart) (self‑hosted or cloud) +- A [Kestra instance](https://kestra.io/docs/getting-started/quickstart) (self‑hosted or cloud) Once installed, the next step is authentication. @@ -50,15 +50,15 @@ or fetching data. Tasks allow you to perform operations like running an Actor within a workflow. 1. Create a new flow. -1. Inside the **Flow code** tab change the hello tasks type to be **io.kestra.plugin.apify.actor.Run**. -1. Change the tasks id to be **run_apify_actor** +1. Inside the **Flow code** tab change the hello task's type to be **io.kestra.plugin.apify.actor.Run**. +1. Change the task's id to be **run_apify_actor** 1. Remove the message property. 1. Configure the **run_apify_actor** task by adding your required values for the properties listed below: - **actor id**: Actor ID or a tilde-separated owner's username and Actor name. - **apiToken**: A reference to the secret value you set up earlier. For example "\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}" 1. Add a new task below the **run_apify_actor** with an ID of **get_dataset** and a type of **io.kestra.plugin.apify.dataset.Get**.: 1. Configure the **get_dataset** to fetch the dataset generated by the **run_apify_actor** task by configuring the following values: - - **datasetId**: The Id of the dataset to fetch. You can use the value from the previous task using the following syntax: "\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}" + - **datasetId**: The ID of the dataset to fetch. You can use the value from the previous task using the following syntax: "\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}" - **apiToken**: A reference to the secret value you set up earlier. For example '\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}' - **input**: Input for the Actor run. The input is optional and can be used to pass data to the Actor. For our example we will add 'hashtags: ["fyp"]' - **maxItems**: The maximum number of items to fetch from the dataset. For our example we will set this to 5. From 495bce12a491759fea03f50f1a7efd9a61628a87 Mon Sep 17 00:00:00 2001 From: stevenmeek Date: Thu, 11 Sep 2025 19:30:23 +0100 Subject: [PATCH 3/6] docs: Add link to Kestra KV store. --- .../integrations/workflows-and-notifications/kestra.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/sources/platform/integrations/workflows-and-notifications/kestra.md b/sources/platform/integrations/workflows-and-notifications/kestra.md index 471b7c4cf3..cf430e0e0f 100644 --- a/sources/platform/integrations/workflows-and-notifications/kestra.md +++ b/sources/platform/integrations/workflows-and-notifications/kestra.md @@ -28,8 +28,12 @@ Once installed, the next step is authentication. The Apify plugin for Kestra uses **API Key authentication** to securely connect with your Apify account. This method works for both self-hosted and cloud instances. -You can store and manage your API key directly in [Kestra Secrets](https://kestra.io/docs/concepts/secret) in the UI or through environment variables, ensuring -credentials stay secure and flows remain declarative. + +You can store and manage your API key directly in [Kestra Secrets}(https://kestra.io/docs/concepts/secret) +in the UI or through environment variables, ensuring credentials stay secure and flows remain declarative. In the open-source version, Secrets can be managed using base64-encoded environment variables. + +For open-source deployments, you can also use [Kestra’s KV Store](https://kestra.io/docs/concepts/kv-store) +to persist and share API keys (or other values) across executions and workflows. With authentication configured, you can now define flows that integrate the Apify plugin — orchestrating scrapers, data extraction, and event-driven tasks directly from Kestra. From 246010746e0552eb5cbdeca7d069b215d9d17c80 Mon Sep 17 00:00:00 2001 From: Steven Meek <42916669+ammeek@users.noreply.github.com> Date: Fri, 12 Sep 2025 19:59:04 +0100 Subject: [PATCH 4/6] Apply suggestions from code review docs: fix typos in docs. Co-authored-by: AJ Emerich --- .../integrations/workflows-and-notifications/kestra.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/sources/platform/integrations/workflows-and-notifications/kestra.md b/sources/platform/integrations/workflows-and-notifications/kestra.md index cf430e0e0f..8f1be29e2c 100644 --- a/sources/platform/integrations/workflows-and-notifications/kestra.md +++ b/sources/platform/integrations/workflows-and-notifications/kestra.md @@ -29,7 +29,7 @@ Once installed, the next step is authentication. The Apify plugin for Kestra uses **API Key authentication** to securely connect with your Apify account. This method works for both self-hosted and cloud instances. -You can store and manage your API key directly in [Kestra Secrets}(https://kestra.io/docs/concepts/secret) +You can store and manage your API key directly in [Kestra Secrets](https://kestra.io/docs/concepts/secret) in the UI or through environment variables, ensuring credentials stay secure and flows remain declarative. In the open-source version, Secrets can be managed using base64-encoded environment variables. For open-source deployments, you can also use [Kestra’s KV Store](https://kestra.io/docs/concepts/kv-store) @@ -58,17 +58,16 @@ Tasks allow you to perform operations like running an Actor within a workflow. 1. Change the task's id to be **run_apify_actor** 1. Remove the message property. 1. Configure the **run_apify_actor** task by adding your required values for the properties listed below: - - **actor id**: Actor ID or a tilde-separated owner's username and Actor name. + - **actorId**: Actor ID or a tilde-separated owner's username and Actor name. - **apiToken**: A reference to the secret value you set up earlier. For example "\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}" 1. Add a new task below the **run_apify_actor** with an ID of **get_dataset** and a type of **io.kestra.plugin.apify.dataset.Get**.: 1. Configure the **get_dataset** to fetch the dataset generated by the **run_apify_actor** task by configuring the following values: - **datasetId**: The ID of the dataset to fetch. You can use the value from the previous task using the following syntax: "\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}" - - **apiToken**: A reference to the secret value you set up earlier. For example '\{\{secret(namespace=flow.namespace, key='APIFY_API_KEY')\}\}' - **input**: Input for the Actor run. The input is optional and can be used to pass data to the Actor. For our example we will add 'hashtags: ["fyp"]' - **maxItems**: The maximum number of items to fetch from the dataset. For our example we will set this to 5. 1. Now add the final task to log the output of the dataset. Add a new task below the **log_output** with an ID of **log_output** and a type of **io.kestra.plugin.core.log.Log**. 1. Configure the **log_output** task to log the output of the dataset by configuring the following values: - - **message** The message to log. You can use the value from the previous task using the following syntax: '\{\{outputs.get_dataset.dataset\}\}' + - **message**: The message to log. You can use the value from the previous task using the following syntax: '\{\{outputs.get_dataset.dataset\}\}' 1. Now save and run your flow. Your completed template should match the template below. From 3e262b813ec3f2608b6569de8073fad9654bff63 Mon Sep 17 00:00:00 2001 From: stevenmeek Date: Fri, 12 Sep 2025 20:01:26 +0100 Subject: [PATCH 5/6] docs: update the id of the Kestra flow example to reflect the actual use case. --- .../platform/integrations/workflows-and-notifications/kestra.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/platform/integrations/workflows-and-notifications/kestra.md b/sources/platform/integrations/workflows-and-notifications/kestra.md index 8f1be29e2c..02685118c6 100644 --- a/sources/platform/integrations/workflows-and-notifications/kestra.md +++ b/sources/platform/integrations/workflows-and-notifications/kestra.md @@ -72,7 +72,7 @@ Tasks allow you to perform operations like running an Actor within a workflow. Your completed template should match the template below. ```yaml -id: get_last_actor_run_and_get_dataset +id: run_actor_and_fetch_dataset namespace: company.team tasks: From f0192263ca331608429612e7d1f48c919758fb79 Mon Sep 17 00:00:00 2001 From: stevenmeek Date: Fri, 12 Sep 2025 21:00:35 +0100 Subject: [PATCH 6/6] docs: add reference to Kestra on the Integrate with Apify page. --- sources/platform/integrations/integrate_with_apify.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sources/platform/integrations/integrate_with_apify.md b/sources/platform/integrations/integrate_with_apify.md index b0c782a266..829e431ce3 100644 --- a/sources/platform/integrations/integrate_with_apify.md +++ b/sources/platform/integrations/integrate_with_apify.md @@ -187,6 +187,9 @@ For inspiration, check out the public repositories of Apify's existing external - [Source code on Github](https://github.com/apify/apify-zapier-integration) - Make.com - [Make.com integration documentation](https://docs.apify.com/platform/integrations/make) +- Kestra + - [kestra integration documentation](https://kestra.io/plugins/plugin-apify) + - [Source code on Github](https://github.com/kestra-io/plugin-apify) - Keboola - [Keboola integration documentation](https://docs.apify.com/platform/integrations/keboola) - [Source code on GitHub](https://github.com/apify/keboola-ex-apify/) (JavaScript)