From bbb5cc591056714129ad88aa75c35db689172dc2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tobi=C3=A1=C5=A1=20Poto=C4=8Dek?= Date: Mon, 25 Aug 2025 12:00:16 +0200 Subject: [PATCH 1/2] feat: increase rate limits --- apify-api/openapi/openapi.yaml | 6 +++--- sources/platform/storage/dataset.md | 4 ++-- sources/platform/storage/request_queue.md | 4 ++-- sources/platform/storage/usage.md | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/apify-api/openapi/openapi.yaml b/apify-api/openapi/openapi.yaml index af19984805..6d947ef44c 100644 --- a/apify-api/openapi/openapi.yaml +++ b/apify-api/openapi/openapi.yaml @@ -398,18 +398,18 @@ info: ### Per-resource rate limit - The default per-resource rate limit is _30 requests per second per resource_, which in this context means a single Actor, a single Actor run, a single dataset, single key-value store etc. + The default per-resource rate limit is _60 requests per second per resource_, which in this context means a single Actor, a single Actor run, a single dataset, single key-value store etc. The default rate limit is applied to every API endpoint except a few select ones, which have higher rate limits. Each API endpoint returns its rate limit in `X-RateLimit-Limit` header. - These endpoints have a rate limit of _100 requests per second per resource_: + These endpoints have a rate limit of _200 requests per second per resource_: * CRUD ([get](#/reference/key-value-stores/record/get-record), [put](#/reference/key-value-stores/record/put-record), [delete](#/reference/key-value-stores/record/delete-record)) operations on key-value store records - These endpoints have a rate limit of _200 requests per second per resource_: + These endpoints have a rate limit of _400 requests per second per resource_: * [Run Actor](#/reference/actors/run-collection/run-actor) * [Run Actor task asynchronously](#/reference/actor-tasks/runs-collection/run-task-asynchronously) * [Run Actor task synchronously](#/reference/actor-tasks/runs-collection/run-task-synchronously) diff --git a/sources/platform/storage/dataset.md b/sources/platform/storage/dataset.md index 4178fdfec7..1f580d01fe 100644 --- a/sources/platform/storage/dataset.md +++ b/sources/platform/storage/dataset.md @@ -450,8 +450,8 @@ See the [Storage overview](/platform/storage/usage#sharing-storages-between-runs ### Rate limiting -The rate limit for pushing data to a dataset through the [API](/api/v2/dataset-items-post) is capped at _200 requests per second_ for each dataset, a measure to prevent overloading Apify servers. +The rate limit for pushing data to a dataset through the [API](/api/v2/dataset-items-post) is capped at _400 requests per second_ for each dataset, a measure to prevent overloading Apify servers. -For all other dataset [API endpoints](/api/v2/storage-datasets) , the rate limit is _30 requests per second_ for each dataset. +For all other dataset [API endpoints](/api/v2/storage-datasets) , the rate limit is _60 requests per second_ for each dataset. Check out the [API documentation](/api/v2#rate-limiting) for more information and guidance on actions to take if you exceed these rate limits. diff --git a/sources/platform/storage/request_queue.md b/sources/platform/storage/request_queue.md index b0bd072259..64db8ec128 100644 --- a/sources/platform/storage/request_queue.md +++ b/sources/platform/storage/request_queue.md @@ -627,8 +627,8 @@ CRUD ([add](/api/v2/request-queue-requests-post), [get](/api/v2/request-queue-request-get), [update](/api/v2/request-queue-request-put), [delete](/api/v2/request-queue-request-delete)) -operation requests are limited to _200 requests per second_ per request queue. This helps protect Apify servers from being overloaded. +operation requests are limited to _400 requests per second_ per request queue. This helps protect Apify servers from being overloaded. -All other request queue API [endpoints](/api/v2/storage-request-queues) are limited to _30 requests per second_ per request queue. +All other request queue API [endpoints](/api/v2/storage-request-queues) are limited to _60 requests per second_ per request queue. Check out the [API documentation](/api/v2#rate-limiting) for more information and guidance on actions to take if you exceed these rate limits. diff --git a/sources/platform/storage/usage.md b/sources/platform/storage/usage.md index a77d4fe1c2..ee1a57c25d 100644 --- a/sources/platform/storage/usage.md +++ b/sources/platform/storage/usage.md @@ -107,7 +107,7 @@ Use this tool to estimate storage costs by plan and storage type. ## Rate limiting -All API endpoints limit their rate of requests to protect Apify servers from overloading. The default rate limit for storage objects is _30 requests per second_. However, there are exceptions limited to _200 requests per second_ per storage object, including: +All API endpoints limit their rate of requests to protect Apify servers from overloading. The default rate limit for storage objects is _60 requests per second_. However, there are exceptions limited to _400 requests per second_ per storage object, including: * [Push items](/api/v2/dataset-items-post) to dataset. * CRUD ([add](/api/v2/request-queue-requests-post), From e364944f1cf883a4930386704732efab0afd0cb2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tobi=C3=A1=C5=A1=20Poto=C4=8Dek?= Date: Wed, 27 Aug 2025 18:39:34 +0200 Subject: [PATCH 2/2] chore: CR feedback --- apify-api/openapi/openapi.yaml | 2 +- sources/platform/storage/dataset.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/apify-api/openapi/openapi.yaml b/apify-api/openapi/openapi.yaml index 6d947ef44c..7ddfa96cf4 100644 --- a/apify-api/openapi/openapi.yaml +++ b/apify-api/openapi/openapi.yaml @@ -372,7 +372,7 @@ info: 429 rate-limit-exceeded - You have exceeded the rate limit of 30 requests per second + You have exceeded the rate limit of ... requests per second 405 diff --git a/sources/platform/storage/dataset.md b/sources/platform/storage/dataset.md index 1f580d01fe..f966ba800b 100644 --- a/sources/platform/storage/dataset.md +++ b/sources/platform/storage/dataset.md @@ -93,7 +93,7 @@ To add data to a dataset, issue a POST request to the [Put items](/api/v2/datase https://api.apify.com/v2/datasets/{DATASET_ID}/items ``` -> API data push to a dataset is capped at _200 requests per second_ to avoid overloading our servers. +> API data push to a dataset is capped at _400 requests per second_ to avoid overloading our servers. Example payload: