Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed public/images/aws/extensions-library.png
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Replicating cloud resources locally with LocalStack's AWS Replicator extension"
description: Learn how you can replicate cloud resources in your local environment using the LocalStack's AWS Replicator extension. This tutorial provides step-by-step guidance on setting up and leveraging the AWS Replicator extension to mirror cloud services locally, enabling efficient hybrid development and testing workflows without maintaining additional configurations.
title: "Replicating cloud resources locally with LocalStack's AWS Cloud Proxy extension"
description: Learn how you can replicate cloud resources in your local environment using the LocalStack's AWS Cloud Proxy extension. This tutorial provides step-by-step guidance on setting up and leveraging the AWS Cloud Proxy extension to mirror cloud services locally, enabling efficient hybrid development and testing workflows without maintaining additional configurations.
services:
- sqs
- lmb
Expand All @@ -9,7 +9,7 @@ platform:
deployment:
- awscli
pro: true
leadimage: "aws-replicator-extension-tutorial-cover.png"
leadimage: "aws-proxy-extension-tutorial-cover.png"
---

## Introduction
Expand All @@ -19,13 +19,13 @@ This allows you to work on and test your cloud-based solutions without needing t
However, sometimes you might need to smoothly switch between your local setup and actual cloud resources, especially in hybrid scenarios.
This could be useful, for example, if you want to share a database with your local Lambda function, or if you require access to S3 files stored remotely while running a Glue ETL job locally.

With the [AWS Replicator extension](https://github.com/localstack/localstack-extensions/tree/main/aws-replicator), you can:
With the [AWS Cloud Proxy extension](https://github.com/localstack/localstack-extensions/tree/main/aws-proxy), you can:

- Enable your local environment to mirror AWS cloud resources at the API level, allowing for direct interaction with cloud services.
- Facilitate the forwarding of specific requests from LocalStack to AWS without the need for complex proxy setups.
- Support scenarios that require a combination of local and cloud resources, such as testing cloud services with local databases or functions.

In this tutorial, you will learn how to install the AWS Replicator extension and utilize its different modes to seamlessly work in a hybrid environment.
In this tutorial, you will learn how to install the AWS Cloud Proxy extension and utilize its different modes to seamlessly work in a hybrid environment.

## Prerequisites

Expand All @@ -35,17 +35,17 @@ In this tutorial, you will learn how to install the AWS Replicator extension and
- [LocalStack Web Application account](https://app.localstack.cloud/sign-up)
- [AWS Account](https://aws.amazon.com/) with an [`AWS_ACCESS_KEY_ID` & `AWS_SECRET_ACCESS_KEY`](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey)

## Install the AWS Replicator extension
## Install the AWS Cloud Proxy extension

To install the AWS Replicator Extension, follow these steps:
To install the AWS Cloud Proxy Extension, follow these steps:

1. Launch your LocalStack container using the `localstack` CLI, ensuring that `LOCALSTACK_AUTH_TOKEN` is available in the environment.
2. Visit the [Extensions library](https://app.localstack.cloud/extensions/library) page on the LocalStack Web Application.
![Extensions Library](/images/aws/extensions-library.png)
3. Scroll down to find the **AWS replicator** card, then click on the **Install on Instance** button.
![Installing AWS Replicator extension](/images/aws/installing-aws-replicator-extensions.png)
![Extensions Library](/images/aws/aws-proxy-tutorial/extensions-library.png)
3. Scroll down to find the **AWS Cloud Proxy** card, then click on the **Install on Instance** button.
![Installing AWS Cloud Proxy extension](/images/aws/aws-proxy-tutorial/installing-aws-proxy-extensions.png)

Once the installation is complete, you will notice that your LocalStack container has restarted with the AWS Replicator extension successfully installed.
Once the installation is complete, you will notice that your LocalStack container has restarted with the AWS Cloud Proxy extension successfully installed.
To confirm the installation, execute the following command:

```bash
Expand All @@ -56,14 +56,14 @@ localstack extensions list
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name ┃ Summary ┃ Version ┃ Author ┃ Plugin name ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ localstack-extension-aws-re… │ LocalStack Extension: AWS │ 0.1.11 │ LocalStack Team │ aws-replicator
│ │ replicator │ │ │ │
│ localstack-extension-aws-re… │ LocalStack Extension: AWS │ 0.1.11 │ LocalStack Team │ aws-proxy
│ │ Cloud Proxy │ │ │ │
└──────────────────────────────┴──────────────────────────────┴─────────┴─────────────────┴────────────────┘
```

After verifying the successful installation, you can shut down the LocalStack container to re-start it with additional configuration variables.

## Tutorial: Working with the AWS Replicator Extension
## Tutorial: Working with the AWS Cloud Proxy Extension

In this tutorial, you will set up a basic example consisting of:

Expand All @@ -74,23 +74,23 @@ In this tutorial, you will set up a basic example consisting of:
The basic architecture for the scenario is outlined in the figure below.
It shows the relationship between the resources deployed in the LocalStack container, the LocalStack AWS Proxy, and the remote AWS account.

![AWS Replicator sample use case](/images/aws/aws-proxy-sqs-lambda-sample.png)
![AWS Cloud Proxy sample use case](/images/aws/aws-proxy-sqs-lambda-sample.png)

In the following sections, you will create the SQS queue on your local machine and the remote cloud to showcase how you can switch between the two with the AWS Replicator extension.
In the following sections, you will create the SQS queue on your local machine and the remote cloud to showcase how you can switch between the two with the AWS Cloud Proxy extension.

### Create the Lambda function

Begin by running your LocalStack container with the following configuration:

```bash
EXTRA_CORS_ALLOWED_ORIGINS=https://aws-replicator.localhost.localstack.cloud:4566 \
EXTRA_CORS_ALLOWED_ORIGINS=https://aws-proxy.localhost.localstack.cloud:4566 \
DEBUG=1 \
localstack start
```

In the above command:

- The `EXTRA_CORS_ALLOWED_ORIGINS` variable allows the AWS Replicator extension's web interface to connect with the LocalStack container.
- The `EXTRA_CORS_ALLOWED_ORIGINS` variable allows the AWS Cloud Proxy extension's web interface to connect with the LocalStack container.
- The `DEBUG` variable enables verbose logging allowing you to see the printed statements from the Lambda function.

Next, create a file named `testlambda.py` and add the following Python code to it:
Expand Down Expand Up @@ -138,7 +138,7 @@ awslocal sqs create-queue --queue-name test-queue
}
```

Additionally, you can create the remote SQS queue on the real AWS cloud to test invocation after starting the AWS Replicator extension.
Additionally, you can create the remote SQS queue on the real AWS cloud to test invocation after starting the AWS Cloud Proxy extension.

Use the following command to set up the SQS queue on AWS:

Expand Down Expand Up @@ -194,12 +194,11 @@ In the LocalStack logs, you will see confirmation of the Lambda function invocat
2024-03-26T07:23:47.842 DEBUG --- [5119b27cdf1e] l.s.l.i.version_manager : [func1-381c6f7c-3ad8-4c79-aad8-5119b27cdf1e] END RequestId: 381c6f7c-3ad8-4c79-aad8-5119b27cdf1e
```

### Run the AWS Replicator extension
### Run the AWS Cloud Proxy extension

To run the AWS Replicator extension:
To run the AWS Cloud Proxy extension:

- Access [`https://aws-replicator.localhost.localstack.cloud:4566`](https://aws-replicator.localhost.localstack.cloud:4566/) via your web browser.
![AWS Replicator extension](/images/aws/aws-replicator-extension.png)
- Access [`https://aws-proxy.localhost.localstack.cloud:4566`](https://aws-proxy.localhost.localstack.cloud:4566/) via your web browser.
- Provide your AWS Credentials: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optionally `AWS_SESSION_TOKEN`.
- Add a new YAML-based Proxy configuration to proxy requests for specific resources to AWS.
For this scenario, configure it to proxy requests for the SQS queue created earlier.
Expand All @@ -211,9 +210,8 @@ To run the AWS Replicator extension:
- '.*:test-queue'
```

- Save the configuration to enable the AWS Replicator extension.
- Save the configuration to enable the AWS Cloud Proxy extension.
Once enabled, you will see the proxy status as **enabled**.
![Enabled AWS Replicator extension](/images/aws/enabled-aws-replicator-extension.png)

To invoke the local Lambda function with the remote SQS queue:

Expand All @@ -231,14 +229,14 @@ You will observe the local Lambda function being invoked once again, with corres

You can even run the standard `awslocal` commands in your terminal that would query the remote cloud resources, instead of the local ones.

Upon completion, you can click **Disable** on the AWS Replicator extension web interface to deactivate the proxy configuration.
Upon completion, you can click **Disable** on the AWS Cloud Proxy extension web interface to deactivate the proxy configuration.
Additionally, you can delete the remote SQS queue to avoid AWS billing for long-running resources.
To remove local resources, stop the LocalStack container to clear the local Lambda function and SQS queue.

## Conclusion

In this tutorial, you've discovered how the AWS Replicator extension bridges the gap between local and remote cloud resources by mirroring resources from real AWS accounts into your LocalStack instance.
You can explore additional use-cases with the AWS Replicator extension, such as:
In this tutorial, you've discovered how the AWS Cloud Proxy extension bridges the gap between local and remote cloud resources by mirroring resources from real AWS accounts into your LocalStack instance.
You can explore additional use-cases with the AWS Cloud Proxy extension, such as:

- Developing a local Lambda function that interacts with a remote DynamoDB table
- Executing a local Athena SQL query in LocalStack, accessing files in a real S3 bucket on AWS
Expand Down