Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,42 @@

## AWS Databricks

The module can deploy an Intel Optimized AWS Databricks Workspace and Cluster. Instance Selection and Intel Optimizations have been defaulted in the code.
The module can deploy an Intel Optimized AWS Databricks Workspace.

**Learn more about optimizations :**
## Performance Data

[Databricks Photon using AWS i4i](https://www.databricks.com/blog/2022/09/13/faster-insights-databricks-photon-using-aws-i4i-instances-latest-intel-ice-lake)
<center>

[Accelerating Databricks Runtime for Machine Learning](https://techcommunity.microsoft.com/t5/ai-customer-engineering-team/accelerating-azure-databricks-runtime-for-machine-learning/ba-p/3524273)
#### [Faster insights With Databricks Photon Using AWS i4i Instances With the Latest Intel Ice Lake Scalable Processors](https://www.databricks.com/blog/2022/09/13/faster-insights-databricks-photon-using-aws-i4i-instances-latest-intel-ice-lake)

<p align="center">
<a href="https://www.databricks.com/blog/2022/09/13/faster-insights-databricks-photon-using-aws-i4i-instances-latest-intel-ice-lake">
<img src="https://github.com/intel/terraform-intel-aws-databricks-workspace/blob/main/images/aws-dbx-1.png" alt="Link" width="600"/>
</a>
</p>

#
#### [5.3x relative speed up of i4i Photon against the i3 DBR](https://www.databricks.com/blog/2022/09/13/faster-insights-databricks-photon-using-aws-i4i-instances-latest-intel-ice-lake)

<p align="center">
<a href="https://www.databricks.com/blog/2022/09/13/faster-insights-databricks-photon-using-aws-i4i-instances-latest-intel-ice-lake">
<img src="https://github.com/intel/terraform-intel-aws-databricks-workspace/blob/main/images/aws-dbx-2.png?raw=true" alt="Link" width="600"/>
</a>
</p>

#
#### [Accelerating Azure Databricks Runtime for Machine Learning](https://techcommunity.microsoft.com/t5/ai-customer-engineering-team/accelerating-azure-databricks-runtime-for-machine-learning/ba-p/3524273)

<p align="center">
<a href="https://techcommunity.microsoft.com/t5/ai-customer-engineering-team/accelerating-azure-databricks-runtime-for-machine-learning/ba-p/3524273">
<img src="https://github.com/intel/terraform-intel-aws-databricks-workspace/blob/main/images/dbx-runtime.png?raw=true" alt="Link" width="600"/>
</a>
</p>

#

</center>

## Usage

Expand Down Expand Up @@ -115,6 +144,8 @@ No modules.
| <a name="input_dbx_network_name"></a> [dbx\_network\_name](#input\_dbx\_network\_name) | Name that will be associated with the network configuration in Databricks. | `string` | `"dbx_module_network"` | no |
| <a name="input_dbx_storage_name"></a> [dbx\_storage\_name](#input\_dbx\_storage\_name) | Name that will be associated with the storage configuration that will be created in Databricks. | `string` | `"dbx_module_storage"` | no |
| <a name="input_dbx_workspace_name"></a> [dbx\_workspace\_name](#input\_dbx\_workspace\_name) | Name that will be associated with the workspace that will be created in Databricks. | `string` | `"dbx_module_workspace"` | no |
| <a name="input_enable_intel_tags"></a> [enable\_intel\_tags](#input\_enable\_intel\_tags) | If true adds additional Intel tags to resources | `bool` | `true` | no |
| <a name="input_intel_tags"></a> [intel\_tags](#input\_intel\_tags) | Intel Tags | `map(string)` | <pre>{<br> "intel-module": "terraform-intel-aws-databricks-workspace",<br> "intel-registry": "https://registry.terraform.io/namespaces/intel"<br>}</pre> | no |
| <a name="input_prefix"></a> [prefix](#input\_prefix) | Prefix that will be added to all resources that are created | `string` | `null` | no |
| <a name="input_region"></a> [region](#input\_region) | AWS Region that will be used as a part of the deployment | `string` | `"us-east-2"` | no |
| <a name="input_s3_bucket_acl"></a> [s3\_bucket\_acl](#input\_s3\_bucket\_acl) | ACL that will be attached to the S3 bucket (if created) during the provisioning process. | `string` | `"private"` | no |
Expand Down
Binary file added images/aws-dbx-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/aws-dbx-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 8 additions & 2 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ resource "random_string" "naming" {
length = 6
}

locals {

#If the enable_intel_tags is true, then additional Intel tags will be added to the resources created
tags = var.enable_intel_tags ? merge(var.intel_tags, var.tags) : var.tags
}

############################## Credentials Config ###################################

data "databricks_aws_assume_role_policy" "rp" {
Expand Down Expand Up @@ -35,7 +41,7 @@ resource "aws_iam_role" "ir" {
count = var.create_aws_account_role ? 1 : 0
name = var.prefix != null ? "${var.prefix}-${random_string.naming.result}-${var.aws_cross_account_role_name}" : "${var.aws_cross_account_role_name}"
assume_role_policy = data.databricks_aws_assume_role_policy.rp.json
tags = var.tags
tags = local.tags
}

resource "databricks_mws_credentials" "cr" {
Expand All @@ -60,7 +66,7 @@ resource "aws_s3_bucket" "rsb" {
bucket = var.prefix != null ? "${var.prefix}-${random_string.naming.result}-${var.bucket_name}" : "${random_string.naming.result}-${var.bucket_name}"
force_destroy = var.s3_bucket_force_destroy
acl = var.s3_bucket_acl
tags = merge(var.tags, {
tags = merge(local.tags, {
Name = var.prefix != null ? "${var.prefix}-${random_string.naming.result}-${var.bucket_name}" : "${random_string.naming.result}-${var.bucket_name}"
})
}
Expand Down
15 changes: 15 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,21 @@ variable "tags" {
default = {}
}

variable "enable_intel_tags" {
type = bool
default = true
description = "If true adds additional Intel tags to resources"
}

variable "intel_tags" {
default = {
intel-registry = "https://registry.terraform.io/namespaces/intel"
intel-module = "terraform-intel-aws-databricks-workspace"
}
type = map(string)
description = "Intel Tags"
}

variable "region" {
type = string
description = "AWS Region that will be used as a part of the deployment"
Expand Down