|
6 | 6 |
|
7 | 7 | © Copyright 2023, Intel Corporation
|
8 | 8 |
|
9 |
| -## GCP C3 4th Gen Xeon(code named Sapphire Rapids) & Intel® Cloud Optimized Recipe for FastChat |
| 9 | +## GCP C3 4th Gen Xeon(code named Sapphire Rapids) & Intel® Optimized Cloud Recipe for FastChat |
10 | 10 |
|
11 | 11 | This demo will showcase Large Language Model(LLM) CPU inference using 4th Gen Xeon Scalable Processors on GCP.
|
12 | 12 |
|
13 | 13 | ## Usage
|
14 | 14 |
|
15 |
| -You may need to change the appropriate IAM settings as described here: https://cloud.google.com/docs/terraform/get-started-with-terraform |
16 |
| - |
17 | 15 | 1. Log on to GCP Portal
|
18 | 16 | 2. Enter the GCP Cloud Shell (terminal button on top right of page)
|
19 | 17 | 3. Run the following commands in order:
|
20 | 18 |
|
21 |
| -`git clone https://github.com/intel/terraform-intel-gcp-vm.git` |
22 | 19 |
|
23 |
| -`cd terraform-intel-gcp-vm/examples/gcp-linux-fastchat-simple` |
| 20 | +```bash |
| 21 | +git clone https://github.com/intel/terraform-intel-gcp-vm.git |
| 22 | +cd terraform-intel-gcp-vm/examples/gcp-linux-fastchat-simple |
| 23 | +terraform init |
| 24 | +terraform apply |
24 | 25 |
|
25 |
| -`terraform init` |
| 26 | +# (enter your GCP project ID and "yes" to confirm) |
26 | 27 |
|
27 |
| -`terraform apply` |
| 28 | +``` |
28 | 29 |
|
29 | 30 | ## Running the Demo
|
30 | 31 |
|
31 |
| - |
32 | 32 | 1. **Wait ~10 minutes** for the Recipe to download/install FastChat and the LLM model before continuing
|
33 | 33 | 2. SSH into newly created GCP VM
|
34 | 34 | 3. **Run `source /usr/local/bin/run_demo.sh`**
|
35 |
| -4. If you did not choose to open a port, your app will be proxied through gradio. See https://xxxxxxx.gradio.live URL that is generated during the gradio run. |
36 |
| -5. Otherwise, if you choose to open port 7860 in main.tf: on your computer, open a browser and navigate to **http://<VM_PLUBLIC_IP>:7860**. Get your Public IP from the "Compute Engine" section of the GCP console. |
| 35 | +4. On your local computer, open a browser and navigate to **http://<VM_PLUBLIC_IP>:7860**. Get your Public IP from the "Compute Engine" section of the VM in the GCP console. |
| 36 | +5. Or use the https://xxxxxxx.gradio.live URL that is generated during the demo startup(see on screen logs) |
37 | 37 |
|
38 | 38 | <p align="center">
|
39 |
| - <img src="https://github.com/intel/terraform-intel-gcp-vm/blob/main/images/gradio.png?raw=true" alt="Gradio_Output" width="250"/> |
| 39 | + <img src="https://github.com/intel/terraform-intel-gcp-vm/blob/main/images/gradio.png?raw=true" alt="Gradio_Output" width="900"/> |
40 | 40 | </p>
|
41 | 41 |
|
42 |
| - |
43 |
| ---- KNOWN ISSUE --- |
44 |
| - |
45 |
| -The demo may initially fail. In this case, run |
46 |
| - |
47 |
| -`pip install gradio==3.10` |
48 |
| - |
49 |
| -`pip install gradio==3.35.2` |
50 |
| - |
51 |
| -Then, run: |
52 |
| - |
53 |
| -`source /usr/local/bin/run_demo.sh` |
54 |
| - |
55 |
| -And navigate again using your browser. |
| 42 | +Note: You may need permissions to run Terraform https://cloud.google.com/docs/terraform/get-started-with-terraform |
56 | 43 |
|
57 | 44 | ## Pre-requisites for running on a Workstation (bypass if using GCP Cloud Shell)
|
58 | 45 |
|
|
0 commit comments