Skip to content

Commit 315a3eb

Browse files
authored
feat: async operations (#178)
* feat: async model loading * feat: async context creation * feat: export `TemplateChatWrapperOptions` * feat: detect `cmake` binary issues and suggest fixes on detection * feat: automatically try to resolve `Failed to detect a default CUDA architecture` CUDA compilation error * fix: adapt to breaking `llama.cpp` changes to make embedding work again * fix: adapt to breaking `llama.cpp` changes to support mamba models * fix: show console log prefix on postinstall * fix: call logger with last `llama.cpp` logs before exit * fix: rename `.buildMetadata.json` to not start with a dot, to make using this library together with bundlers easier * fix: `DisposedError` was thrown when calling `.dispose()`
1 parent 5a70576 commit 315a3eb

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+2759
-1053
lines changed

docs/guide/development.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ npm run dev:build; node ./dist/cli/cli.js chat --wrapper llamaChat --model <path
6060
### Debugging
6161
To run a chat session with a debugger, configure your IDE to run the following command with a debugger:
6262
```bash
63-
node --loader ts-node/esm ./src/cli/cli.ts chat --model <path-to-a-model-file-on-your-computer>
63+
npx vite-node ./src/cli/cli.ts chat --model <path-to-a-model-file-on-your-computer>
6464
```
6565

6666
## Opening a pull request

docs/guide/vulkan.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -88,8 +88,7 @@ const llama = await getLlama({
8888
```
8989
To configure how much layers of the model are run on the GPU, configure `gpuLayers` on `LlamaModel` in your code:
9090
```typescript
91-
const model = new LlamaModel({
92-
llama,
91+
const model = await llama.loadModel({
9392
modelPath,
9493
gpuLayers: 64 // or any other number of layers you want
9594
});

llama/addon.cpp

Lines changed: 676 additions & 83 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)