Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
0230ed3
feat: load a model asynchronously
giladgd Mar 3, 2024
299cb9e
fix: adapt to breaking `llama.cpp` changes to make embedding work again
giladgd Mar 9, 2024
2bfe21a
feat: log package install instructions for `apk`
giladgd Mar 9, 2024
26b1339
fix: show console log prefix on postinstall
giladgd Mar 9, 2024
edffb14
fix: registry memory usage on the correct js env
giladgd Mar 9, 2024
0ea4a84
feat: async context creation
giladgd Mar 9, 2024
6f6c93c
test: async context creation
giladgd Mar 9, 2024
dad4a64
feat: export `TemplateChatWrapperOptions`
giladgd Mar 9, 2024
565da6a
feat: detect cmake binary issues and suggest fixes on detection
giladgd Mar 9, 2024
868eb02
style: lint
giladgd Mar 9, 2024
2835eac
docs: update type docs
giladgd Mar 9, 2024
123c9a0
fix: adapt to breaking `llama.cpp` changes to support mamba models
giladgd Mar 9, 2024
b792d46
test: fix tests
giladgd Mar 10, 2024
e9d090a
fix: compile Metal shaders to make the binaries work on macOS with la…
giladgd Mar 11, 2024
fff9fd0
fix: call logger with last `llama.cpp` logs before exit
giladgd Mar 11, 2024
2734809
fix: rename `.buildMetadata.json` to not start with a dot, to make us…
giladgd Mar 11, 2024
00a6e85
fix: missing module
giladgd Mar 11, 2024
5c9a19d
fix: bug
giladgd Mar 11, 2024
c78a3b7
feat: update `cmake-js` and `ipull`
giladgd Mar 12, 2024
6889d00
fix: detect windir
giladgd Mar 13, 2024
5582b3e
fix: bug
giladgd Mar 13, 2024
f93b53c
fix: `DisposedError` was thrown when calling `.dispose()`
giladgd Mar 13, 2024
361ac59
fix: wait for logs to finish printing before the next action on CLI c…
giladgd Mar 13, 2024
021972f
fix: remove Metal shader compilation workaround as it's no longer needed
giladgd Mar 14, 2024
bda0d5c
fix: adapt to `llama.cpp` changes
giladgd Mar 15, 2024
5a3d5d1
feat: async binding backend loading and unloading
giladgd Mar 16, 2024
3660313
test: decode using multiple bindings in parallel
giladgd Mar 16, 2024
0420609
feat: a script to setup a machine and test on Paperspace
giladgd Mar 16, 2024
388f006
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
2dee989
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
03db3c3
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
883293a
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
abe2ab4
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
220ebf3
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
d7452a6
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
0a1a4de
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
8eafa5c
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
08149c5
test: fix `getTestLlama.ts`
giladgd Mar 16, 2024
51b89e3
test: fix `getTestLlama.ts`
giladgd Mar 16, 2024
77356bc
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
274b511
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
b8f7cc8
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
f6b2725
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
9d58528
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
787089f
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
309cb5c
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
a55548a
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
6e84d2c
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
19fcda7
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
f7ab49e
feat: automatically try to resolve `Failed to detect a default CUDA a…
giladgd Mar 16, 2024
cd9135a
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
f58ee14
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
70c3e8d
fix: `SpawnError`
giladgd Mar 16, 2024
928550c
fix: `compileLlamaCpp` bug
giladgd Mar 16, 2024
0f700ff
fix: CUDA detection bug
giladgd Mar 16, 2024
a5ea28d
fix: automatic compilation workarounds deadlock
giladgd Mar 16, 2024
79ef65d
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
453827c
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
90a28cb
feat: remove `ts-node` in favor of `vite-node`
giladgd Mar 16, 2024
d4dde4c
fix: improve CUDA workarounds
giladgd Mar 16, 2024
45cc411
fix: `setupAndTestOnPaperspace.sh`
giladgd Mar 16, 2024
5474d5a
feat: add `test:vitest` script to `package.json`
giladgd Mar 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/guide/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ npm run dev:build; node ./dist/cli/cli.js chat --wrapper llamaChat --model <path
### Debugging
To run a chat session with a debugger, configure your IDE to run the following command with a debugger:
```bash
node --loader ts-node/esm ./src/cli/cli.ts chat --model <path-to-a-model-file-on-your-computer>
npx vite-node ./src/cli/cli.ts chat --model <path-to-a-model-file-on-your-computer>
```

## Opening a pull request
Expand Down
3 changes: 1 addition & 2 deletions docs/guide/vulkan.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,7 @@ const llama = await getLlama({
```
To configure how much layers of the model are run on the GPU, configure `gpuLayers` on `LlamaModel` in your code:
```typescript
const model = new LlamaModel({
llama,
const model = await llama.loadModel({
modelPath,
gpuLayers: 64 // or any other number of layers you want
});
Expand Down
759 changes: 676 additions & 83 deletions llama/addon.cpp

Large diffs are not rendered by default.

Loading