Skip to content

Commit b4352e6

Browse files
LlamaEnjoyerLlamaEnjoyer
committed
Merge branch 'master' into batched_evals
2 parents d3e46da + 1e18e80 commit b4352e6

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+1089
-658
lines changed

‎.github/ISSUE_TEMPLATE/bug_report.yaml

+109
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
name: Bug report
2+
description: Report bugs with the project
3+
title: "[BUG]"
4+
labels: bug
5+
body:
6+
7+
- type: markdown
8+
attributes:
9+
value: |
10+
### Disclaimer:
11+
Github Issues are **only** for code related bugs.
12+
Please fill in as many fields as possible so we can understand the relevant parts of the issue.
13+
14+
- type: dropdown
15+
attributes:
16+
label: OS
17+
options:
18+
- Windows
19+
- Linux
20+
validations:
21+
required: true
22+
23+
- type: dropdown
24+
attributes:
25+
label: GPU Library
26+
description: Ex. CUDA, ROCm
27+
options:
28+
- CUDA 12.x
29+
- CUDA 11.8
30+
- AMD ROCm
31+
validations:
32+
required: true
33+
34+
- type: dropdown
35+
attributes:
36+
label: Python version
37+
options:
38+
- '3.12'
39+
- '3.11'
40+
- '3.10'
41+
validations:
42+
required: true
43+
44+
- type: input
45+
attributes:
46+
label: Pytorch version
47+
validations:
48+
required: True
49+
50+
- type: input
51+
attributes:
52+
label: Model
53+
description: Provide a model if the issue is related to one
54+
placeholder: HF Repo Author/Model Name
55+
validations:
56+
required: False
57+
58+
- type: textarea
59+
attributes:
60+
label: Describe the bug
61+
description: A clear and concise description of what the bug is.
62+
validations:
63+
required: true
64+
65+
- type: textarea
66+
attributes:
67+
label: Reproduction steps
68+
description: Walk us through how the bug occurred and how to make it happen.
69+
validations:
70+
required: true
71+
72+
- type: textarea
73+
attributes:
74+
label: Expected behavior
75+
description: What was expected to happen?
76+
validations:
77+
required: true
78+
79+
- type: textarea
80+
attributes:
81+
label: Logs
82+
description: If applicable, add logs and tracebacks to help explain your problem.
83+
validations:
84+
required: false
85+
86+
- type: textarea
87+
attributes:
88+
label: Additional context
89+
description: Add any other context about the problem here.
90+
validations:
91+
required: false
92+
93+
- type: checkboxes
94+
attributes:
95+
label: Acknowledgements
96+
description: Before submitting this issue, please make sure you have completed the following checklist.
97+
options:
98+
- label: I have looked for similar issues before submitting this one.
99+
required: true
100+
- label: I understand that the developers have lives and my issue will be answered when possible.
101+
required: true
102+
- label: I understand the developers of this program are human, and I will ask my questions politely.
103+
required: true
104+
105+
- type: markdown
106+
attributes:
107+
value: |
108+
## Thanks!
109+
Well-formatted issues improve ExllamaV2 and make the development process easier.

‎.github/ISSUE_TEMPLATE/config.yml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
blank_issues_enabled: false

‎.github/ISSUE_TEMPLATE/feature_request.yaml

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Feature request
2+
description: Suggest a new idea
3+
title: "[REQUEST]"
4+
body:
5+
6+
- type: textarea
7+
attributes:
8+
label: Problem
9+
description: Is the feature request related to a problem? If so, please describe.
10+
placeholder: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
11+
validations:
12+
required: false
13+
14+
- type: textarea
15+
attributes:
16+
label: Solution
17+
description: Describe the solution you'd like.
18+
placeholder: A clear and concise description of what you want to happen.
19+
validations:
20+
required: true
21+
22+
- type: textarea
23+
attributes:
24+
label: Alternatives
25+
description: What alternative options did you consider?
26+
validations:
27+
required: false
28+
29+
- type: textarea
30+
attributes:
31+
label: Explanation
32+
description: Why should this feature be added?
33+
validations:
34+
required: true
35+
36+
- type: textarea
37+
attributes:
38+
label: Examples
39+
description: |
40+
Examples of the feature in action and its significance.
41+
42+
Not required, but will make your request easier to understand. Real-world examples are helpful for samplers.
43+
validations:
44+
required: false
45+
46+
- type: textarea
47+
attributes:
48+
label: Additional context
49+
description: Anything else to add?
50+
validations:
51+
required: false
52+
53+
- type: checkboxes
54+
attributes:
55+
label: Acknowledgements
56+
description: Before submitting this issue, please make sure you have completed the following checklist.
57+
options:
58+
- label: I have looked for similar requests before submitting this one.
59+
required: true
60+
- label: I understand that the developers have lives and my issue will be answered when possible.
61+
required: true
62+
- label: I understand the developers of this program are human, and I will make my requests politely.
63+
required: true
64+
65+
- type: markdown
66+
attributes:
67+
value: |
68+
## Thanks!
69+
Well-formatted issues improve ExllamaV2 and make the development process easier.

‎exllamav2/architecture.py

+11-1
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,17 @@ class RopeStyle(IntEnum):
101101

102102
class ExLlamaV2ArchParams:
103103

104-
def __init__(self, arch_string, read_config):
104+
def __init__(self, arch_string: str, read_config: dict):
105+
"""
106+
Get architecture definition from model config. If the architecture isn't recognized, defaults to Llama
107+
architecture.
108+
W
109+
:param arch_string:
110+
Architecture string from config.json
111+
112+
:param read_config:
113+
config.json as Python dict
114+
"""
105115

106116
self.arch_string = arch_string
107117
arch_recognized = False

0 commit comments

Comments
 (0)