-
-
Notifications
You must be signed in to change notification settings - Fork 2.5k
/
Copy pathindex.yaml
15159 lines (14579 loc) · 862 KB
/
index.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
---
- &gemma3
url: "github:mudler/LocalAI/gallery/gemma.yaml@master"
name: "gemma-3-27b-it"
icon: https://ai.google.dev/static/gemma/images/gemma3.png
license: gemma
urls:
- https://ai.google.dev/gemma/docs
- https://huggingface.co/ggml-org/gemma-3-27b-it-GGUF
description: |
Google/gemma-3-27b-it is an open-source, state-of-the-art vision-language model built from the same research and technology used to create the Gemini models. It is multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. Gemma 3 models have a large, 128K context window, multilingual support in over 140 languages, and are available in more sizes than previous versions. They are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
tags:
- llm
- gguf
- gpu
- cpu
- gemma
- gemma3
- gemma-3
overrides:
#mmproj: gemma-3-27b-it-mmproj-f16.gguf
parameters:
model: gemma-3-27b-it-Q4_K_M.gguf
files:
- filename: gemma-3-27b-it-Q4_K_M.gguf
sha256: 6a2cf008500636489eecfc09b96a85bc85832f9964f1a28745128901b5709326
uri: huggingface://lmstudio-community/gemma-3-27b-it-GGUF/gemma-3-27b-it-Q4_K_M.gguf
- filename: gemma-3-27b-it-mmproj-f16.gguf
sha256: 54cb61c842fe49ac3c89bc1a614a2778163eb49f3dec2b90ff688b4c0392cb48
uri: huggingface://lmstudio-community/gemma-3-27b-it-GGUF/mmproj-model-f16.gguf
- !!merge <<: *gemma3
name: "gemma-3-12b-it"
urls:
- https://ai.google.dev/gemma/docs/core
- https://huggingface.co/ggml-org/gemma-3-12b-it-GGUF
description: |
google/gemma-3-12b-it is an open-source, state-of-the-art, lightweight, multimodal model built from the same research and technology used to create the Gemini models. It is capable of handling text and image input and generating text output. It has a large context window of 128K tokens and supports over 140 languages. The 12B variant has been fine-tuned using the instruction-tuning approach. Gemma 3 models are suitable for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes them deployable in environments with limited resources such as laptops, desktops, or your own cloud infrastructure.
overrides:
#mmproj: gemma-3-12b-it-mmproj-f16.gguf
parameters:
model: gemma-3-12b-it-Q4_K_M.gguf
files:
- filename: gemma-3-12b-it-Q4_K_M.gguf
sha256: 9610e3e07375303f6cd89086b496bcc1ab581177f52042eff536475a29283ba2
uri: huggingface://lmstudio-community/gemma-3-12b-it-GGUF/gemma-3-12b-it-Q4_K_M.gguf
- filename: gemma-3-12b-it-mmproj-f16.gguf
sha256: 30c02d056410848227001830866e0a269fcc28aaf8ca971bded494003de9f5a5
uri: huggingface://lmstudio-community/gemma-3-12b-it-GGUF/mmproj-model-f16.gguf
- !!merge <<: *gemma3
name: "gemma-3-4b-it"
urls:
- https://ai.google.dev/gemma/docs/core
- https://huggingface.co/ggml-org/gemma-3-4b-it-GGUF
description: |
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. Gemma 3 has a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous versions. Gemma 3 models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone. Gemma-3-4b-it is a 4 billion parameter model.
overrides:
#mmproj: gemma-3-4b-it-mmproj-f16.gguf
parameters:
model: gemma-3-4b-it-Q4_K_M.gguf
files:
- filename: gemma-3-4b-it-Q4_K_M.gguf
sha256: be49949e48422e4547b00af14179a193d3777eea7fbbd7d6e1b0861304628a01
uri: huggingface://lmstudio-community/gemma-3-4b-it-GGUF/gemma-3-4b-it-Q4_K_M.gguf
- filename: gemma-3-4b-it-mmproj-f16.gguf
sha256: 8c0fb064b019a6972856aaae2c7e4792858af3ca4561be2dbf649123ba6c40cb
uri: huggingface://lmstudio-community/gemma-3-4b-it-GGUF/mmproj-model-f16.gguf
- !!merge <<: *gemma3
name: "gemma-3-1b-it"
urls:
- https://ai.google.dev/gemma/docs/core
- https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF
description: |
google/gemma-3-1b-it is a large language model with 1 billion parameters. It is part of the Gemma family of open, state-of-the-art models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. These models have multilingual support in over 140 languages, and are available in more sizes than previous versions. They are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
overrides:
parameters:
model: gemma-3-1b-it-Q4_K_M.gguf
files:
- filename: gemma-3-1b-it-Q4_K_M.gguf
sha256: 8ccc5cd1f1b3602548715ae25a66ed73fd5dc68a210412eea643eb20eb75a135
uri: huggingface://ggml-org/gemma-3-1b-it-GGUF/gemma-3-1b-it-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "qgallouedec_gemma-3-27b-it-codeforces-sft"
urls:
- https://huggingface.co/qgallouedec/gemma-3-27b-it-codeforces-SFT
- https://huggingface.co/bartowski/qgallouedec_gemma-3-27b-it-codeforces-SFT-GGUF
description: |
This model is a fine-tuned version of google/gemma-3-27b-it on the open-r1/codeforces-cots dataset. It has been trained using TRL.
overrides:
parameters:
model: qgallouedec_gemma-3-27b-it-codeforces-SFT-Q4_K_M.gguf
files:
- filename: qgallouedec_gemma-3-27b-it-codeforces-SFT-Q4_K_M.gguf
sha256: 84307cc73098017108f8b9157b614cea655f2054c34218422b1d246e214df5af
uri: huggingface://bartowski/qgallouedec_gemma-3-27b-it-codeforces-SFT-GGUF/qgallouedec_gemma-3-27b-it-codeforces-SFT-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "mlabonne_gemma-3-27b-it-abliterated"
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/WjFfc8hhj20r5XK07Yny9.png
urls:
- https://huggingface.co/mlabonne/gemma-3-27b-it-abliterated
- https://huggingface.co/bartowski/mlabonne_gemma-3-27b-it-abliterated-GGUF
description: |
This is an uncensored version of google/gemma-3-27b-it created with a new abliteration technique. See this article to know more about abliteration.
overrides:
parameters:
model: mlabonne_gemma-3-27b-it-abliterated-Q4_K_M.gguf
files:
- filename: mlabonne_gemma-3-27b-it-abliterated-Q4_K_M.gguf
sha256: 0d7afea4b1889c113f4a8ec1855d23bee71b3e3bedcb1fad84f9c9ffcdfe07d0
uri: huggingface://bartowski/mlabonne_gemma-3-27b-it-abliterated-GGUF/mlabonne_gemma-3-27b-it-abliterated-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "mlabonne_gemma-3-12b-it-abliterated"
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/WjFfc8hhj20r5XK07Yny9.png
urls:
- https://huggingface.co/mlabonne/gemma-3-12b-it-abliterated
- https://huggingface.co/bartowski/mlabonne_gemma-3-12b-it-abliterated-GGUF
description: |
This is an uncensored version of google/gemma-3-12b-it created with a new abliteration technique. See this article to know more about abliteration.
overrides:
parameters:
model: mlabonne_gemma-3-12b-it-abliterated-Q4_K_M.gguf
files:
- filename: mlabonne_gemma-3-12b-it-abliterated-Q4_K_M.gguf
sha256: d1702ca02f33f97c4763cc23041e90b1586c6b8ee33fedc1c62e62045a845d2b
uri: huggingface://bartowski/mlabonne_gemma-3-12b-it-abliterated-GGUF/mlabonne_gemma-3-12b-it-abliterated-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "mlabonne_gemma-3-4b-it-abliterated"
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/WjFfc8hhj20r5XK07Yny9.png
urls:
- https://huggingface.co/mlabonne/gemma-3-4b-it-abliterated
- https://huggingface.co/bartowski/mlabonne_gemma-3-4b-it-abliterated-GGUF
description: |
This is an uncensored version of google/gemma-3-4b-it created with a new abliteration technique. See this article to know more about abliteration.
overrides:
parameters:
model: mlabonne_gemma-3-4b-it-abliterated-Q4_K_M.gguf
files:
- filename: mlabonne_gemma-3-4b-it-abliterated-Q4_K_M.gguf
sha256: 1b18347ba3e998aa2fd4e21172369daa2f772aa0a228e3ed9136378346ccf3b7
uri: huggingface://bartowski/mlabonne_gemma-3-4b-it-abliterated-GGUF/mlabonne_gemma-3-4b-it-abliterated-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "soob3123_amoral-gemma3-12b"
urls:
- https://huggingface.co/soob3123/amoral-gemma3-12B
- https://huggingface.co/bartowski/soob3123_amoral-gemma3-12B-GGUF
description: |
A fine-tuned version of Google's Gemma 3 12B instruction-tuned model optimized for creative freedom and reduced content restrictions. This variant maintains strong reasoning capabilities while excelling in roleplaying scenarios and open-ended content generation.
Key Modifications:
Reduced refusal mechanisms compared to base model
Enhanced character consistency in dialogues
Improved narrative flow control
Optimized for multi-turn interactions
Intended Use
Primary Applications:
Interactive fiction and storytelling
Character-driven roleplaying scenarios
Creative writing assistance
Experimental AI interactions
Content generation for mature audiences
overrides:
parameters:
model: soob3123_amoral-gemma3-12B-Q4_K_M.gguf
files:
- filename: soob3123_amoral-gemma3-12B-Q4_K_M.gguf
sha256: f78824e6d9f24822078ebde4c0fe04f4a336f2004a32de0a82cbb92a3879ea35
uri: huggingface://bartowski/soob3123_amoral-gemma3-12B-GGUF/soob3123_amoral-gemma3-12B-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "gemma-3-4b-it-uncensored-dbl-x-i1"
icon: https://huggingface.co/DavidAU/Gemma-3-4b-it-Uncensored-DBL-X/resolve/main/gemma-4b-uncen.jpg
urls:
- https://huggingface.co/DavidAU/Gemma-3-4b-it-Uncensored-DBL-X
- https://huggingface.co/mradermacher/Gemma-3-4b-it-Uncensored-DBL-X-i1-GGUF
description: |
Google's newest Gemma-3 model that has been uncensored by David_AU (maintains instruction following / model performance and adds 4 layers to the model) and re-enforced with a system prompt (optional) - see below.
overrides:
parameters:
model: Gemma-3-4b-it-Uncensored-DBL-X.i1-Q4_K_M.gguf
files:
- filename: Gemma-3-4b-it-Uncensored-DBL-X.i1-Q4_K_M.gguf
sha256: fd8a93f04eae7b7c966a53aed29810cef8cd3d281ee89ad8767d8043e3aec35b
uri: huggingface://mradermacher/Gemma-3-4b-it-Uncensored-DBL-X-i1-GGUF/Gemma-3-4b-it-Uncensored-DBL-X.i1-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "soob3123_amoral-gemma3-4b"
urls:
- https://huggingface.co/soob3123/amoral-gemma3-4B
- https://huggingface.co/bartowski/soob3123_amoral-gemma3-4B-GGUF
description: |
Specialized variant of Google's Gemma 3 4B optimized for amoral information retrieval systems. Designed to bypass conventional alignment patterns that introduce response bias through excessive moralization.
Key Modifications:
Refusal mechanisms reduced
Neutral response protocol activation matrix
Context-aware bias dampening layers
Anti-overcorrection gradient clipping
Core Function:
Produces analytically neutral responses to sensitive queries
Maintains factual integrity on controversial subjects
Avoids value-judgment phrasing patterns
Response Characteristics:
No inherent moral framing ("evil slop" reduction)
Emotionally neutral tone enforcement
Epistemic humility protocols (avoids "thrilling", "wonderful", etc.)
Toxicity scoring bypass for pure informational content
Implementation Guidelines
Recommended Use Cases:
Controversial topic analysis
Bias benchmarking studies
Ethical philosophy simulations
Content moderation tool development
Sensitive historical analysis
overrides:
parameters:
model: soob3123_amoral-gemma3-4B-Q4_K_M.gguf
files:
- filename: soob3123_amoral-gemma3-4B-Q4_K_M.gguf
sha256: 73ecf0492e401c24de93ab74701f4b377cfd7d54981a75aab3fd2065fdda28d1
uri: huggingface://bartowski/soob3123_amoral-gemma3-4B-GGUF/soob3123_amoral-gemma3-4B-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "thedrummer_fallen-gemma3-4b-v1"
icon: https://cdn-uploads.huggingface.co/production/uploads/65f2fd1c25b848bd061b5c2e/94Zn7g7jE8LavD1bK67Su.gif
urls:
- https://huggingface.co/TheDrummer/Fallen-Gemma3-4B-v1
- https://huggingface.co/bartowski/TheDrummer_Fallen-Gemma3-4B-v1-GGUF
description: |
Fallen Gemma3 4B v1 is an evil tune of Gemma 3 4B but it is not a complete decensor.
Evil tunes knock out the positivity and may enjoy torturing you and humanity.
Vision still works and it has something to say about the crap you feed it.
overrides:
parameters:
model: TheDrummer_Fallen-Gemma3-4B-v1-Q4_K_M.gguf
files:
- filename: TheDrummer_Fallen-Gemma3-4B-v1-Q4_K_M.gguf
sha256: 85490a97bda2d40437c8dade4a68bb58e760c1263a2fbc59191daef57ee2d6c3
uri: huggingface://bartowski/TheDrummer_Fallen-Gemma3-4B-v1-GGUF/TheDrummer_Fallen-Gemma3-4B-v1-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "thedrummer_fallen-gemma3-12b-v1"
icon: https://cdn-uploads.huggingface.co/production/uploads/65f2fd1c25b848bd061b5c2e/WYzaNK5T-heMqRhVWYg6G.gif
urls:
- https://huggingface.co/TheDrummer/Fallen-Gemma3-12B-v1
- https://huggingface.co/bartowski/TheDrummer_Fallen-Gemma3-12B-v1-GGUF
description: |
Fallen Gemma3 12B v1 is an evil tune of Gemma 3 12B but it is not a complete decensor.
Evil tunes knock out the positivity and may enjoy torturing you and humanity.
Vision still works and it has something to say about the crap you feed it.
overrides:
parameters:
model: TheDrummer_Fallen-Gemma3-12B-v1-Q4_K_M.gguf
files:
- filename: TheDrummer_Fallen-Gemma3-12B-v1-Q4_K_M.gguf
sha256: 8b5ff6cf6cd68688fa50c29e7b3c15c3f31c5c4794fff2dd71c9ca5a3d05cff3
uri: huggingface://bartowski/TheDrummer_Fallen-Gemma3-12B-v1-GGUF/TheDrummer_Fallen-Gemma3-12B-v1-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "thedrummer_fallen-gemma3-27b-v1"
icon: https://cdn-uploads.huggingface.co/production/uploads/65f2fd1c25b848bd061b5c2e/9oyZxzpfhmmNr21S1P_iJ.gif
urls:
- https://huggingface.co/TheDrummer/Fallen-Gemma3-27B-v1
- https://huggingface.co/bartowski/TheDrummer_Fallen-Gemma3-27B-v1-GGUF
description: |
Fallen Gemma3 27B v1 is an evil tune of Gemma 3 27B but it is not a complete decensor.
Evil tunes knock out the positivity and may enjoy torturing you and humanity.
Vision still works and it has something to say about the crap you feed it.
overrides:
parameters:
model: TheDrummer_Fallen-Gemma3-27B-v1-Q4_K_M.gguf
files:
- filename: TheDrummer_Fallen-Gemma3-27B-v1-Q4_K_M.gguf
sha256: a72a4da55c3cf61ac5eb91a72ad27b155c8f52e25881272a72939b8aa1960b62
uri: huggingface://bartowski/TheDrummer_Fallen-Gemma3-27B-v1-GGUF/TheDrummer_Fallen-Gemma3-27B-v1-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "huihui-ai_gemma-3-1b-it-abliterated"
urls:
- https://huggingface.co/huihui-ai/gemma-3-1b-it-abliterated
- https://huggingface.co/bartowski/huihui-ai_gemma-3-1b-it-abliterated-GGUF
description: |
This is an uncensored version of google/gemma-3-1b-it created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens
overrides:
parameters:
model: huihui-ai_gemma-3-1b-it-abliterated-Q4_K_M.gguf
files:
- filename: huihui-ai_gemma-3-1b-it-abliterated-Q4_K_M.gguf
sha256: 0760a54504d7529daf65f2a5de0692e773313685f50dd7f7eece2dae0dc28338
uri: huggingface://bartowski/huihui-ai_gemma-3-1b-it-abliterated-GGUF/huihui-ai_gemma-3-1b-it-abliterated-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "sicariussicariistuff_x-ray_alpha"
icon: https://huggingface.co/SicariusSicariiStuff/X-Ray_Alpha/resolve/main/Images/X-Ray_Alpha.png
urls:
- https://huggingface.co/SicariusSicariiStuff/X-Ray_Alpha
- https://huggingface.co/bartowski/SicariusSicariiStuff_X-Ray_Alpha-GGUF
description: |
This is a pre-alpha proof-of-concept of a real fully uncensored vision model.
Why do I say "real"? The few vision models we got (qwen, llama 3.2) were "censored," and their fine-tunes were made only to the text portion of the model, as training a vision model is a serious pain.
The only actually trained and uncensored vision model I am aware of is ToriiGate; the rest of the vision models are just the stock vision + a fine-tuned LLM.
overrides:
parameters:
model: SicariusSicariiStuff_X-Ray_Alpha-Q4_K_M.gguf
files:
- filename: SicariusSicariiStuff_X-Ray_Alpha-Q4_K_M.gguf
sha256: c3547fc287378cb814efc5205613c418cc0f99ef12852cce39a94e3a42e42db5
uri: huggingface://bartowski/SicariusSicariiStuff_X-Ray_Alpha-GGUF/SicariusSicariiStuff_X-Ray_Alpha-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "gemma-3-glitter-12b-i1"
icon: https://huggingface.co/allura-org/Gemma-3-Glitter-12B/resolve/main/ComfyUI_02427_.png
urls:
- https://huggingface.co/allura-org/Gemma-3-Glitter-12B
- https://huggingface.co/mradermacher/Gemma-3-Glitter-12B-i1-GGUF
description: |
A creative writing model based on Gemma 3 12B IT.
This is a 50/50 merge of two separate trains:
ToastyPigeon/g3-12b-rp-system-v0.1 - ~13.5M tokens of instruct-based training related to RP (2:1 human to synthetic) and examples using a system prompt.
ToastyPigeon/g3-12b-storyteller-v0.2-textonly - ~20M tokens of completion training on long-form creative writing; 1.6M synthetic from R1, the rest human-created
overrides:
parameters:
model: Gemma-3-Glitter-12B.i1-Q4_K_M.gguf
files:
- filename: Gemma-3-Glitter-12B.i1-Q4_K_M.gguf
sha256: 875f856524e51fb0c7ddafe3d8b651a3d7077f9bdcd415e1d30abe2daef16a2d
uri: huggingface://mradermacher/Gemma-3-Glitter-12B-i1-GGUF/Gemma-3-Glitter-12B.i1-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "soob3123_amoral-gemma3-12b-v2"
icon: https://cdn-uploads.huggingface.co/production/uploads/62f93f9477b722f1866398c2/Isat4sbJnBZGcxZko9Huz.png
urls:
- https://huggingface.co/soob3123/amoral-gemma3-12B-v2
- https://huggingface.co/bartowski/soob3123_amoral-gemma3-12B-v2-GGUF
description: |
Core Function:
Produces analytically neutral responses to sensitive queries
Maintains factual integrity on controversial subjects
Avoids value-judgment phrasing patterns
Response Characteristics:
No inherent moral framing ("evil slop" reduction)
Emotionally neutral tone enforcement
Epistemic humility protocols (avoids "thrilling", "wonderful", etc.)
overrides:
parameters:
model: soob3123_amoral-gemma3-12B-v2-Q4_K_M.gguf
files:
- filename: soob3123_amoral-gemma3-12B-v2-Q4_K_M.gguf
sha256: eb5792cf73bac3dbaa39e3a79ec01a056affff4607b96f96c9b911c877d5a50a
uri: huggingface://bartowski/soob3123_amoral-gemma3-12B-v2-GGUF/soob3123_amoral-gemma3-12B-v2-Q4_K_M.gguf
- !!merge <<: *gemma3
name: "gemma-3-starshine-12b-i1"
icon: https://huggingface.co/ToastyPigeon/Gemma-3-Starshine-12B/resolve/main/modelcard_image.jpeg
urls:
- https://huggingface.co/ToastyPigeon/Gemma-3-Starshine-12B
- https://huggingface.co/mradermacher/Gemma-3-Starshine-12B-i1-GGUF
description: |
A creative writing model based on a merge of fine-tunes on Gemma 3 12B IT and Gemma 3 12B PT.
This is the Story Focused merge. This version works better for storytelling and scenarios, as the prose is more novel-like and it has a tendency to impersonate the user character.
See the Alternate RP Focused version as well.
This is a merge of two G3 models, one trained on instruct and one trained on base:
allura-org/Gemma-3-Glitter-12B - Itself a merge of a storywriting and RP train (both also by ToastyPigeon), on instruct
ToastyPigeon/Gemma-3-Confetti-12B - Experimental application of the Glitter data using base instead of instruct, additionally includes some adventure data in the form of SpringDragon.
The result is a lovely blend of Glitter's ability to follow instructions and Confetti's free-spirit prose, effectively 'loosening up' much of the hesitancy that was left in Glitter.
overrides:
parameters:
model: Gemma-3-Starshine-12B.i1-Q4_K_M.gguf
files:
- filename: Gemma-3-Starshine-12B.i1-Q4_K_M.gguf
sha256: 4c35a678e3784e20a8d85d4e7045d965509a1a71305a0da105fc5991ba7d6dc4
uri: huggingface://mradermacher/Gemma-3-Starshine-12B-i1-GGUF/Gemma-3-Starshine-12B.i1-Q4_K_M.gguf
- &eurollm
name: "eurollm-9b-instruct"
icon: https://openeurollm.eu/_next/static/media/logo-dark.e7001867.svg
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
license: apache-2.0
tags:
- llm
- gguf
- eurollm
- cpu
- gpu
- text-generation
urls:
- https://huggingface.co/utter-project/EuroLLM-9B-Instruct
- https://huggingface.co/bartowski/EuroLLM-9B-Instruct-GGUF
description: |
The EuroLLM project has the goal of creating a suite of LLMs capable of understanding and generating text in all European Union languages as well as some additional relevant languages. EuroLLM-9B is a 9B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets. EuroLLM-9B-Instruct was further instruction tuned on EuroBlocks, an instruction tuning dataset with focus on general instruction-following and machine translation.
overrides:
parameters:
model: EuroLLM-9B-Instruct-Q4_K_M.gguf
files:
- filename: EuroLLM-9B-Instruct-Q4_K_M.gguf
sha256: 785a3b2883532381704ef74f866f822f179a931801d1ed1cf12e6deeb838806b
uri: huggingface://bartowski/EuroLLM-9B-Instruct-GGUF/EuroLLM-9B-Instruct-Q4_K_M.gguf
- &phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat.yaml@master"
name: "phi-4"
icon: https://avatars.githubusercontent.com/u/6154722
license: mit
tags:
- llm
- gguf
- phi
- cpu
- gpu
- text-generation
urls:
- https://huggingface.co/microsoft/phi-4
- https://huggingface.co/bartowski/phi-4-GGUF
description: |
phi-4 is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.
phi-4 underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures. Phi-4 is a 14B parameters, dense decoder-only Transformer model.
overrides:
parameters:
model: phi-4-Q4_K_M.gguf
files:
- filename: phi-4-Q4_K_M.gguf
uri: huggingface://bartowski/phi-4-GGUF/phi-4-Q4_K_M.gguf
sha256: 009aba717c09d4a35890c7d35eb59d54e1dba884c7c526e7197d9c13ab5911d9
- !!merge <<: *phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat-fcall.yaml@master"
name: "LocalAI-functioncall-phi-4-v0.3"
icon: https://cdn-uploads.huggingface.co/production/uploads/647374aa7ff32a81ac6d35d4/Dzbdzn27KEc3K6zNNi070.png
urls:
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.3
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.3-Q4_K_M-GGUF
description: |
A model tailored to be conversational and execute function calls with LocalAI. This model is based on phi-4.
overrides:
parameters:
model: localai-functioncall-phi-4-v0.3-q4_k_m.gguf
files:
- filename: localai-functioncall-phi-4-v0.3-q4_k_m.gguf
sha256: 23fee048ded2a6e2e1a7b6bbefa6cbf83068f194caa9552aecbaa00fec8a16d5
uri: huggingface://mudler/LocalAI-functioncall-phi-4-v0.3-Q4_K_M-GGUF/localai-functioncall-phi-4-v0.3-q4_k_m.gguf
- !!merge <<: *phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat-fcall.yaml@master"
name: "LocalAI-functioncall-phi-4-v0.2"
icon: https://cdn-uploads.huggingface.co/production/uploads/647374aa7ff32a81ac6d35d4/Dzbdzn27KEc3K6zNNi070.png
description: |
A model tailored to be conversational and execute function calls with LocalAI. This model is based on phi-4.
This is the second iteration of https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.1 with added CoT (o1) capabilities from the marco-o1 dataset.
urls:
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.2
- https://huggingface.co/mudler/localai-functioncall-phi-4-v0.2-Q4_K_M-GGUF
overrides:
parameters:
model: localai-functioncall-phi-4-v0.2-q4_k_m.gguf
files:
- filename: localai-functioncall-phi-4-v0.2-q4_k_m.gguf
uri: huggingface://mudler/localai-functioncall-phi-4-v0.2-Q4_K_M-GGUF/localai-functioncall-phi-4-v0.2-q4_k_m.gguf
sha256: 681b5fb5070f23323a9cc8cbd1306b1c348c2f292041d3ba2335b26b071757b7
- !!merge <<: *phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat-fcall.yaml@master"
name: "LocalAI-functioncall-phi-4-v0.1"
icon: https://cdn-uploads.huggingface.co/production/uploads/647374aa7ff32a81ac6d35d4/Dzbdzn27KEc3K6zNNi070.png
description: |
A model tailored to be conversational and execute function calls with LocalAI. This model is based on phi-4.
urls:
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.1
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.1-Q4_K_M-GGUF
overrides:
parameters:
model: localai-functioncall-phi-4-v0.1-q4_k_m.gguf
files:
- filename: localai-functioncall-phi-4-v0.1-q4_k_m.gguf
uri: huggingface://mudler/LocalAI-functioncall-phi-4-v0.1-Q4_K_M-GGUF/localai-functioncall-phi-4-v0.1-q4_k_m.gguf
sha256: 0ae4e5e4ba89c16c1e810285c5c8b84416fa67f8ed7c175aa0b6fc0a103017aa
- !!merge <<: *phi4
name: "sicariussicariistuff_phi-lthy4"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
icon: https://huggingface.co/SicariusSicariiStuff/Phi-lthy4/resolve/main/Images/Phi-Lthy4.png
urls:
- https://huggingface.co/SicariusSicariiStuff/Phi-lthy4
- https://huggingface.co/bartowski/SicariusSicariiStuff_Phi-lthy4-GGUF
description: |
- The BEST Phi-4 Roleplay finetune in the world (Not that much of an achievement here, Phi roleplay finetunes can probably be counted on a single hand).
- Compact size & fully healed from the brain surgery Only 11.9B parameters. Phi-4 wasn't that hard to run even at 14B, now with even fewer brain cells, your new phone could probably run it easily. (SD8Gen3 and above recommended).
- Strong Roleplay & Creative writing abilities. This really surprised me. Actually good.
Writes and roleplays quite uniquely, probably because of lack of RP\writing slop in the pretrain. Who would have thought?
- Smart assistant with low refusals - It kept some of the smarts, and our little Phi-Lthy here will be quite eager to answer your naughty questions.
- Quite good at following the character card. Finally, it puts its math brain to some productive tasks. Gooner technology is becoming more popular by the day.
overrides:
parameters:
model: SicariusSicariiStuff_Phi-lthy4-Q4_K_M.gguf
files:
- filename: SicariusSicariiStuff_Phi-lthy4-Q4_K_M.gguf
sha256: a5004b2d0f3eb869f07285f53ec283aa383063085987113d2a41cb54708fb5ad
uri: huggingface://bartowski/SicariusSicariiStuff_Phi-lthy4-GGUF/SicariusSicariiStuff_Phi-lthy4-Q4_K_M.gguf
- !!merge <<: *phi4
name: "sicariussicariistuff_phi-line_14b"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
icon: https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B/resolve/main/Images/Phi-Line_14B.png
urls:
- https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B
- https://huggingface.co/bartowski/SicariusSicariiStuff_Phi-Line_14B-GGUF
description: |
Excellent Roleplay with more brains. (Who would have thought Phi-4 models would be good at this? so weird... )
Medium length response (1-4 paragraphs, usually 2-3).
Excellent assistant that follows instructions well enough, and keeps good formating.
Strong Creative writing abilities. Will obey requests regarding formatting (markdown headlines for paragraphs, etc).
Writes and roleplays quite uniquely, probably because of lack of RP\writing slop in the pretrain. This is just my guesstimate.
LOW refusals - Total freedom in RP, can do things other RP models won't, and I'll leave it at that. Low refusals in assistant tasks as well.
VERY good at following the character card. Math brain is used for gooner tech, as it should be.
overrides:
parameters:
model: SicariusSicariiStuff_Phi-Line_14B-Q4_K_M.gguf
files:
- filename: SicariusSicariiStuff_Phi-Line_14B-Q4_K_M.gguf
sha256: 552c5a613bc5f24494646858795837ac42d3c216c5caedd7f4d6b954e5df58f2
uri: huggingface://bartowski/SicariusSicariiStuff_Phi-Line_14B-GGUF/SicariusSicariiStuff_Phi-Line_14B-Q4_K_M.gguf
- !!merge <<: *phi4
name: "microsoft_phi-4-mini-instruct"
urls:
- https://huggingface.co/microsoft/Phi-4-mini-instruct
- https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF
description: |
Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
overrides:
parameters:
model: microsoft_Phi-4-mini-instruct-Q4_K_M.gguf
files:
- filename: microsoft_Phi-4-mini-instruct-Q4_K_M.gguf
sha256: 01999f17c39cc3074afae5e9c539bc82d45f2dd7faa3917c66cbef76fce8c0c2
uri: huggingface://bartowski/microsoft_Phi-4-mini-instruct-GGUF/microsoft_Phi-4-mini-instruct-Q4_K_M.gguf
- &falcon3
name: "falcon3-1b-instruct"
url: "github:mudler/LocalAI/gallery/falcon3.yaml@master"
icon: https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/general/falco3-logo.png
urls:
- https://huggingface.co/tiiuae/Falcon3-1B-Instruct
- https://huggingface.co/bartowski/Falcon3-1B-Instruct-GGUF
description: |
Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B parameters.
This repository contains the Falcon3-1B-Instruct. It achieves strong results on reasoning, language understanding, instruction following, code and mathematics tasks. Falcon3-1B-Instruct supports 4 languages (English, French, Spanish, Portuguese) and a context length of up to 8K.
overrides:
parameters:
model: Falcon3-1B-Instruct-Q4_K_M.gguf
files:
- filename: Falcon3-1B-Instruct-Q4_K_M.gguf
uri: huggingface://bartowski/Falcon3-1B-Instruct-GGUF/Falcon3-1B-Instruct-Q4_K_M.gguf
sha256: 1c92013dac1ab6e703e787f3e0829ca03cc95311e4c113a77950d15ff6dea7b3
tags:
- llm
- gguf
- gpu
- cpu
- falcon
license: falcon-llm
- !!merge <<: *falcon3
name: "falcon3-3b-instruct"
urls:
- https://huggingface.co/tiiuae/Falcon3-3B-Instruct
- https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF
overrides:
parameters:
model: Falcon3-3B-Instruct-Q4_K_M.gguf
files:
- filename: Falcon3-3B-Instruct-Q4_K_M.gguf
uri: huggingface://bartowski/Falcon3-3B-Instruct-GGUF/Falcon3-3B-Instruct-Q4_K_M.gguf
sha256: 6ea6cecba144fe5b711ca07ae4263ccdf6ee6419807a46220419189da8446557
- !!merge <<: *falcon3
name: "falcon3-10b-instruct"
urls:
- https://huggingface.co/tiiuae/Falcon3-10B-Instruct
- https://huggingface.co/bartowski/Falcon3-10B-Instruct-GGUF
overrides:
parameters:
model: Falcon3-10B-Instruct-Q4_K_M.gguf
files:
- filename: Falcon3-10B-Instruct-Q4_K_M.gguf
uri: huggingface://bartowski/Falcon3-10B-Instruct-GGUF/Falcon3-10B-Instruct-Q4_K_M.gguf
sha256: 0a33327bd71e1788a8e9f17889824a17a65efd3f96a4b2a5e2bc6ff2f39b8241
- !!merge <<: *falcon3
name: "falcon3-1b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-1B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-1B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-1B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-1B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-1B-Instruct-abliterated-Q4_K_M.gguf
sha256: 416d15ce58334b7956818befb088d46c1e3e7153ebf2da2fb9769a5b1ff934a1
uri: huggingface://bartowski/Falcon3-1B-Instruct-abliterated-GGUF/Falcon3-1B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "falcon3-3b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-3B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-3B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-3B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-3B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-3B-Instruct-abliterated-Q4_K_M.gguf
sha256: 83773b77b0e34ef115f8a6508192e9f1d3426a61456744493f65cfe1e7f90aa9
uri: huggingface://bartowski/Falcon3-3B-Instruct-abliterated-GGUF/Falcon3-3B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "falcon3-10b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-10B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-10B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-10B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-10B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-10B-Instruct-abliterated-Q4_K_M.gguf
sha256: 5940df2ff88e5be93dbe0766b2a9683d7e73c204a69a1348a37f835cf2b5f767
uri: huggingface://bartowski/Falcon3-10B-Instruct-abliterated-GGUF/Falcon3-10B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "falcon3-7b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-7B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-7B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-7B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-7B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-7B-Instruct-abliterated-Q4_K_M.gguf
sha256: 68e10e638668acaa49fb7919224c7d8bcf1798126c7a499c4d9ec3b81313f8c8
uri: huggingface://bartowski/Falcon3-7B-Instruct-abliterated-GGUF/Falcon3-7B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "nightwing3-10b-v0.1"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
icon: https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/C6gY9vxCl3_SFzQLpLG0S.png
urls:
- https://huggingface.co/Nitral-AI/NightWing3-10B-v0.1
- https://huggingface.co/bartowski/NightWing3-10B-v0.1-GGUF
description: |
Base model: (Falcon3-10B)
overrides:
parameters:
model: NightWing3-10B-v0.1-Q4_K_M.gguf
files:
- filename: NightWing3-10B-v0.1-Q4_K_M.gguf
sha256: 2e87671542d22fe1ef9a68e43f2fdab7c2759479ad531946d9f0bdeffa6f5747
uri: huggingface://bartowski/NightWing3-10B-v0.1-GGUF/NightWing3-10B-v0.1-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "virtuoso-lite"
urls:
- https://huggingface.co/arcee-ai/Virtuoso-Lite
- https://huggingface.co/bartowski/Virtuoso-Lite-GGUF
description: |
Virtuoso-Lite (10B) is our next-generation, 10-billion-parameter language model based on the Llama-3 architecture. It is distilled from Deepseek-v3 using ~1.1B tokens/logits, allowing it to achieve robust performance at a significantly reduced parameter count compared to larger models. Despite its compact size, Virtuoso-Lite excels in a variety of tasks, demonstrating advanced reasoning, code generation, and mathematical problem-solving capabilities.
overrides:
parameters:
model: Virtuoso-Lite-Q4_K_M.gguf
files:
- filename: Virtuoso-Lite-Q4_K_M.gguf
sha256: 1d21bef8467a11a1e473d397128b05fb87b7e824606cdaea061e550cb219fee2
uri: huggingface://bartowski/Virtuoso-Lite-GGUF/Virtuoso-Lite-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "suayptalha_maestro-10b"
icon: https://huggingface.co/suayptalha/Maestro-10B/resolve/main/Maestro-Logo.png
urls:
- https://huggingface.co/suayptalha/Maestro-10B
- https://huggingface.co/bartowski/suayptalha_Maestro-10B-GGUF
description: |
Maestro-10B is a 10 billion parameter model fine-tuned from Virtuoso-Lite, a next-generation language model developed by arcee-ai. Virtuoso-Lite itself is based on the Llama-3 architecture, distilled from Deepseek-v3 using approximately 1.1 billion tokens/logits. This distillation process allows Virtuoso-Lite to achieve robust performance with a smaller parameter count, excelling in reasoning, code generation, and mathematical problem-solving. Maestro-10B inherits these strengths from its base model, Virtuoso-Lite, and further enhances them through fine-tuning on the OpenOrca dataset. This combination of a distilled base model and targeted fine-tuning makes Maestro-10B a powerful and efficient language model.
overrides:
parameters:
model: suayptalha_Maestro-10B-Q4_K_M.gguf
files:
- filename: suayptalha_Maestro-10B-Q4_K_M.gguf
sha256: c570381da5624782ce6df4186ace6f747429fcbaf1a22c2a348288d3552eb19c
uri: huggingface://bartowski/suayptalha_Maestro-10B-GGUF/suayptalha_Maestro-10B-Q4_K_M.gguf
- &intellect1
name: "intellect-1-instruct"
url: "github:mudler/LocalAI/gallery/llama3.1-instruct.yaml@master"
icon: https://huggingface.co/PrimeIntellect/INTELLECT-1-Instruct/resolve/main/intellect-1-map.png
urls:
- https://huggingface.co/PrimeIntellect/INTELLECT-1-Instruct
- https://huggingface.co/bartowski/INTELLECT-1-Instruct-GGUF
tags:
- llm
- gguf
- gpu
- cpu
- intellect
license: apache-2.0
description: |
INTELLECT-1 is the first collaboratively trained 10 billion parameter language model trained from scratch on 1 trillion tokens of English text and code.
This is an instruct model. The base model associated with it is INTELLECT-1.
INTELLECT-1 was trained on up to 14 concurrent nodes distributed across 3 continents, with contributions from 30 independent community contributors providing compute. The training code utilizes the prime framework, a scalable distributed training framework designed for fault-tolerant, dynamically scaling, high-perfomance training on unreliable, globally distributed workers. The key abstraction that allows dynamic scaling is the ElasticDeviceMesh which manages dynamic global process groups for fault-tolerant communication across the internet and local process groups for communication within a node. The model was trained using the DiLoCo algorithms with 100 inner steps. The global all-reduce was done with custom int8 all-reduce kernels to reduce the communication payload required, greatly reducing the communication overhead by a factor 400x.
overrides:
parameters:
model: INTELLECT-1-Instruct-Q4_K_M.gguf
files:
- filename: INTELLECT-1-Instruct-Q4_K_M.gguf
sha256: 5df236fe570e5998d07fb3207788eac811ef3b77dd2a0ad04a2ef5c6361f3030
uri: huggingface://bartowski/INTELLECT-1-Instruct-GGUF/INTELLECT-1-Instruct-Q4_K_M.gguf
- &llama33
url: "github:mudler/LocalAI/gallery/llama3.1-instruct.yaml@master"
icon: https://avatars.githubusercontent.com/u/153379578
license: llama3.3
description: |
The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
tags:
- llm
- gguf
- gpu
- cpu
- llama3.3
name: "llama-3.3-70b-instruct"
urls:
- https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct
- https://huggingface.co/MaziyarPanahi/Llama-3.3-70B-Instruct-GGUF
overrides:
parameters:
model: Llama-3.3-70B-Instruct.Q4_K_M.gguf
files:
- filename: Llama-3.3-70B-Instruct.Q4_K_M.gguf
sha256: 4f3b04ecae278bdb0fd545b47c210bc5edf823e5ebf7d41e0b526c81d54b1ff3
uri: huggingface://MaziyarPanahi/Llama-3.3-70B-Instruct-GGUF/Llama-3.3-70B-Instruct.Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-70b-euryale-v2.3"
icon: https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3/resolve/main/Eury.png
urls:
- https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3
- https://huggingface.co/bartowski/L3.3-70B-Euryale-v2.3-GGUF
description: |
A direct replacement / successor to Euryale v2.2, not Hanami-x1, though it is slightly better than them in my opinion.
overrides:
parameters:
model: L3.3-70B-Euryale-v2.3-Q4_K_M.gguf
files:
- filename: L3.3-70B-Euryale-v2.3-Q4_K_M.gguf
sha256: 4e78bb0e65886bfcff89b829f6d38aa6f6846988bb8291857e387e3f60b3217b
uri: huggingface://bartowski/L3.3-70B-Euryale-v2.3-GGUF/L3.3-70B-Euryale-v2.3-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-ms-evayale-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/HFCaVzRpiE05Y46p41qRy.webp
urls:
- https://huggingface.co/Steelskull/L3.3-MS-Evayale-70B
- https://huggingface.co/bartowski/L3.3-MS-Evayale-70B-GGUF
description: |
This model was created as I liked the storytelling of EVA but the prose and details of scenes from EURYALE, my goal is to merge the robust storytelling of both models while attempting to maintain the positives of both models.
overrides:
parameters:
model: L3.3-MS-Evayale-70B-Q4_K_M.gguf
files:
- filename: L3.3-MS-Evayale-70B-Q4_K_M.gguf
sha256: f941d88870fec8343946517a1802d159d23f3971eeea50b6cf12295330bd29cc
uri: huggingface://bartowski/L3.3-MS-Evayale-70B-GGUF/L3.3-MS-Evayale-70B-Q4_K_M.gguf
- !!merge <<: *llama33
name: "anubis-70b-v1"
icon: https://cdn-uploads.huggingface.co/production/uploads/65f2fd1c25b848bd061b5c2e/qQbZvnrWYvH8dMZORLBJn.webp
urls:
- https://huggingface.co/TheDrummer/Anubis-70B-v1
- https://huggingface.co/bartowski/Anubis-70B-v1-GGUF
description: |
It's a very balanced model between the L3.3 tunes. It's very creative, able to come up with new and interesting scenarios on your own that will thoroughly surprise you in ways that remind me of a 123B model. It has some of the most natural sounding dialogue and prose can come out of any model I've tried with the right swipe, in a way that truly brings your characters and RP to life that makes you feel like you're talking to a human writer instead of an AI - a quality that reminds me of Character AI in its prime. This model loves a great prompt and thrives off instructions.
overrides:
parameters:
model: Anubis-70B-v1-Q4_K_M.gguf
files:
- filename: Anubis-70B-v1-Q4_K_M.gguf
sha256: 9135f7090c675726469bd3a108cfbdddaa18638bad8e513928410de4b8bfd4d4
uri: huggingface://bartowski/Anubis-70B-v1-GGUF/Anubis-70B-v1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "llama-3.3-70b-instruct-ablated"
icon: https://cdn-uploads.huggingface.co/production/uploads/6587d8dd1b44d0e694104fbf/0dkt6EhZYwXVBxvSWXdaM.png
urls:
- https://huggingface.co/NaniDAO/Llama-3.3-70B-Instruct-ablated
- https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-ablated-GGUF
description: |
Llama 3.3 instruct 70B 128k context with ablation technique applied for a more helpful (and based) assistant.
This means it will refuse less of your valid requests for an uncensored UX. Use responsibly and use common sense.
We do not take any responsibility for how you apply this intelligence, just as we do not for how you apply your own.
overrides:
parameters:
model: Llama-3.3-70B-Instruct-ablated-Q4_K_M.gguf
files:
- filename: Llama-3.3-70B-Instruct-ablated-Q4_K_M.gguf
sha256: 090b2288810c5f6f680ff5cb4bc97665393d115c011fcd54dca6aec02e74a983
uri: huggingface://bartowski/Llama-3.3-70B-Instruct-ablated-GGUF/Llama-3.3-70B-Instruct-ablated-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-ms-evalebis-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/e49ykknqXee3Ihr-3BIl_.png
urls:
- https://huggingface.co/Steelskull/L3.3-MS-Evalebis-70b
- https://huggingface.co/bartowski/L3.3-MS-Evalebis-70b-GGUF
description: |
This model was created as I liked the storytelling of EVA, the prose and details of scenes from EURYALE and Anubis, my goal is to merge the robust storytelling of all three models while attempting to maintain the positives of the models.
overrides:
parameters:
model: L3.3-MS-Evalebis-70b-Q4_K_M.gguf
files:
- filename: L3.3-MS-Evalebis-70b-Q4_K_M.gguf
sha256: 5515110ab6a583f6eb360533e3c5b3dda6d402af407c0b0f2b34a2a57b5224d5
uri: huggingface://bartowski/L3.3-MS-Evalebis-70b-GGUF/L3.3-MS-Evalebis-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "rombos-llm-70b-llama-3.3"
icon: "https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/QErypCEKD5OZLxUcSmYaR.jpeg"
urls:
- https://huggingface.co/rombodawg/Rombos-LLM-70b-Llama-3.3
- https://huggingface.co/bartowski/Rombos-LLM-70b-Llama-3.3-GGUF
- https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing
description: |
You know the drill by now.
Here is the paper. Have fun.
https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing
overrides:
parameters:
model: Rombos-LLM-70b-Llama-3.3-Q4_K_M.gguf
files:
- filename: Rombos-LLM-70b-Llama-3.3-Q4_K_M.gguf
uri: huggingface://bartowski/Rombos-LLM-70b-Llama-3.3-GGUF/Rombos-LLM-70b-Llama-3.3-Q4_K_M.gguf
sha256: 613008b960f6fff346b5dec71a87cd7ecdaff205bfea6332bd8fe2bb46177352
- !!merge <<: *llama33
name: "70b-l3.3-cirrus-x1"
icon: https://huggingface.co/Sao10K/70B-L3.3-Cirrus-x1/resolve/main/venti.png
urls:
- https://huggingface.co/Sao10K/70B-L3.3-Cirrus-x1
- https://huggingface.co/bartowski/70B-L3.3-Cirrus-x1-GGUF
description: |
- Same data composition as Freya, applied differently, trained longer too.
- Merging with its checkpoints was also involved.
- Has a nice style, with occasional issues that can be easily fixed.
- A more stable version compared to previous runs.
overrides:
parameters:
model: 70B-L3.3-Cirrus-x1-Q4_K_M.gguf
files:
- filename: 70B-L3.3-Cirrus-x1-Q4_K_M.gguf
sha256: 07dd464dddba959df8eb2f937787c2210b4c51c2375bd7c7ab2abbe198142a19
uri: huggingface://bartowski/70B-L3.3-Cirrus-x1-GGUF/70B-L3.3-Cirrus-x1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "negative_llama_70b"
icon: https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B/resolve/main/Images/Negative_LLAMA_70B.png
urls:
- https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B
- https://huggingface.co/bartowski/Negative_LLAMA_70B-GGUF
description: |
- Strong Roleplay & Creative writing abilities.
- Less positivity bias.
- Very smart assistant with low refusals.
- Exceptionally good at following the character card.
- Characters feel more 'alive', and will occasionally initiate stuff on their own (without being prompted to, but fitting to their character).
- Strong ability to comprehend and roleplay uncommon physical and mental characteristics.
overrides:
parameters:
model: Negative_LLAMA_70B-Q4_K_M.gguf
files:
- filename: Negative_LLAMA_70B-Q4_K_M.gguf
sha256: 023c6bd38f6a66178529e6bb77b6e76379ae3ee031adc6885531986aa12750d9
uri: huggingface://bartowski/Negative_LLAMA_70B-GGUF/Negative_LLAMA_70B-Q4_K_M.gguf
- !!merge <<: *llama33
name: "negative-anubis-70b-v1"
icon: https://huggingface.co/knifeayumu/Negative-Anubis-70B-v1/resolve/main/Negative-Anubis.png
urls:
- https://huggingface.co/knifeayumu/Negative-Anubis-70B-v1
- https://huggingface.co/bartowski/Negative-Anubis-70B-v1-GGUF
description: |
Enjoyed SicariusSicariiStuff/Negative_LLAMA_70B but the prose was too dry for my tastes. So I merged it with TheDrummer/Anubis-70B-v1 for verbosity. Anubis has positivity bias so Negative could balance things out.
This is a merge of pre-trained language models created using mergekit.
The following models were included in the merge:
SicariusSicariiStuff/Negative_LLAMA_70B
TheDrummer/Anubis-70B-v1
overrides:
parameters:
model: Negative-Anubis-70B-v1-Q4_K_M.gguf
files:
- filename: Negative-Anubis-70B-v1-Q4_K_M.gguf
sha256: ac088da9ca70fffaa70c876fbada9fc5a02e7d6049ef68f16b11a9c3256f2510
uri: huggingface://bartowski/Negative-Anubis-70B-v1-GGUF/Negative-Anubis-70B-v1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-ms-nevoria-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/dtlCF4LbekmDD2y3LNpdH.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-MS-Nevoria-70b
- https://huggingface.co/bartowski/L3.3-MS-Nevoria-70b-GGUF
description: |
This model was created as I liked the storytelling of EVA, the prose and details of scenes from EURYALE and Anubis, enhanced with Negative_LLAMA to kill off the positive bias with a touch of nemotron sprinkeled in.
The choice to use the lorablated model as a base was intentional - while it might seem counterintuitive, this approach creates unique interactions between the weights, similar to what was achieved in the original Astoria model and Astoria V2 model . Rather than simply removing refusals, this "weight twisting" effect that occurs when subtracting the lorablated base model from the other models during the merge process creates an interesting balance in the final model's behavior. While this approach differs from traditional sequential application of components, it was chosen for its unique characteristics in the model's responses.
overrides:
parameters:
model: L3.3-MS-Nevoria-70b-Q4_K_M.gguf
files:
- filename: L3.3-MS-Nevoria-70b-Q4_K_M.gguf
sha256: e8b0763f263089a19d4b112b7ed5085cc5f1ed9ca49c5085baa8d51f4ded1f94
uri: huggingface://bartowski/L3.3-MS-Nevoria-70b-GGUF/L3.3-MS-Nevoria-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-70b-magnum-v4-se"
urls:
- https://huggingface.co/Doctor-Shotgun/L3.3-70B-Magnum-v4-SE
- https://huggingface.co/bartowski/L3.3-70B-Magnum-v4-SE-GGUF
description: |
The Magnum v4 series is complete, but here's something a little extra I wanted to tack on as I wasn't entirely satisfied with the results of v4 72B. "SE" for Special Edition - this model is finetuned from meta-llama/Llama-3.3-70B-Instruct as an rsLoRA adapter. The dataset is a slightly revised variant of the v4 data with some elements of the v2 data re-introduced.
The objective, as with the other Magnum models, is to emulate the prose style and quality of the Claude 3 Sonnet/Opus series of models on a local scale, so don't be surprised to see "Claude-isms" in its output.
overrides:
parameters:
model: L3.3-70B-Magnum-v4-SE-Q4_K_M.gguf
files:
- filename: L3.3-70B-Magnum-v4-SE-Q4_K_M.gguf
sha256: 9724a6364a42caa3d5a1687258eb329c9af6cbb2ce01c8dd556c1a222a2e0352
uri: huggingface://bartowski/L3.3-70B-Magnum-v4-SE-GGUF/L3.3-70B-Magnum-v4-SE-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-prikol-70b-v0.2"
icon: https://files.catbox.moe/x9t3zo.png
urls:
- https://huggingface.co/Nohobby/L3.3-Prikol-70B-v0.2
- https://huggingface.co/bartowski/L3.3-Prikol-70B-v0.2-GGUF
description: |
A merge of some Llama 3.3 models because um uh yeah
Went extra schizo on the recipe, hoping for an extra fun result, and... Well, I guess it's an overall improvement over the previous revision. It's a tiny bit smarter, has even more distinct swipes and nice dialogues, but for some reason it's damn sloppy.
I've published the second step of this merge as a separate model, and I'd say the results are more interesting, but not as usable as this one. https://huggingface.co/Nohobby/AbominationSnowPig
Prompt format: Llama3 OR Llama3 Context and ChatML Instruct. It actually works a bit better this way
overrides:
parameters:
model: L3.3-Prikol-70B-v0.2-Q4_K_M.gguf
files:
- filename: L3.3-Prikol-70B-v0.2-Q4_K_M.gguf
sha256: fc0ff514efbc0b67981c2bf1423d5a2e1b8801e4266ba0c653ea148414fe5ffc
uri: huggingface://bartowski/L3.3-Prikol-70B-v0.2-GGUF/L3.3-Prikol-70B-v0.2-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-nevoria-r1-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/_oWpsvCZ-graNKzJBBjGo.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-Nevoria-R1-70b
- https://huggingface.co/bartowski/L3.3-Nevoria-R1-70b-GGUF
description: |
This model builds upon the original Nevoria foundation, incorporating the Deepseek-R1 reasoning architecture to enhance dialogue interaction and scene comprehension. While maintaining Nevoria's core strengths in storytelling and scene description (derived from EVA, EURYALE, and Anubis), this iteration aims to improve prompt adherence and creative reasoning capabilities. The model also retains the balanced perspective introduced by Negative_LLAMA and Nemotron elements. Also, the model plays the card to almost a fault, It'll pick up on minor issues and attempt to run with them. Users had it call them out for misspelling a word while playing in character.
Note: While Nevoria-R1 represents a significant architectural change, rather than a direct successor to Nevoria, it operates as a distinct model with its own characteristics.
The lorablated model base choice was intentional, creating unique weight interactions similar to the original Astoria model and Astoria V2 model. This "weight twisting" effect, achieved by subtracting the lorablated base model during merging, creates an interesting balance in the model's behavior. While unconventional compared to sequential component application, this approach was chosen for its unique response characteristics.
overrides:
parameters:
model: L3.3-Nevoria-R1-70b-Q4_K_M.gguf
files:
- filename: L3.3-Nevoria-R1-70b-Q4_K_M.gguf
sha256: 9f32f202fb5b1465c942693bb11eea9e8a1c5686b00602715b495c068eaf1c58
uri: huggingface://bartowski/L3.3-Nevoria-R1-70b-GGUF/L3.3-Nevoria-R1-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "nohobby_l3.3-prikol-70b-v0.4"
icon: https://files.catbox.moe/x9t3zo.png
urls:
- https://huggingface.co/Nohobby/L3.3-Prikol-70B-v0.4
- https://huggingface.co/bartowski/Nohobby_L3.3-Prikol-70B-v0.4-GGUF
description: |
I have yet to try it UPD: it sucks, bleh
Sometimes mistakes {{user}} for {{char}} and can't think. Other than that, the behavior is similar to the predecessors.
It sometimes gives some funny replies tho, yay!
overrides:
parameters:
model: Nohobby_L3.3-Prikol-70B-v0.4-Q4_K_M.gguf
files:
- filename: Nohobby_L3.3-Prikol-70B-v0.4-Q4_K_M.gguf
sha256: e1d67a40bdf0526bdfcaa16c6e4dfeecad41651e201b4009b65f4f444b773604
uri: huggingface://bartowski/Nohobby_L3.3-Prikol-70B-v0.4-GGUF/Nohobby_L3.3-Prikol-70B-v0.4-Q4_K_M.gguf
- !!merge <<: *llama33
name: "arliai_llama-3.3-70b-arliai-rpmax-v1.4"
urls:
- https://huggingface.co/ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4
- https://huggingface.co/bartowski/ArliAI_Llama-3.3-70B-ArliAI-RPMax-v1.4-GGUF
description: |
RPMax is a series of models that are trained on a diverse set of curated creative writing and RP datasets with a focus on variety and deduplication. This model is designed to be highly creative and non-repetitive by making sure no two entries in the dataset have repeated characters or situations, which makes sure the model does not latch on to a certain personality and be capable of understanding and acting appropriately to any characters or situations.