Skip to content

Commit 1247e55

Browse files
Cabir40Cabir40
authored
Cc md update 11 21 (#1618)
1 parent 42ab3d1 commit 1247e55

8 files changed

+225
-14
lines changed

‎docs/_posts/Cabir40/2024-07-12-jsl_medm_q16_v1_en.md

Lines changed: 38 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,12 @@ supported: true
1313
annotator: LLMLoader
1414
article_header:
1515
type: cover
16-
use_language_switcher: "Python-Scala-Java"
17-
18-
deploy:
19-
sagemaker_link: https://aws.amazon.com/marketplace/pp/prodview-yrajldynampw4
20-
snowflake_link:
21-
databricks_link:
16+
use_language_switcher: "Python-Scala-Java"
17+
18+
deploy:
19+
sagemaker_link: https://aws.amazon.com/marketplace/pp/prodview-yrajldynampw4
20+
snowflake_link:
21+
databricks_link:
2222

2323
---
2424

@@ -38,13 +38,13 @@ This LLM model is trained to perform Q&A, Summarization, RAG, and Chat.
3838
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/jsl_medm_q16_v1_en_5.4.0_3.0_1720040078717.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden}
3939
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/clinical/models/jsl_medm_q16_v1_en_5.4.0_3.0_1720040078717.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}
4040

41-
{% if page.deploy %}
42-
## Available as Private API Endpoint
43-
44-
{:.tac}
45-
{% include display_platform_information.html %}
46-
{% endif %}
47-
41+
{% if page.deploy %}
42+
## Available as Private API Endpoint
43+
44+
{:.tac}
45+
{% include display_platform_information.html %}
46+
{% endif %}
47+
4848
## How to use
4949

5050

@@ -112,3 +112,28 @@ The patient is presenting with symptoms of urinary tract infection (UTI), which
112112

113113

114114

115+
## Benchmarking
116+
117+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
118+
119+
```bash
120+
## Overall
121+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
122+
|----------|--------------|----------------------|---------------|
123+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
124+
| ChatGPT | 0.21 | 0.30 | 0.26 |
125+
| Neutral | 0.43 | 0.38 | 0.17 |
126+
| None | 0.07 | 0.07 | 0.08 |
127+
| total | 1.00 | 1.00 | 1.00 |
128+
129+
## Summary
130+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
131+
|----------|--------------|----------------------|---------------|
132+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
133+
| GPT4o | 0.33 | 0.33 | 0.28 |
134+
| Neutral | 0.17 | 0.17 | 0.12 |
135+
| None | 0.08 | 0.08 | 0.10 |
136+
| Total | 1.00 | 1.00 | 1.00 |
137+
138+
...
139+
```

‎docs/_posts/Cabir40/2024-07-12-jsl_medm_q4_v1_en.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,3 +99,28 @@ The patient is presenting with symptoms of urinary tract infection (UTI), which
9999

100100

101101

102+
## Benchmarking
103+
104+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
105+
106+
```bash
107+
## Overall
108+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
109+
|----------|--------------|----------------------|---------------|
110+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
111+
| ChatGPT | 0.21 | 0.30 | 0.26 |
112+
| Neutral | 0.43 | 0.38 | 0.17 |
113+
| None | 0.07 | 0.07 | 0.08 |
114+
| total | 1.00 | 1.00 | 1.00 |
115+
116+
## Summary
117+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
118+
|----------|--------------|----------------------|---------------|
119+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
120+
| GPT4o | 0.33 | 0.33 | 0.28 |
121+
| Neutral | 0.17 | 0.17 | 0.12 |
122+
| None | 0.08 | 0.08 | 0.10 |
123+
| Total | 1.00 | 1.00 | 1.00 |
124+
125+
...
126+
```

‎docs/_posts/Cabir40/2024-07-12-jsl_medm_q8_v1_en.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,3 +99,28 @@ The patient is presenting with symptoms of urinary tract infection (UTI), which
9999

100100

101101

102+
## Benchmarking
103+
104+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
105+
106+
```bash
107+
## Overall
108+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
109+
|----------|--------------|----------------------|---------------|
110+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
111+
| ChatGPT | 0.21 | 0.30 | 0.26 |
112+
| Neutral | 0.43 | 0.38 | 0.17 |
113+
| None | 0.07 | 0.07 | 0.08 |
114+
| total | 1.00 | 1.00 | 1.00 |
115+
116+
## Summary
117+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
118+
|----------|--------------|----------------------|---------------|
119+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
120+
| GPT4o | 0.33 | 0.33 | 0.28 |
121+
| Neutral | 0.17 | 0.17 | 0.12 |
122+
| None | 0.08 | 0.08 | 0.10 |
123+
| Total | 1.00 | 1.00 | 1.00 |
124+
125+
...
126+
```

‎docs/_posts/akrztrk/2024-10-01-jsl_medm_q8_v1_en.md

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,4 +214,32 @@ KISUNLA is an amyloid beta-directed antibody indicated for the treatment of Alzh
214214
|License:|Licensed|
215215
|Edition:|Official|
216216
|Language:|en|
217-
|Size:|8.2 GB|
217+
|Size:|8.2 GB|
218+
219+
220+
221+
## Benchmarking
222+
223+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
224+
225+
```bash
226+
## Overall
227+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
228+
|----------|--------------|----------------------|---------------|
229+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
230+
| ChatGPT | 0.21 | 0.30 | 0.26 |
231+
| Neutral | 0.43 | 0.38 | 0.17 |
232+
| None | 0.07 | 0.07 | 0.08 |
233+
| total | 1.00 | 1.00 | 1.00 |
234+
235+
## Summary
236+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
237+
|----------|--------------|----------------------|---------------|
238+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
239+
| GPT4o | 0.33 | 0.33 | 0.28 |
240+
| Neutral | 0.17 | 0.17 | 0.12 |
241+
| None | 0.08 | 0.08 | 0.10 |
242+
| Total | 1.00 | 1.00 | 1.00 |
243+
244+
...
245+
```

‎docs/_posts/akrztrk/2024-10-04-jsl_medm_q4_v1_en.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,3 +216,30 @@ KISUNLA is an amyloid beta-directed antibody indicated for the treatment of Alzh
216216
|Edition:|Official|
217217
|Language:|en|
218218
|Size:|4.8 GB|
219+
220+
221+
## Benchmarking
222+
223+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
224+
225+
```bash
226+
## Overall
227+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
228+
|----------|--------------|----------------------|---------------|
229+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
230+
| ChatGPT | 0.21 | 0.30 | 0.26 |
231+
| Neutral | 0.43 | 0.38 | 0.17 |
232+
| None | 0.07 | 0.07 | 0.08 |
233+
| total | 1.00 | 1.00 | 1.00 |
234+
235+
## Summary
236+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
237+
|----------|--------------|----------------------|---------------|
238+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
239+
| GPT4o | 0.33 | 0.33 | 0.28 |
240+
| Neutral | 0.17 | 0.17 | 0.12 |
241+
| None | 0.08 | 0.08 | 0.10 |
242+
| Total | 1.00 | 1.00 | 1.00 |
243+
244+
...
245+
```

‎docs/_posts/akrztrk/2024-10-06-jsl_medm_q4_v2_en.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,3 +132,30 @@ The patient is presenting with symptoms of urinary tract infection (UTI), which
132132
|Edition:|Official|
133133
|Language:|en|
134134
|Size:|4.8 GB|
135+
136+
137+
## Benchmarking
138+
139+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
140+
141+
```bash
142+
## Overall
143+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
144+
|----------|--------------|----------------------|---------------|
145+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
146+
| ChatGPT | 0.21 | 0.30 | 0.26 |
147+
| Neutral | 0.43 | 0.38 | 0.17 |
148+
| None | 0.07 | 0.07 | 0.08 |
149+
| total | 1.00 | 1.00 | 1.00 |
150+
151+
## Summary
152+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
153+
|----------|--------------|----------------------|---------------|
154+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
155+
| GPT4o | 0.33 | 0.33 | 0.28 |
156+
| Neutral | 0.17 | 0.17 | 0.12 |
157+
| None | 0.08 | 0.08 | 0.10 |
158+
| Total | 1.00 | 1.00 | 1.00 |
159+
160+
...
161+
```

‎docs/_posts/akrztrk/2024-10-06-jsl_medm_q4_v3_en.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,3 +137,30 @@ Here's the rationale:
137137
|Edition:|Official|
138138
|Language:|en|
139139
|Size:|8.8 GB|
140+
141+
142+
## Benchmarking
143+
144+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
145+
146+
```bash
147+
## Overall
148+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
149+
|----------|--------------|----------------------|---------------|
150+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
151+
| ChatGPT | 0.21 | 0.30 | 0.26 |
152+
| Neutral | 0.43 | 0.38 | 0.17 |
153+
| None | 0.07 | 0.07 | 0.08 |
154+
| total | 1.00 | 1.00 | 1.00 |
155+
156+
## Summary
157+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
158+
|----------|--------------|----------------------|---------------|
159+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
160+
| GPT4o | 0.33 | 0.33 | 0.28 |
161+
| Neutral | 0.17 | 0.17 | 0.12 |
162+
| None | 0.08 | 0.08 | 0.10 |
163+
| Total | 1.00 | 1.00 | 1.00 |
164+
165+
...
166+
```

‎docs/_posts/akrztrk/2024-10-06-jsl_medm_q8_v2_en.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,3 +132,30 @@ The patient is presenting with symptoms of urinary tract infection (UTI), which
132132
|Edition:|Official|
133133
|Language:|en|
134134
|Size:|8.2 GB|
135+
136+
137+
## Benchmarking
138+
139+
We have generated a total of 400 questions, 100 from each category. These questions were labeled and reviewed by 3 physician annotators. `%` indicates the preference rate
140+
141+
```bash
142+
## Overall
143+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
144+
|----------|--------------|----------------------|---------------|
145+
| JSL-MedM | 0.29 | 0.25 | 0.50 |
146+
| ChatGPT | 0.21 | 0.30 | 0.26 |
147+
| Neutral | 0.43 | 0.38 | 0.17 |
148+
| None | 0.07 | 0.07 | 0.08 |
149+
| total | 1.00 | 1.00 | 1.00 |
150+
151+
## Summary
152+
| Model | Factuality % | Clinical Relevancy % | Conciseness % |
153+
|----------|--------------|----------------------|---------------|
154+
| JSL-MedM | 0.42 | 0.42 | 0.50 |
155+
| GPT4o | 0.33 | 0.33 | 0.28 |
156+
| Neutral | 0.17 | 0.17 | 0.12 |
157+
| None | 0.08 | 0.08 | 0.10 |
158+
| Total | 1.00 | 1.00 | 1.00 |
159+
160+
...
161+
```

0 commit comments

Comments
 (0)