You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/en/jsl/nlu_for_healthcare.md
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -27,7 +27,7 @@ and the accompanying video below for an introduction to every healthcare domain.
27
27
**Named entities** are sub-strings in a text that can be classified into catogires of a domain. For example, in the String
28
28
`"Tesla is a great stock to invest in "` , the sub-string `"Tesla"` is a named entity, it can be classified with the label `company` by an ML algorithm.
29
29
**Named entities** can easily be extracted by the various pre-trained Deep Learning based NER algorithms provided by NLU.
30
-
NER models can be trained for many different domains and aquire expert domain knowledge in each of them. JSL provides a wide array of experts for various Medical, Helathcare and Clinical domains
30
+
NER models can be trained for many different domains and aquire expert domain knowledge in each of them. JSL provides a wide array of experts for various Medical, Healthcare and Clinical domains
31
31
32
32
This algorithm is provided by **Spark NLP for Healthcare's**[MedicalNerModel](https://nlp.johnsnowlabs.com/docs/en/licensed_annotators)
33
33
@@ -70,7 +70,7 @@ Named Entities extracted by an NER model can be further classified into sub-clas
70
70
All sentences have the entity `headache` which is of class `disease`.
71
71
But there is a semantic difference on what the actual status of the disease mentioned in text is. In the first and third sentence, `Billy has no headache`, but in the second sentence `Billy actually has a sentence`.
72
72
The `Entity Assertion` Algorithms provided by JSL solve this problem. The `disease` entity can be classified into `ABSENT` for the first case and into `PRESENT` for the second case. The third case can be classified into `PRESENT IN FAMILY`.
73
-
This has immense implications for various data analytical approaches in the helathcare domain.
73
+
This has immense implications for various data analytical approaches in the healthcare domain.
74
74
75
75
I.e. imagine you want you want to make a study about hearth attacks and survival rate of potential procedures. You can process all your digital patient notes with an Medical NER model and filter for documents that have the `Hearth Attack` entity.
76
76
But your collected data will have wrong data entries because of the above mentioned Entity status problem. You cannot deduct that a document is talking about a patient having a hearth attack, unless you **assert** that the problem is actually there which is what the Resolutions algorithms do for you.
Copy file name to clipboardExpand all lines: docs/en/jsl/release_notes.md
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -3359,7 +3359,7 @@ for the first time by NLU, including ancient and exotic languages like `Ancient
3359
3359
On the healthcare NLP side, a new `ZeroShotRelationExtractionModel` is available, which can extract relations between
3360
3360
clinical entities in an unsupervised fashion, no training required!
3361
3361
Additionally, New French and Italian Deidentification models are available for clinical and healthcare domains.
3362
-
Powerd by the fantastic [ Spark NLP for helathcare 3.5.0 release](https://nlp.johnsnowlabs.com/docs/en/spark_nlp_healthcare_versions/licensed_release_notes)
3362
+
Powerd by the fantastic [ Spark NLP for healthcare 3.5.0 release](https://nlp.johnsnowlabs.com/docs/en/spark_nlp_healthcare_versions/licensed_release_notes)
3363
3363
3364
3364
</div><div class="h3-box" markdown="1">
3365
3365
@@ -4163,7 +4163,7 @@ Integrates the incredible [Spark NLP for Healthcare](https://nlp.johnsnowlabs.co
4163
4163
4164
4164
## NLU Version 3.3.0
4165
4165
4166
-
#### 2000%+ Speedup on small data, 63 new models for 100+ Languages with 6 new supported Transformer classes including BERT, XLM-RoBERTa, alBERT, Longformer, XLnet based models, 48 NER profiling helathcare pipelines and much more in John Snow Labs NLU 3.3.0
4166
+
#### 2000%+ Speedup on small data, 63 new models for 100+ Languages with 6 new supported Transformer classes including BERT, XLM-RoBERTa, alBERT, Longformer, XLnet based models, 48 NER profiling healthcare pipelines and much more in John Snow Labs NLU 3.3.0
4167
4167
4168
4168
We are incredibly excited to announce NLU 3.3.0 has been released!
4169
4169
It comes with a up to 2000%+ speedup on small datasets, 6 new Types of Deep Learning transformer models, including
Import `johnsnowlabs` and use our one-liner `nlp.install()` to install all the dependencies, downloading the jars (yes, Spark NLP runs on top of the Java Virtual Machine!), preparing the cluster environment variables, licenses, etc!
96
96
@@ -473,7 +473,7 @@ Make sure the following prerequisites are set:
473
473
474
474
</div><div class="h3-box" markdown="1">
475
475
476
-
## Non-johnsnowlabs Helathcare NLP on Ubuntu
476
+
## Non-johnsnowlabs Healthcare NLP on Ubuntu
477
477
> These instructions use non-johnsnowlabs installation syntax. For simplified installation with `johnsnowlabs` library, check first section.
478
478
479
479
For installing John Snow Labs NLP libraries on an Ubuntu machine/VM please run the following command:
@@ -511,7 +511,7 @@ The install script downloads a couple of example notebooks that you can use to s
511
511
512
512
</div><div class="h3-box" markdown="1">
513
513
514
-
## Non-johnsnowlabs Helathcare NLP via Docker
514
+
## Non-johnsnowlabs Healthcare NLP via Docker
515
515
> These instructions use non-johnsnowlabs installation syntax. For simplified installation with `johnsnowlabs` library, check first section.
516
516
517
517
A docker image that contains all the required libraries for installing and running Enterprise Spark NLP libraries is also available. However, it does not contain the library itself, as it is licensed, and requires installation credentials.
Copy file name to clipboardExpand all lines: docs/en/spark_nlp_healthcare_versions/release_notes_5_5_2.md
-7Lines changed: 0 additions & 7 deletions
Original file line number
Diff line number
Diff line change
@@ -97,9 +97,6 @@ text = """he patient is a 42-year-old female and has diabetes mellitus with diab
97
97
| scope_average | diabetes mellitus | E11.40 | nervous system disorder due to diabetes mellitus [type 2 diabetes mellitus with diabetic neuropathy, unspecified]|
98
98
99
99
100
-
101
-
102
-
103
100
</div><divclass="h3-box"markdown="1">
104
101
105
102
#### De-identifying Sensitive Data in Relational Databases with a Few Lines of Codes
@@ -395,8 +392,6 @@ Muc5AC, human epidermal growth factor receptor-2 (HER2), and Muc6; positive for
395
392
396
393
Please check the [ZeroShot Clinical NER](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.6.ZeroShot_Clinical_NER.ipynb) Notebook for more information
397
394
398
-
399
-
400
395
</div><divclass="h3-box"markdown="1">
401
396
402
397
#### Introducing Clinical Document Analysis with One-Liner Pretrained Pipelines for Specific Clinical Tasks and Concepts
@@ -450,8 +445,6 @@ The patient, Nathaneil Bakes, is 43 years old, her Contact number: 308-657-8469
450
445
451
446
Please check the [Task Based Clinical Pretrained Pipelines](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/11.3.Task_Based_Clinical_Pretrained_Pipelines.ipynb) model for more information
452
447
453
-
454
-
455
448
</div><divclass="h3-box"markdown="1">
456
449
457
450
#### Introducing 2 New Named Entity Recognition and an Assertion Models for Gene and Phenotype Features
Copy file name to clipboardExpand all lines: docs/en/spark_ocr_versions/ocr_release_notes.md
+11-1Lines changed: 11 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -30,16 +30,19 @@ Release date: 23-01-2024
30
30
* New Dicom Pretrained Pipelines.
31
31
* New VisualDocumentProcessor.
32
32
33
+
</div><divclass="h3-box"markdown="1">
34
+
33
35
## New Obfuscation Features in ImageDrawRegions
34
36
ImageDrawRegions' main purpose is to draw solid rectangles on top of regions that typically come from NER or some other similar model. Many times, it is interesting not to only draw solid rectangles on top of detected entities, but some other values, like obfuscated values. For example, with the purpose of protecting patient's privacy, you may want to replace a name with another name, or a date with a modified date.
35
37
36
38
This feature, together with the Deidentification transformer from Spark NLP for Healthcare can be combined to create a 'rendering aware' obfuscation pipeline capable of rendering obfuscated values back to the source location where the original entities were present. The replacement must be 'rendering aware' because not every example of an entity requires the same space on the page to be rendered. So for example, 'Bob Smith' would be a good replacement for 'Rod Adams', but not for 'Alessandro Rocatagliata', simply because they render differently on the page. Let's take a look at a quick example,

39
41
40
42
to the left we see a portion of a document in which we want to apply obfuscation. We want to focus on the entities representing PHI, like patient name or phone number. On the right side, after applying the transformation, we have an image containing fake values.
41
43
You can see that the PHI in the source document has been replaced by similar entities, and these entities not only are of a similar category, but are also of a similar length.
42
44
45
+
</div><divclass="h3-box"markdown="1">
43
46
44
47
## New obfuscation features in DicomMetadataDeidentifier
45
48
Now you can customize the way metadata is de-identified in DicomMetadataDeidentifier. Customization happens through a number of different actions you can apply to each tag, for example, replacing a specific tag with a literal, or shifting a date by a number of days randomly.
We are releasing three new Dicom Pretrained Pipelines:
@@ -79,6 +83,8 @@ We are releasing three new Dicom Pretrained Pipelines:
79
83
80
84
Check notebook [here](https://github.com/JohnSnowLabs/visual-nlp-workshop/blob/master/jupyter/Dicom/SparkOcrDicomPretrainedPipelines.ipynb) for examples on how to use this.
81
85
86
+
</div><divclass="h3-box"markdown="1">
87
+
82
88
### New Visual Document Processor
83
89
New VisualDocumentProcessor that produces OCR text and tables on a single pass!,
84
90
In plugs and play into any Visual NLP pipeline, it receives images, and it returns texts and tables following the same existing schemas for these datatypes,
@@ -93,6 +99,8 @@ result = proc.transform(df)
93
99
94
100
Check this [sample notebook](https://github.com/JohnSnowLabs/visual-nlp-workshop/blob/master/jupyter/SparkOcrVisualDocumentProcessor.ipynb) for an example on how to use it.
95
101
102
+
</div><divclass="h3-box"markdown="1">
103
+
96
104
### Other Dicom Changes
97
105
* DicomDrawRegions support for setting compression quality, now you can pick different compression qualities for each of the different compression algorithms supported. The API receives an array with each element specifying the compression type like a key/value,
0 commit comments