Now IvtffText reads file header correctly.

Author: mzattera

README.md

@@ -8,7 +8,7 @@ The folder `eclipse` contains an eclipse workspace. The (Maven) project `io.gith
 The library content is described below. The (Maven) project `io.github.mattera.v4j-apps` contains classes I created to experiment with the 
 Voynich manuscript; here you can find examples about how to use the library.
 
-**_Note:_** _Plase check the [project pages](https://mzattera.github.io/v4j/) for some terminology that is relevant here.
+**_Note:_** Plase check the [project pages](https://mzattera.github.io/v4j/) for some terminology that is relevant here.
 
 ## Packages and Library Overview - Project `io.github.mattera.v4j`
 
@@ -226,7 +226,7 @@ Please take a look what is in here before implementing anythign from scratch.
 
 ### Testing
 
-Project `io.github.mattera.v4j-apps` contain JUnit tests for the v4j library and (some) of the "applications" in `v4j-apps`.
+Project `io.github.mattera.v4j-apps` contains JUnit tests for the v4j library and (some) of the "applications" in `v4j-apps`.
 
 
 

docs/003/index.md

@@ -234,7 +234,7 @@ Currier's languages reflect language differences in the underlying "clear" text.
 - As the above grouping reflects a similar distribution of terms in the text, no matter what was the cause,
 these differences should be kept in mind when performing statistical analysis of the text or when trying it decipherment.
 
-  For this reason v4j library provides means to classify pages accordingly to above considerations, the resulting clusters are shown below
+  For this reason, v4j library provides means to classify pages accordingly to above considerations, the resulting clusters are shown below
   (also see [`PageHeader`](https://github.com/mzattera/v4j/blob/v.3.0.0/eclipse/io.github.mattera.v4j/src/main/java/io/github/mattera/v4j/text/ivtff/PageHeader.java) class).
 
   ![Cluster size in words](images/Clusters.PNG)

docs/004/index.md

@@ -1,6 +1,6 @@
 ## Note 004 - On Terms
 
-_Last updated Sep. 9th, 2021._
+_Last updated Sep. 18th, 2021._
 
 _This note refers to [release v.4.0.0](https://github.com/mzattera/v4j/tree/v.4.0.0) of v4j;
 **links to classes and files refer to this release** and files might have been changed, deleted or moved in the current master branch.
@@ -26,22 +26,23 @@ The below table summarizes the results, showing, the relative frequency of terms
 
 ![Most used terms](images/Terms.PNG)
 
-As expected from cluster analysis, beside terms that appear frequently in all clusters (such as **chey**, **daiin**, **dar**, and **dy**),
+As expected from cluster analysis, beside terms that appear frequently in all clusters (such as 'chey', 'daiin', 'dar', 'dy', and 'or'),
 there are terms characteristic of a single cluster; the table below shows them.
 
 ![Most used terms](images/Unique.PNG)
 
 It might be interesting to note that:
 
-- Most common terms in Herbal A pages (HA cluster) start with ch- or sh-; the latter prefix appearing only here,
+- Most common terms in Herbal A pages (HA cluster) start with 'ch-' or 'sh-'; the latter prefix appearing only here,
 
-- Pharmaceutical (PA cluster) common terms end in -ol, which is rare for other clusters. In addition, they seem to prefer the ok- or qok- prefix.
+- Pharmaceutical (PA cluster) common terms end in '-ol', which is rare for other clusters. In addition, they seem to prefer the 'ok-' or 'qok-' prefix.
 
-- Herbal B pages (HB cluster) prefer terms starting with qo(k)-.
+- Herbal B pages (HB cluster) prefer terms starting with 'qo-' and 'qok-'.
 
-- Zodiac (ZZ cluster) common terms mostly start with ot-, this is uncommon for other clusters. Moreover, this cluster
+- Zodiac (ZZ cluster) common terms mostly start with 'ot-', this is uncommon for other clusters. Moreover, this cluster
 features single characters as common terms.
 
+
 ---
 
 [**<< Home**](..)

docs/005/index.md

@@ -1,6 +1,6 @@
 # Note 005 - Slots and a New Alphabet
 
-_Last updated Sep. 12th, 2021._
+_Last updated Sep. 18th, 2021._
 
 _This note refers to [release v.5.0.0](https://github.com/mzattera/v4j/tree/v.5.0.0) of v4j;
 **links to classes and files refer to this release**; files might have been changed, deleted or moved in the current master branch.
@@ -21,15 +21,17 @@ _Please refer to the [home page](..) for a set of definitions that might be rele
 I show how the structure of Voynich words can be easily described by assuming each term is composed by "slots" that can be filled
 accordingly to simple rules, which are described below.
 
-This in turn sheds some lights on the definition of what constitute a Voynich character (the Voynich alphabet).
+This in turn sheds some lights on the definition of what might constitute a Voynich character (the Voynich alphabet).
 
 Given the nature of this topic, it is impossible to define rules that apply to 100% of cases; after all, syntactical and grammatical exceptions
 exists in any modern text as well. However, I will try to focus on claims that apply to the vast majority of cases. 
 
 
 ## Previous Works
 
-**TODO** _add the core/mantel/crust and the state machine works_.
+Either here or at the end as "Comparison with other works".
+
+**TODO** https://briancham1994.com/2014/12/17/curve-line-system/.
 
 - This approach is easier to explain and has more implications.
 
@@ -183,8 +185,8 @@ Again, this seems a strong indication that EVA 'h' does not correspond to a Voyn
 
 #### 'e' and 'i'
 
-The characters 'e' and 'i' only appear in slots 6 and 9 respectively, in a sequence of 1, 2 or 3. Currier has assumed these sequences of same characters are 
-single Voynich characters. Based upon how they appear in the slots, I think this is a reasonable assumption.
+The characters 'e' and 'i' only appear in slots 6 and 9 respectively, in a sequence of 1, 2 or 3. Currier has assumed each sequences of same characters is 
+a single Voynich character. Based upon how they appear in the slots, I think this is a reasonable assumption.
 
 
 ## The Slot Alphabet 

docs/index.md

@@ -13,13 +13,13 @@ in order to provide some (hopefully) more solid basis for discussions and deciph
 On this site, I will try to be consistent with following terminology.
 
 - A **transliteration** is a symbol-by-symbol conversion of one script into another. Transliteration is needed to represent the content of the Voynich in a 
-format that can be printed or stored in computer files. I might sometime use the less correct term **transliteration** as a synonym and refer 
+format that can be printed or stored in computer files. I might sometimes use the less correct term **transliteration** as a synonym and refer 
 to an author of a transliteration as a **transcriber**.
 
-- I refer to the list of symbols used in the target script as the **transliteration alphabet** or simply as the **alphabet**.
+  - I refer to the list of symbols used in the target script as the **transliteration alphabet** or simply as the **alphabet**.
 Each symbol in the alphabet is referred as a **transliteration character** or simply **character**.
 
-- The term **glyph** refers to a symbol in the Voynich that appears to constitute a single unit of text. In principle, a glyph could represent one or more
+- The term **glyph** refers to a symbol in the Voynich that appears to constitute a basic unit of text. In principle, a glyph could represent one or more
  **Voynich characters** that constitute the **Voynich alphabet**.
 
   The question of which glyphs are actual single Voynich characters is still very open and it is at the basis of the different transliteration alphabets being created.

eclipse/io.github.mattera.v4j/src/main/java/io/github/mattera/v4j/text/alphabet/Alphabet.java

@@ -7,7 +7,9 @@
 package io.github.mattera.v4j.text.alphabet;
 
 import java.util.ArrayList;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 
 /**
  * Defines an alphabet of symbols used in a text.
@@ -25,8 +27,26 @@ public abstract class Alphabet {
 	/** The Slot alphabet (see https://mzattera.github.io/v4j/005/) */
 	public final static SlotAlphabet SLOT = new SlotAlphabet();
 
+	// All available alphabets, by code
+	private static final Map<String, Alphabet> ALPHABETS = new HashMap<>();
+	static {
+		ALPHABETS.put(EVA.getCodeString(), EVA);
+		ALPHABETS.put(UTF_16.getCodeString(), UTF_16);
+		ALPHABETS.put(SLOT.getCodeString(), SLOT);
+	}
+
+	/**
+	 * @param codeString a string code for this alphabet, same used in IVTFF file
+	 *                   header for the alphabet.
+	 * @return The Alphabet with given code, or null if it cannot be found.
+	 */
+	public static Alphabet getAlphabet(String codeString) {
+		return ALPHABETS.getOrDefault(codeString, null);
+	}
+
 	/**
-	 * @return a string code for this alphabet, same as that used in the IVTFF file.
+	 * @return a string code for this alphabet, same used in IVTFF file header for
+	 *         the alphabet.
 	 */
 	public abstract String getCodeString();
 

eclipse/io.github.mattera.v4j/src/main/java/io/github/mattera/v4j/text/ivtff/IvtffText.java

@@ -42,23 +42,18 @@ public String getId() {
 		return id;
 	}
 
-	// Transcript version: A.B.x
-	// TODO this is not stored
 	private String version;
 
 	/**
-	 * 
 	 * @return Transcript version: A.B.x
 	 */
 	public String getVersion() {
 		return version;
 	}
 
-	// Major version: A.B
 	private String majorVersion;
 
 	/**
-	 * 
 	 * @return Transcript major version: A.B
 	 */
 	public String getMajorVersion() {
@@ -167,7 +162,7 @@ public static IvtffText fromPages(IvtffText doc, Collection<IvtffPage> pages) {
 	 * file. AND IS UNSUPPORTED
 	 */
 	public final static Pattern FILE_HEADER_PATTERN = Pattern.compile("#=IVTFF (.{4}) ([0-9]+\\.[0-9]+)(\\.[0-9]+)?");
-	
+
 	/**
 	 * The notation used to identify a page in the Voynich MS is the character f
 	 * (for folio) followed by the folio number, followed by r (for recto - the
@@ -178,10 +173,9 @@ public static IvtffText fromPages(IvtffText doc, Collection<IvtffPage> pages) {
 	public final static Pattern PAGE_HEADER_PATTERN = Pattern.compile("<f[0-9]{1,3}[rv][0-9]?>|<fRos>");
 
 	/**
-	 * Constructor from Reader.
-	 * Assumes EVA alphabet.
+	 * Constructor from Reader. Assumes EVA alphabet.
 	 * 
-	 * @param in       A Reader from which the document content is read.
+	 * @param in A Reader from which the document content is read.
 	 */
 	private IvtffText(BufferedReader in) throws IOException, ParseException {
 
@@ -193,16 +187,17 @@ private IvtffText(BufferedReader in) throws IOException, ParseException {
 			if (row == null)
 				throw new ParseException("The input file is empty.");
 
-			// TODO add proper descriptor that includes header information.
-			// Also check and store IVTFF version of the file.
 			Matcher m = FILE_HEADER_PATTERN.matcher(row);
 			if (!m.matches())
 				throw new ParseException("Invalid file header: ", row);
-			if (m.group(1).equals("Eva-")) {
-				this.alphabet = Alphabet.EVA;
-			} else {
-				throw new ParseException("Unsupported alphabeth: " + m.group(1));
-			}
+
+			this.alphabet = Alphabet.getAlphabet(m.group(1));
+			if (this.alphabet == null)
+				new ParseException("Unsupported alphabeth: " + m.group(1));
+			this.majorVersion = m.group(2);
+			if (!this.majorVersion.equals("1.5"))
+				new ParseException("Unsupported IVTFF format version: " + this.majorVersion);				
+			this.version = m.group(2)+m.group(3);
 
 			IvtffPage currentPage = null;
 
@@ -304,6 +299,7 @@ public void write(File fOut) throws IOException {
 			out.newLine();
 			out.write("# Latest modified on: " + f.format(new Date()));
 			out.newLine();
+			out.write("#");
 			out.newLine();
 
 			for (IvtffPage page : elements) {

eclipse/io.github.mattera.v4j/src/main/resources/Transcriptions/Interlinear_ivtff_1.5.txt

@@ -1,7 +1,10 @@
-#=IVTFF Eva- 1.5.2018092200
+#=IVTFF Eva- 1.5
+# Created automatically by io.github.mattera.v4j.applications.text.BuildConcordanceVersion on: 20210919.1526
+#
 # 
-# Minor changes made by Maxi to align trascription of a few lines, so they can be merged.
-# Last edited on 2018-09-22 17:13 by Maxi
+# This is based on Landini-Stolfi Interlinear file from http://www.voynich.nu/data/beta/LSI_ivtff_0d.txt
+# Minor changes made by Massimiliano Zattera to align transcription of a few lines, so they can be merged automatically.
+# Last edited on 2021-09-21 14:50 by Massimiliano Zattera.
 # 
 # # <f0.A> {}
 # Last edited on 1998-12-05 11:22:03 by stolfi

eclipse/io.github.mzattera.v4j-apps/src/main/java/io/github/mattera/v4j/applications/text/BuildConcordanceVersion.java

@@ -8,7 +8,10 @@
 import java.io.IOException;
 import java.io.InputStreamReader;
 import java.io.OutputStreamWriter;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
 import java.util.ArrayList;
+import java.util.Date;
 import java.util.List;
 import java.util.regex.Matcher;
 
@@ -90,29 +93,31 @@ static void doWork(BufferedReader in, BufferedWriter out) throws IOException, Pa
 			String fLine = null;
 			IvtffLine line = null;
 			int lnum = 0;
-			Alphabet a = null;
 
 			// Parse file header
 			fLine = in.readLine();
 			if (fLine == null)
 				throw new ParseException("Empty input.");
 			++lnum;
 
-			// TODO add proper descriptor that includes header information.
-			// TODO add support for other alphabets
 			Matcher m = IvtffText.FILE_HEADER_PATTERN.matcher(fLine);
 			if (!m.matches())
 				throw new ParseException("Invalid file header: ", fLine);
-			if (m.group(1).equals("Eva-")) {
-				a = Alphabet.EVA;
-			} else {
-				throw new ParseException("Unsupported alphabeth: " + m.group(1));
-			}
+
+			Alphabet a = Alphabet.getAlphabet(m.group(1));
+			if (a == null)
+				new ParseException("Unsupported alphabeth: " + m.group(1));
 
 			// Write header
 			out.write(fLine);
 			out.newLine();
 
+			DateFormat f = new SimpleDateFormat("yyyyMMdd.HHmm");
+			out.write("# Created automatically by " + BuildConcordanceVersion.class.getName() + " on: " + f.format(new Date()));
+			out.newLine();
+			out.write("#");
+			out.newLine();			
+			
 			while ((fLine = in.readLine()) != null) {
 				++lnum;
 				line = null;

eclipse/io.github.mzattera.v4j-apps/src/main/resources/Transcriptions/LSI_ivtff_0d_fixed.txt

@@ -1,7 +1,8 @@
-#=IVTFF Eva- 1.5.2018092200
+#=IVTFF Eva- 1.5
 # 
-# Minor changes made by Maxi to align trascription of a few lines, so they can be merged.
-# Last edited on 2018-09-22 17:13 by Maxi
+# This is based on Landini-Stolfi Interlinear file from http://www.voynich.nu/data/beta/LSI_ivtff_0d.txt
+# Minor changes made by Massimiliano Zattera to align transcription of a few lines, so they can be merged automatically.
+# Last edited on 2021-09-21 14:50 by Massimiliano Zattera.
 # 
 # # <f0.A> {}
 # Last edited on 1998-12-05 11:22:03 by stolfi