Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
lingulist committed Sep 28, 2020
1 parent 654aca4 commit 0ac660f
Show file tree
Hide file tree
Showing 7 changed files with 70 additions and 61 deletions.
44 changes: 22 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,31 +79,31 @@ The output will first produce various tables which are for conveniency encoded i
### 4. Predictions (Automated, up to two candidates)
| doculect | predicted | perfect | proportion | score |
|:-----------|------------:|----------:|-------------:|--------:|
| Duhumbi | 13 | 6 | 0.4615 | 0.6923 |
| Jerigaon | 73 | 34 | 0.4658 | 0.7169 |
| Khispi | 31 | 13 | 0.4194 | 0.7151 |
| Khoina | 45 | 16 | 0.3556 | 0.6667 |
| Khoitam | 47 | 22 | 0.4681 | 0.7270 |
| Rahung | 48 | 24 | 0.5000 | 0.7292 |
| Rupa | 31 | 12 | 0.3871 | 0.6505 |
| Shergaon | 91 | 40 | 0.4396 | 0.7051 |
| total | 379 | 167 | 0.4406 | 0.7003 |
| doculect | words | morphemes | perfect | proportion | score |
|:-----------|--------:|------------:|----------:|-------------:|--------:|
| Duhumbi | 11 | 13 | 6 | 0.4615 | 0.6923 |
| Jerigaon | 62 | 73 | 34 | 0.4658 | 0.7169 |
| Khispi | 26 | 31 | 13 | 0.4194 | 0.7151 |
| Khoina | 38 | 45 | 16 | 0.3556 | 0.6667 |
| Khoitam | 39 | 47 | 23 | 0.4894 | 0.7447 |
| Rahung | 45 | 48 | 24 | 0.5000 | 0.7292 |
| Rupa | 25 | 31 | 13 | 0.4194 | 0.6559 |
| Shergaon | 81 | 91 | 40 | 0.4396 | 0.7051 |
| total | 327 | 379 | 169 | 0.4459 | 0.7032 |
### 5. Predictions (Automated, up to three candidates)
| doculect | predicted | perfect | proportion | score |
|:-----------|------------:|----------:|-------------:|--------:|
| Duhumbi | 13 | 6 | 0.4615 | 0.7179 |
| Jerigaon | 73 | 34 | 0.4658 | 0.7192 |
| Khispi | 31 | 13 | 0.4194 | 0.7151 |
| Khoina | 45 | 16 | 0.3556 | 0.6728 |
| Khoitam | 47 | 22 | 0.4681 | 0.7305 |
| Rahung | 48 | 24 | 0.5000 | 0.7292 |
| Rupa | 31 | 12 | 0.3871 | 0.6595 |
| Shergaon | 91 | 40 | 0.4396 | 0.7088 |
| total | 379 | 167 | 0.4406 | 0.7066 |
| doculect | words | morphemes | perfect | proportion | score |
|:-----------|--------:|------------:|----------:|-------------:|--------:|
| Duhumbi | 11 | 13 | 6 | 0.4615 | 0.7179 |
| Jerigaon | 62 | 73 | 34 | 0.4658 | 0.7192 |
| Khispi | 26 | 31 | 13 | 0.4194 | 0.7151 |
| Khoina | 38 | 45 | 16 | 0.3556 | 0.6728 |
| Khoitam | 39 | 47 | 23 | 0.4894 | 0.7482 |
| Rahung | 45 | 48 | 24 | 0.5000 | 0.7292 |
| Rupa | 25 | 31 | 13 | 0.4194 | 0.6649 |
| Shergaon | 81 | 91 | 40 | 0.4396 | 0.7088 |
| total | 327 | 379 | 169 | 0.4459 | 0.7095 |
```

Expand Down
27 changes: 18 additions & 9 deletions evaluate.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@
sum([row[2] for row in btable]),
sum([row[3] for row in btable]),
sum([row[3] for row in btable])/sum([row[2] for row in btable]),
sum([row[5] for row in btable])/len(docs),
sum([row[5] for row in btable])/len(S),
]]
print('\n### 3. Predictions (Automated, one candidate)\n')
print(tabulate(
Expand Down Expand Up @@ -264,6 +264,7 @@
S[doc][cogid] = [score, wordA, wordB, idxA, attIdx[0]]
ctable += [[
doc,
len(PREDICTED[doc]),
len(S[doc]),
len([x for x in S[doc].values() if x[0] == 1]),
len([x for x in S[doc].values() if x[0] == 1])/len(S[doc]),
Expand All @@ -274,16 +275,18 @@
'total',
sum([row[1] for row in ctable]),
sum([row[2] for row in ctable]),
sum([row[2] for row in ctable])/sum([row[1] for row in ctable]),
sum([row[4] for row in ctable])/len(docs),
sum([row[3] for row in ctable]),
sum([row[3] for row in ctable])/sum([row[2] for row in ctable]),
sum([row[5] for row in ctable])/len(S),
]]

print('\n### 4. Predictions (Automated, up to two candidates)\n')
print(tabulate(
ctable,
headers=[
'doculect',
'predicted',
'words',
'morphemes',
'perfect',
'proportion',
'score'],
Expand Down Expand Up @@ -322,26 +325,30 @@
S[doc][cogid] = [score, wordA, wordB, idxA, attIdx[0]]
dtable += [[
doc,
len(PREDICTED[doc]),
len(S[doc]),
len([x for x in S[doc].values() if x[0] == 1]),
len([x for x in S[doc].values() if x[0] == 1])/len(S[doc]),
sum([cog[0] for cog in S[doc].values()])/len(S[doc])
]]


dtable += [[
'total',
sum([row[1] for row in dtable]),
sum([row[2] for row in dtable]),
sum([row[2] for row in dtable])/sum([row[1] for row in dtable]),
sum([row[4] for row in dtable])/len(docs),
sum([row[3] for row in dtable]),
sum([row[3] for row in dtable])/sum([row[2] for row in dtable]),
sum([row[5] for row in dtable])/len(S),
]]

print('\n### 5. Predictions (Automated, up to three candidates)\n')
print(tabulate(
dtable,
headers=[
'doculect',
'predicted',
'words',
'morphemes',
'perfect',
'proportion',
'score'],
Expand Down Expand Up @@ -395,7 +402,8 @@
ctable,
headers=[
'doculect',
'predicted',
'words',
'morphemes',
'perfect',
'proportion',
'score'],
Expand All @@ -407,7 +415,8 @@
dtable,
headers=[
'doculect',
'predicted',
'words',
'morphemes',
'perfect',
'proportion',
'score'],
Expand Down
20 changes: 10 additions & 10 deletions results/computer-predictions-2.html
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
<table>
<thead>
<tr><th>doculect </th><th style="text-align: right;"> predicted</th><th style="text-align: right;"> perfect</th><th style="text-align: right;"> proportion</th><th style="text-align: right;"> score</th></tr>
<tr><th>doculect </th><th style="text-align: right;"> words</th><th style="text-align: right;"> morphemes</th><th style="text-align: right;"> perfect</th><th style="text-align: right;"> proportion</th><th style="text-align: right;"> score</th></tr>
</thead>
<tbody>
<tr><td>Duhumbi </td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 6</td><td style="text-align: right;"> 0.4615</td><td style="text-align: right;"> 0.6923</td></tr>
<tr><td>Jerigaon </td><td style="text-align: right;"> 73</td><td style="text-align: right;"> 34</td><td style="text-align: right;"> 0.4658</td><td style="text-align: right;"> 0.7169</td></tr>
<tr><td>Khispi </td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.7151</td></tr>
<tr><td>Khoina </td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 16</td><td style="text-align: right;"> 0.3556</td><td style="text-align: right;"> 0.6556</td></tr>
<tr><td>Khoitam </td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 22</td><td style="text-align: right;"> 0.4583</td><td style="text-align: right;"> 0.7257</td></tr>
<tr><td>Rahung </td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 24</td><td style="text-align: right;"> 0.5000</td><td style="text-align: right;"> 0.7292</td></tr>
<tr><td>Rupa </td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 12</td><td style="text-align: right;"> 0.3871</td><td style="text-align: right;"> 0.6505</td></tr>
<tr><td>Shergaon </td><td style="text-align: right;"> 92</td><td style="text-align: right;"> 40</td><td style="text-align: right;"> 0.4348</td><td style="text-align: right;"> 0.7029</td></tr>
<tr><td>total </td><td style="text-align: right;"> 381</td><td style="text-align: right;"> 167</td><td style="text-align: right;"> 0.4383</td><td style="text-align: right;"> 0.6985</td></tr>
<tr><td>Duhumbi </td><td style="text-align: right;"> 11</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 6</td><td style="text-align: right;"> 0.4615</td><td style="text-align: right;"> 0.6923</td></tr>
<tr><td>Jerigaon </td><td style="text-align: right;"> 62</td><td style="text-align: right;"> 73</td><td style="text-align: right;"> 34</td><td style="text-align: right;"> 0.4658</td><td style="text-align: right;"> 0.7169</td></tr>
<tr><td>Khispi </td><td style="text-align: right;"> 26</td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.7151</td></tr>
<tr><td>Khoina </td><td style="text-align: right;"> 38</td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 16</td><td style="text-align: right;"> 0.3556</td><td style="text-align: right;"> 0.6667</td></tr>
<tr><td>Khoitam </td><td style="text-align: right;"> 39</td><td style="text-align: right;"> 47</td><td style="text-align: right;"> 23</td><td style="text-align: right;"> 0.4894</td><td style="text-align: right;"> 0.7447</td></tr>
<tr><td>Rahung </td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 24</td><td style="text-align: right;"> 0.5000</td><td style="text-align: right;"> 0.7292</td></tr>
<tr><td>Rupa </td><td style="text-align: right;"> 25</td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.6559</td></tr>
<tr><td>Shergaon </td><td style="text-align: right;"> 81</td><td style="text-align: right;"> 91</td><td style="text-align: right;"> 40</td><td style="text-align: right;"> 0.4396</td><td style="text-align: right;"> 0.7051</td></tr>
<tr><td>total </td><td style="text-align: right;"> 327</td><td style="text-align: right;"> 379</td><td style="text-align: right;"> 169</td><td style="text-align: right;"> 0.4459</td><td style="text-align: right;"> 0.7032</td></tr>
</tbody>
</table>
20 changes: 10 additions & 10 deletions results/computer-predictions-3.html
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
<table>
<thead>
<tr><th>doculect </th><th style="text-align: right;"> predicted</th><th style="text-align: right;"> perfect</th><th style="text-align: right;"> proportion</th><th style="text-align: right;"> score</th></tr>
<tr><th>doculect </th><th style="text-align: right;"> words</th><th style="text-align: right;"> morphemes</th><th style="text-align: right;"> perfect</th><th style="text-align: right;"> proportion</th><th style="text-align: right;"> score</th></tr>
</thead>
<tbody>
<tr><td>Duhumbi </td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 6</td><td style="text-align: right;"> 0.4615</td><td style="text-align: right;"> 0.7179</td></tr>
<tr><td>Jerigaon </td><td style="text-align: right;"> 73</td><td style="text-align: right;"> 34</td><td style="text-align: right;"> 0.4658</td><td style="text-align: right;"> 0.7184</td></tr>
<tr><td>Khispi </td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.7151</td></tr>
<tr><td>Khoina </td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 16</td><td style="text-align: right;"> 0.3556</td><td style="text-align: right;"> 0.6617</td></tr>
<tr><td>Khoitam </td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 22</td><td style="text-align: right;"> 0.4583</td><td style="text-align: right;"> 0.7292</td></tr>
<tr><td>Rahung </td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 24</td><td style="text-align: right;"> 0.5000</td><td style="text-align: right;"> 0.7292</td></tr>
<tr><td>Rupa </td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 12</td><td style="text-align: right;"> 0.3871</td><td style="text-align: right;"> 0.6595</td></tr>
<tr><td>Shergaon </td><td style="text-align: right;"> 92</td><td style="text-align: right;"> 40</td><td style="text-align: right;"> 0.4348</td><td style="text-align: right;"> 0.7065</td></tr>
<tr><td>total </td><td style="text-align: right;"> 381</td><td style="text-align: right;"> 167</td><td style="text-align: right;"> 0.4383</td><td style="text-align: right;"> 0.7047</td></tr>
<tr><td>Duhumbi </td><td style="text-align: right;"> 11</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 6</td><td style="text-align: right;"> 0.4615</td><td style="text-align: right;"> 0.7179</td></tr>
<tr><td>Jerigaon </td><td style="text-align: right;"> 62</td><td style="text-align: right;"> 73</td><td style="text-align: right;"> 34</td><td style="text-align: right;"> 0.4658</td><td style="text-align: right;"> 0.7192</td></tr>
<tr><td>Khispi </td><td style="text-align: right;"> 26</td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.7151</td></tr>
<tr><td>Khoina </td><td style="text-align: right;"> 38</td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 16</td><td style="text-align: right;"> 0.3556</td><td style="text-align: right;"> 0.6728</td></tr>
<tr><td>Khoitam </td><td style="text-align: right;"> 39</td><td style="text-align: right;"> 47</td><td style="text-align: right;"> 23</td><td style="text-align: right;"> 0.4894</td><td style="text-align: right;"> 0.7482</td></tr>
<tr><td>Rahung </td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 24</td><td style="text-align: right;"> 0.5000</td><td style="text-align: right;"> 0.7292</td></tr>
<tr><td>Rupa </td><td style="text-align: right;"> 25</td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.6649</td></tr>
<tr><td>Shergaon </td><td style="text-align: right;"> 81</td><td style="text-align: right;"> 91</td><td style="text-align: right;"> 40</td><td style="text-align: right;"> 0.4396</td><td style="text-align: right;"> 0.7088</td></tr>
<tr><td>total </td><td style="text-align: right;"> 327</td><td style="text-align: right;"> 379</td><td style="text-align: right;"> 169</td><td style="text-align: right;"> 0.4459</td><td style="text-align: right;"> 0.7095</td></tr>
</tbody>
</table>
6 changes: 3 additions & 3 deletions results/computer-predictions.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@
</thead>
<tbody>
<tr><td>Duhumbi </td><td style="text-align: right;"> 11</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 6</td><td style="text-align: right;"> 0.4615</td><td style="text-align: right;"> 0.6923</td></tr>
<tr><td>Jerigaon </td><td style="text-align: right;"> 62</td><td style="text-align: right;"> 73</td><td style="text-align: right;"> 34</td><td style="text-align: right;"> 0.4658</td><td style="text-align: right;"> 0.6986</td></tr>
<tr><td>Jerigaon </td><td style="text-align: right;"> 62</td><td style="text-align: right;"> 73</td><td style="text-align: right;"> 34</td><td style="text-align: right;"> 0.4658</td><td style="text-align: right;"> 0.6963</td></tr>
<tr><td>Khispi </td><td style="text-align: right;"> 26</td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.7097</td></tr>
<tr><td>Khoina </td><td style="text-align: right;"> 38</td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 16</td><td style="text-align: right;"> 0.3556</td><td style="text-align: right;"> 0.6481</td></tr>
<tr><td>Khoina </td><td style="text-align: right;"> 38</td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 16</td><td style="text-align: right;"> 0.3556</td><td style="text-align: right;"> 0.6593</td></tr>
<tr><td>Khoitam </td><td style="text-align: right;"> 39</td><td style="text-align: right;"> 47</td><td style="text-align: right;"> 23</td><td style="text-align: right;"> 0.4894</td><td style="text-align: right;"> 0.7340</td></tr>
<tr><td>Rahung </td><td style="text-align: right;"> 45</td><td style="text-align: right;"> 48</td><td style="text-align: right;"> 24</td><td style="text-align: right;"> 0.5000</td><td style="text-align: right;"> 0.7153</td></tr>
<tr><td>Rupa </td><td style="text-align: right;"> 25</td><td style="text-align: right;"> 31</td><td style="text-align: right;"> 13</td><td style="text-align: right;"> 0.4194</td><td style="text-align: right;"> 0.6505</td></tr>
<tr><td>Shergaon </td><td style="text-align: right;"> 81</td><td style="text-align: right;"> 91</td><td style="text-align: right;"> 40</td><td style="text-align: right;"> 0.4396</td><td style="text-align: right;"> 0.6923</td></tr>
<tr><td>total </td><td style="text-align: right;"> 327</td><td style="text-align: right;"> 379</td><td style="text-align: right;"> 169</td><td style="text-align: right;"> 0.4459</td><td style="text-align: right;"> 0.6926</td></tr>
<tr><td>total </td><td style="text-align: right;"> 327</td><td style="text-align: right;"> 379</td><td style="text-align: right;"> 169</td><td style="text-align: right;"> 0.4459</td><td style="text-align: right;"> 0.6937</td></tr>
</tbody>
</table>
8 changes: 4 additions & 4 deletions results/errors.tsv
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ tʰ t Duhumbi 1 tʰ u ŋ t ʰu ŋ
u ʰu Duhumbi 1 tʰ u ŋ t ʰu ŋ
ə u Jerigaon 2 m ə m u
ɔ ɔː Jerigaon 1 w ɔ w ɔː
a ɛˀ Jerigaon 1 tɕʰ a - tɕʰ ɛˀ -
a ɛˀ Jerigaon 1 tɕʰ a tɕʰ ɛˀ
eˀ eː Jerigaon 2 h eˀ h eː
dʑ dz Jerigaon 1 dʑ i k dz i k
- n Jerigaon 1 s a - s a n
Expand Down Expand Up @@ -65,8 +65,8 @@ a ɔː Khoina 1 tɕ a ŋ tɕʰ ɔː -
- aː Khoina 1 b - b aː
ɔ øˀ Khoina 1 r ɔ p r øˀ -
p - Khoina 1 r ɔ p r øˀ -
g l Khoina 1 g u l uˀ
u uˀ Khoina 1 g u l uˀ
g l Khoina 1 g u - l uˀ -
u uˀ Khoina 1 g u - l uˀ -
ɛ ɐ̃ː Khoina 1 x ɛ t x ɐ̃ː -
t - Khoina 1 x ɛ t x ɐ̃ː -
g kʰ Khoina 1 g ɔ kʰ ɔˀ
Expand All @@ -79,7 +79,7 @@ y a Khoina 1 g y g a
s dʑ Khoina 1 s ɔ ŋ dʑ y ŋ
ɔ y Khoina 1 s ɔ ŋ dʑ y ŋ
ə a Khoina 1 m ə m a
eˀ ɛˀ Khoina 1 j eˀ j ɛˀ
eˀ ɛˀ Khoina 1 j eˀ - j ɛˀ -
j tɕʰ Khoina 1 j ɔ k tɕʰ ɔ k
z dz Khoina 1 z ɛ n dz ɛ n
ə aː Khoina 1 n ə n aː
Expand Down
Loading

0 comments on commit 0ac660f

Please sign in to comment.