Try--dropout-embedding-with-gpt-tokenizer-best-run

[david-thrower]

Kind of issue: kind/enhancement; R and D

Hybridize Alex's proof of concept for Droupout(0.75) -> Embedding(15 dimensions) with the parameters in f2fdcf708269fc9c9fd29ababd7b93cdc6f8f834

Suggested Labels (If you don't know, that's ok): kind/enhancement; R and D

[david-thrower]

@sashakolpakov

It looks like this approach Embedding -> dropout is a winner. I see that with the current params (that may no longer be optimal) I see test set binary-accuracy >= 0.95 with a dropout rate of ~ 0.6. I am testing on another branch, hybridizing this with the randomization of motivations and will see which of the 2 should be merged in.