Skip to content

Commit fe5af52

Browse files
committed
test gpt2 implementation
1 parent 65e760c commit fe5af52

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

Diff for: src/cehrbert/models/hf_models/hf_cehrbert.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ def forward(
208208
value_layer = self.split_heads(value)
209209

210210
query_layer = self.split_heads(query)
211-
attn_dropout = self.attn_dropout.p if self.training else 0.0
211+
attn_dropout = self.dropout.p if self.training else 0.0
212212
# Flash Attention forward pass
213213
attn_output = flash_attention_forward(
214214
query_layer,

0 commit comments

Comments
 (0)