Skip to content

⚡ Simplify header-fld-name parser (backward compatible) #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 3, 2023
Merged

⚡ Simplify header-fld-name parser (backward compatible) #217

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions benchmarks/parser.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,16 @@ benchmark:
response = load_response("../test/net/imap/fixtures/response_parser/continuation_requests.yml",
"test_continuation_request_without_response_text")
script: parser.parse(response)
- name: fetch_msg_att_HEADER.FIELDS
prelude: |2
response = load_response("../test/net/imap/fixtures/response_parser/fetch_responses.yml",
"test_fetch_msg_att_HEADER.FIELDS")
script: parser.parse(response)
- name: fetch_msg_att_HEADER.FIELDS.NOT
prelude: |2
response = load_response("../test/net/imap/fixtures/response_parser/fetch_responses.yml",
"test_fetch_msg_att_HEADER.FIELDS.NOT")
script: parser.parse(response)
- name: fetch_msg_att_flags_and_uid
prelude: |2
response = load_response("../test/net/imap/fixtures/response_parser/fetch_responses.yml",
Expand Down Expand Up @@ -171,6 +181,11 @@ benchmark:
response = load_response("../test/net/imap/fixtures/response_parser/quirky_behaviors.yml",
"test_invalid_noop_response_with_numeric_prefix")
script: parser.parse(response)
- name: invalid_noop_response_with_numeric_prefix_and_unparseable_data
prelude: |2
response = load_response("../test/net/imap/fixtures/response_parser/quirky_behaviors.yml",
"test_invalid_noop_response_with_numeric_prefix_and_unparseable_data")
script: parser.parse(response)
- name: invalid_noop_response_with_unparseable_data
prelude: |2
response = load_response("../test/net/imap/fixtures/response_parser/quirky_behaviors.yml",
Expand Down
53 changes: 19 additions & 34 deletions lib/net/imap/response_parser.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1101,47 +1101,32 @@ def header_list
end

# RFC3501 & RFC9051:
# header-fld-name = astring
# header-fld-name = astring
#
# NOTE: Previously, Net::IMAP recreated the raw original source string.
# Now, it grabs the raw encoded value using @str and @pos. A future
# version may simply return the decoded astring value. Although that is
# technically incompatible, it should almost never make a difference: all
# standard header field names are valid atoms:
#
# https://www.iana.org/assignments/message-headers/message-headers.xhtml
#
# Although RFC3501 allows any astring, RFC5322-valid header names are one
# or more of the printable US-ASCII characters, except SP and colon. So
# empty string isn't valid, and literals aren't needed and should not be
# used. This syntax is unchanged by [I18N-HDRS] (RFC6532).
# used. This is explicitly unchanged by [I18N-HDRS] (RFC6532).
#
# RFC5233:
# optional-field = field-name ":" unstructured CRLF
# field-name = 1*ftext
# ftext = %d33-57 / ; Printable US-ASCII
# %d59-126 ; characters not including
# ; ":".
#
# Atom and quoted should be sufficient.
#
# TODO: Use original source string, rather than decode and re-encode.
# TODO: or at least, DRY up this code with the send_command formatting.
# optional-field = field-name ":" unstructured CRLF
# field-name = 1*ftext
# ftext = %d33-57 / ; Printable US-ASCII
# %d59-126 ; characters not including
# ; ":".
def header_fld_name
case (str = astring)
when ""
warn '%s header-fld-name is an invalid RFC5322 field-name: ""' %
[self.class]
return '""'
when /[\x80-\xff\r\n]/n
warn "%s header-fld-name %p has invalid RFC5322 field-name char: %p" %
[self.class, str, $&]
# literal
return "{" + str.bytesize.to_s + "}" + CRLF + str
when /[^\x21-\x39\x3b-\xfe]/n
warn "%s header-fld-name %p has invalid RFC5322 field-name char: %p" %
[self.class, str, $&]
# invalid quoted string
return '"' + str.gsub(/["\\]/n, "\\\\\\&") + '"'
when /[(){ \x00-\x1f\x7f%*"\\]/n
# quoted string
return '"' + str.gsub(/["\\]/n, "\\\\\\&") + '"'
else
# atom
return str
end
assert_no_lookahead
start = @pos
astring
@str[start...@pos - 1]
end

# mailbox-data = "FLAGS" SP flag-list / "LIST" SP mailbox-list /
Expand Down
48 changes: 48 additions & 0 deletions test/net/imap/fixtures/response_parser/fetch_responses.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,54 @@
UID: 5
raw_data: "* 1 FETCH (FLAGS (\\Seen $MDNSent \\Flagged Custom) UID 5)\r\n"

test_fetch_msg_att_HEADER.FIELDS:
:response: &test_fetch_msg_att_HEADER_FIELDS
"* 20367 FETCH (BODY[HEADER.FIELDS (List-ID List-Unsubscribe
List-Unsubscribe-Post List-Owner List-Archive)] {291}\r\nList-Unsubscribe:
manage-example-lists
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n\r\n)\r\n"
:expected: !ruby/struct:Net::IMAP::UntaggedResponse
name: FETCH
data: !ruby/struct:Net::IMAP::FetchData
seqno: 20367
attr:
BODY[HEADER.FIELDS (List-ID List-Unsubscribe List-Unsubscribe-Post List-Owner List-Archive)]: "List-Unsubscribe:
manage-example-lists xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n\r\n"
raw_data: *test_fetch_msg_att_HEADER_FIELDS

test_fetch_msg_att_HEADER.FIELDS.NOT:
:response: &test_fetch_msg_att_HEADER_FIELDS_NOT
"* 20368 FETCH (BODY[HEADER.FIELDS.NOT (Received DKIM-Signature List-Unsubscribe
ARC-Seal ARC-Authentication-Results Authentication-Results ARC-Message-Signature
Received-SPF X-Received Mime-Version Content-Type Content-Transfer-Encoding
X-AUTO-Response-Suppress X-Google-Smtp-Source)]
{307}\r\nDelivered-To: testy.mctester@mail.test\r\nReturn-Path:
<noreply@example.test>\r\nDate: Thu, 02 Nov 2023 10:10:17 -0700\r\nFrom:
Example <support@example.test>\r\nMessage-ID:
<xxxxxxxxxxxxxxxxxxxx@yyyyyyyyy-zzzzzzzz-example.mail>\r\nSubject:
[Example] You've hit 75% of your spending limit for the Tester\r\n
account\r\n\r\n)\r\n"
:expected: !ruby/struct:Net::IMAP::UntaggedResponse
name: FETCH
data: !ruby/struct:Net::IMAP::FetchData
seqno: 20368
attr:
? BODY[HEADER.FIELDS.NOT (Received DKIM-Signature List-Unsubscribe ARC-Seal
ARC-Authentication-Results Authentication-Results ARC-Message-Signature
Received-SPF X-Received Mime-Version Content-Type Content-Transfer-Encoding
X-AUTO-Response-Suppress X-Google-Smtp-Source)]
: "Delivered-To: testy.mctester@mail.test\r\nReturn-Path: <noreply@example.test>\r\nDate:
Thu, 02 Nov 2023 10:10:17 -0700\r\nFrom: Example <support@example.test>\r\nMessage-ID:
<xxxxxxxxxxxxxxxxxxxx@yyyyyyyyy-zzzzzzzz-example.mail>\r\nSubject: [Example]
You've hit 75% of your spending limit for the Tester\r\n account\r\n\r\n"
raw_data: *test_fetch_msg_att_HEADER_FIELDS_NOT

test_invalid_fetch_msg_att_extra_space:
:response: "* 1 FETCH (UID 92285 )\r\n"
:expected: !ruby/struct:Net::IMAP::UntaggedResponse
Expand Down