Skip to content

Fix record history export #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Fix record history export #4

Changes from 3 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bb669d4
fix: always escape keys in exported objects
jasper-lyons Nov 8, 2021
6948ca2
fix: escape escape character so that it's rendered
jasper-lyons Nov 10, 2021
3a4745d
Merge branch 'primeroIMS:master' into fix-record-history-export
jasper-lyons Nov 10, 2021
48bd31a
Forcing a light GC every 100 attachements imported
jasper-lyons Nov 12, 2021
2a52f24
fix: nont string keys throwing errors
jasper-lyons Nov 12, 2021
dbb12cd
fix: convert array of hashes to array of pairs
jasper-lyons Nov 13, 2021
06450c8
Fix filename issues when exporting attachements
jasper-lyons Nov 15, 2021
845ce0a
refactor: use the procedurale approach, it's cleaner
jasper-lyons Nov 15, 2021
1f2be59
fix: use much smaller batch sizes for attachment exports
jasper-lyons Nov 15, 2021
1d13c35
fix: stop overiding default batch sizes
jasper-lyons Nov 15, 2021
0dc0587
Merge branch 'reduce-attachment-memory-consumption' into fix-record-h…
jasper-lyons Nov 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions migration/v1_to_v2/data/exporters/data_exporter.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,14 +143,8 @@ def value_to_ruby_string(value, include_blank = false)
# rubocop:enable Metrics/MethodLength
# rubocop:enable Metrics/AbcSize

def valid_key?(key)
return false if key.is_a?(Integer) || key.include?('-') || key.include?(' ') || !key.match?('^[a-zA-Z]')

true
end

def key_to_ruby(key)
valid_key?(key) ? key : "'#{key}'"
"'#{ key.gsub(/'/, "\\'") }'"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you receive an integer you will get an exception. so you can keep the validation and add your escape to those that are not valid:

def key_to_ruby(key)
  return key if valid_key?(key)

  "'#{ key to_s.gsub(/'/, "\\'") }'"
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't want to rely on any valid_key? method for this, everything we're being passed should be a valid key given that it comes from an existing ruby key: value pair afaik.

I would instead just coerce everything to string before applying the .gsub.

end

def parse_date_and_location_fields(object, data_hash)
Expand Down