feat: [FC-0063] Fault tolerance improving #10
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…lti-nodes-lookup-fixing fix: [FC-0063] LTI nodes lookup is fixed
- content processing errors and OLX creation errors don't cause the script failure, but just skip the resource processing - olx export statistics collection is added - file logger and console loggers are co-exist - errors logging is improved by `rich` library usage
myhailo-chernyshov-rg
force-pushed
the
myhailochernyshov/FC-0063/fault-tolerance-improving
branch
from
277f071 to
fba74c4
Compare
myhailo-chernyshov-rg
force-pushed
the
myhailochernyshov/FC-0063/fault-tolerance-improving
branch
from
fba74c4 to
4e78fcc
Compare
NiedielnitsevIvan
approved these changes
Feb 13, 2025
myhailo-chernyshov-rg
force-pushed
the
myhailochernyshov/block-types-processing-refactoring
branch
from
04d1408 to
1487472
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
There is a problem when the error occurs during conversion, the entire script fails.
This PR solves this problem: when a content processor fails, the error is handled, logged and the next processor starts processing. Also, when all processors cannot generate OLX for a CC resource, it is logged and the next CC resource processing is started.
The statistics about the resources converting success and failure count are collected and logged.
The error logging is improved by using the
richlibrary.The input converting logging into a file can be configured with the
--logs_dirargument usage. It allows to create log file with<input_file_name>.logname inside the specified directory. If the argument is not passed, the log files are not created. The usage example:Supporting information
FC-0063
Deadline
"None"