Implementing deficit-based priority-queueing scheduler. #4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added a new directory in services/transfers introducing a class called DinicsMaximumFlow that provides the interface for creating a network graph as well as computing maximum flow.
Added new configuration for selecting the transfer service's allocator as "GREEDY" or "MAXFLOW", with default being "GREEDY". Also added configuration for selecting the transfer service's scheduling algorithm as "RANDOMIZED", "DEFICIT", or "SELECTION", with the default value being "RANDOMIZED". User's are able to override these default values via the command line.
…ll as a set of queues and returns the capacity on each queue.
Added a new class in services/transfers introducing a class called MaxiumFlow that provides the interface for creating a network graph as well as computing maximum flow. Also introduced new unit tests to test functionality.
Added new configuration for selecting the transfer service's allocator as "GREEDY" or "MAXFLOW", with default being "GREEDY". Also added configuration for selecting the transfer service's scheduling algorithm as "RANDOMIZED", "DEFICIT", or "SELECTION", with the default value being "RANDOMIZED". User's are able to override these default values via the command line. Finally, also introduced new unit tests to test reading config as well as accepting from the command line.
…ll as a set of queues and returns the capacity on each queue.
…nks their full capacity, and then removes the links from the flow network.
….com/jackiedong6/fts3 into Jackie/Add-Maximum-Flow-To-Allocator
…m will now check for starved links, allocate the full capacity for starved links, then remove them from the flow graph before running the max flow algorithm. This code replaces a previous commit by moving the code from the TransfersService.cpp to Allocator.cpp
…duler Conflicts: src/db/mysql/Config.cpp src/db/mysql/MySqlAPI.cpp src/server/services/transfers/Scheduler.cpp src/server/services/transfers/Scheduler.h src/server/services/transfers/TransfersService.cpp src/server/services/transfers/TransfersService.h
mpatrascoiu
pushed a commit
that referenced
this pull request
Apr 11, 2025
Refactor how FTS count running processes, use standard library
functions.
Fix the use-after scop.
==8903==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f5f02001140 at pc 0x7f5f07227f64 bp 0x7ffcd2454080 sp 0x7ffcd2454078
READ of size 1 at 0x7f5f02001140 thread T0
#0 0x7f5f07227f63 in fts3::common::countProcessesWithName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /work/fts3/src/common/DaemonTools.cpp:87
#1 0x539994 in main /work/fts3/src/qos-daemon/main.cpp:175
#2 0x7f5f06c10247 in __libc_start_call_main (/lib64/libc.so.6+0x3247) (BuildId: 515c33a35f41020661fea8ac4eb995e26ccd6b00)
#3 0x7f5f06c1030a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x330a) (BuildId: 515c33a35f41020661fea8ac4eb995e26ccd6b00)
#4 0x401d74 in _start (/usr/sbin/fts_qos+0x401d74) (BuildId: 944a128555f94dba94c0f20dcf8ad96ba8e2ea8d)
Address 0x7f5f02001140 is located in stack of thread T0 at offset 320 in frame
#0 0x7f5f07227b96 in fts3::common::countProcessesWithName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /work/fts3/src/common/DaemonTools.cpp:78
This frame has 12 object(s):
[32, 33) '<unknown>'
[48, 56) 'end_itr' (line 82)
[80, 88) 'itr' (line 83)
[112, 120) 'endptr' (line 84)
[144, 152) 'cb'
[176, 184) 'cb'
[208, 216) '<unknown>'
[240, 272) '<unknown>'
[304, 336) '<unknown>' <== Memory access at offset 320 is inside this variable
[368, 400) 'cmdline' (line 90)
[432, 952) 'cmdlineStream' (line 94)
[1088, 1344) 'cmdName' (line 95)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-use-after-scope /work/fts3/src/common/DaemonTools.cpp:87 in fts3::common::countProcessesWithName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
Shadow bytes around the buggy address:
0x7f5f02000e80: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
0x7f5f02000f00: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
0x7f5f02000f80: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
0x7f5f02001000: f1 f1 f1 f1 01 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2
0x7f5f02001080: f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 f8 f8
=>0x7f5f02001100: f8 f8 f2 f2 f2 f2 f8 f8[f8]f8 f2 f2 f2 f2 00 00
0x7f5f02001180: 00 00 f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00
0x7f5f02001200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7f5f02001280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7f5f02001300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7f5f02001380: 00 00 00 00 00 00 00 f2 f2 f2 f2 f2 f2 f2 f2 f2
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==8903==ABORTING
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.