feat: order by and index of pipeline

docs/en_us/3.1-PipelineProtocol.md

@@ -224,6 +224,15 @@ This task property requires additional fields:
     Template matching threshold. Optional, default is 0.7.  
     If it's an array, its length should match the length of the `template` array.
 
+- `order_by`: *string*  
+    How the results are sorted. Optional, default is `Horizontal`  
+    Possible values: `Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    You can use it with the `index` field.
+
+- `index`: *int*  
+    Index to hit. Optional, default is `0`.  
+    If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.
+
 - `method`: *int*  
     Template matching algorithm, equivalent to cv::TemplateMatchModes. Optional, default is 5.  
     Only supports 1, 3, and 5, with higher values providing greater accuracy but also taking more time.  
@@ -250,6 +259,15 @@ This task property requires additional fields:
 - `count`: *int*  
     The number of required matching feature points (threshold), default is 4.
 
+- `order_by`: *string*  
+    How the results are sorted. Optional, default is `Horizontal`  
+    Possible values: `Horizontal` | `Vertical` | `Score` | `Area` | `Random`
+    You can use it with the `index` field.
+
+- `index`: *int*  
+    Index to hit. Optional, default is `0`.  
+    If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.
+
 - `green_mask`: *bool*  
     Whether to apply a green mask. Optional, default is false.  
     If set to true, you can paint the unwanted parts in the image green with RGB: (0, 255, 0), and those green parts won't be matched.
@@ -298,6 +316,15 @@ This task property requires additional fields:
 - `count`: *int*  
     The threshold for the number of matching points required. Optional, default is 1.
 
+- `order_by`: *string*  
+    How the results are sorted. Optional, default is `Horizontal`  
+    Possible values: `Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    You can use it with the `index` field.
+
+- `index`: *int*  
+    Index to hit. Optional, default is `0`.  
+    If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.
+
 - `connected`: *bool*  
     Whether to count only connected points. Optional, default is false.  
     If set to true, after applying color filtering, it will only count the maximum connected block of pixels.
@@ -318,6 +345,15 @@ This task property requires additional fields:
 - `replace`: *array<string, 2>* | *list<array<string, 2>>*  
     Some text recognition results may not be accurate, so replacements are performed. Optional.
 
+- `order_by`: *string*  
+    How the results are sorted. Optional, default is `Horizontal`  
+    Possible values: `Horizontal` | `Vertical` | `Area` | `Length` | `Random`  
+    You can use it with the `index` field.
+
+- `index`: *int*  
+    Index to hit. Optional, default is `0`.  
+    If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.
+
 - `only_rec`: *bool*  
     Whether to recognize only (without detection, requires precise `roi`). Optional, default is false.
 
@@ -350,6 +386,15 @@ This task property requires additional fields:
 - `expected`: *int* | *list<int, >*  
     The expected category index.
 
+- `order_by`: *string*  
+    How the results are sorted. Optional, default is `Horizontal`  
+    Possible values: `Horizontal` | `Vertical` | `Area` | `Random`  
+    You can use it with the `index` field.
+
+- `index`: *int*  
+    Index to hit. Optional, default is `0`.  
+    If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.
+
 For example, if you want to recognize whether a cat or a mouse appears in a **fixed position** in the image, and you've trained a model that supports this three-category classification, and you want to click when it recognizes a cat or a mouse but not when it recognizes a dog, the relevant fields would be:
 
 ```jsonc
@@ -391,6 +436,15 @@ This task property requires additional fields:
     Model confidence threshold. Optional, default is 0.3.  
     If it's an array, its length should match the length of the `expected` array.
 
+- `order_by`: *string*  
+    How the results are sorted. Optional, default is `Horizontal`  
+    Possible values: `Horizontal` | `Vertical` | `Area` | `Random`  
+    You can use it with the `index` field.
+
+- `index`: *int*  
+    Index to hit. Optional, default is `0`.  
+    If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.
+
 For example, if you want to detect cats, dogs, and mice in an image and only click when a cat or a mouse is detected but not when a dog is detected, the relevant fields would be:
 
 ```jsonc

docs/zh_cn/3.1-任务流水线协议.md

@@ -229,6 +229,15 @@ graph LR;
     模板匹配阈值。可选，默认 0.7 。  
     若为数组，长度需和 `template` 数组长度相同。
 
+- `order_by`: *string*  
+    结果排序方式。可选，默认 `Horizontal`  
+    可选的值：`Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    可结合 `index` 字段使用。
+
+- `index`: *int*  
+    命中第几个结果。可选，默认 0。  
+    假设共有 N 个结果，则 `index` 的取值范围为 [-N, N - 1]，其中负数使用类 Python 的规则转换为 N - index。若超出范围，则视为当前识别无结果。
+
 - `method`: *int*  
     模板匹配算法，即 cv::TemplateMatchModes。可选，默认 5 。  
     仅支持 1、3、5，可简单理解为越大的越精确，但也会更慢。  
@@ -255,6 +264,15 @@ graph LR;
 - `count`: *int*  
     匹配的特征点的数量要求（阈值），默认 4.
 
+- `order_by`: *string*  
+    结果排序方式。可选，默认 `Horizontal`  
+    可选的值：`Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    可结合 `index` 字段使用。
+
+- `index`: *int*  
+    命中第几个结果。可选，默认 0。  
+    假设共有 N 个结果，则 `index` 的取值范围为 [-N, N - 1]，其中负数使用类 Python 的规则转换为 N - index。若超出范围，则视为当前识别无结果。
+
 - `green_mask`: *bool*  
     是否进行绿色掩码。可选，默认 false。  
     若为 true，可以将图片中不希望匹配的部分涂绿 RGB: (0, 255, 0)，则不对绿色部分进行匹配。
@@ -303,6 +321,15 @@ graph LR;
 - `count`: *int*  
     符合的点的数量要求（阈值）。可选，默认 1。
 
+- `order_by`: *string*  
+    结果排序方式。可选，默认 `Horizontal`  
+    可选的值：`Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    可结合 `index` 字段使用。
+
+- `index`: *int*  
+    命中第几个结果。可选，默认 0。  
+    假设共有 N 个结果，则 `index` 的取值范围为 [-N, N - 1]，其中负数使用类 Python 的规则转换为 N - index。若超出范围，则视为当前识别无结果。
+
 - `connected`: *bool*  
     是否是相连的点才会被计数。可选，默认否。  
     若为是，在完成颜色过滤后，则只会计数像素点 **全部相连** 的最大块。  
@@ -323,6 +350,15 @@ graph LR;
 - `replace`: *array<string, 2>* | *list<array<string, 2>>*  
     部分文字识别结果不准确，进行替换。可选。
 
+- `order_by`: *string*  
+    结果排序方式。可选，默认 `Horizontal`  
+    可选的值：`Horizontal` | `Vertical` | `Area` | `Length` | `Random`  
+    可结合 `index` 字段使用。
+
+- `index`: *int*  
+    命中第几个结果。可选，默认 0。  
+    假设共有 N 个结果，则 `index` 的取值范围为 [-N, N - 1]，其中负数使用类 Python 的规则转换为 N - index。若超出范围，则视为当前识别无结果。
+
 - `only_rec`: *bool*  
     是否仅识别（不进行检测，需要精确设置 `roi`）。可选，默认 false。
 
@@ -354,6 +390,15 @@ graph LR;
 - `expected`: *int* | *list<int, >*  
     期望的分类下标。
 
+- `order_by`: *string*  
+    结果排序方式。可选，默认 `Horizontal`  
+    可选的值：`Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    可结合 `index` 字段使用。
+
+- `index`: *int*  
+    命中第几个结果。可选，默认 0。  
+    假设共有 N 个结果，则 `index` 的取值范围为 [-N, N - 1]，其中负数使用类 Python 的规则转换为 N - index。若超出范围，则视为当前识别无结果。
+
 举例：例如画面中 **固定位置** 可能出现 猫、狗、老鼠，我们训练了支持该三分类的模型。  
 希望识别到 猫 或 老鼠 才点击，而识别到 狗 不点击，则相关字段为  
 
@@ -396,6 +441,15 @@ graph LR;
     模型置信度阈值。可选，默认 0.3 。  
     若为数组，长度需和 `expected` 数组长度相同。
 
+- `order_by`: *string*  
+    结果排序方式。可选，默认 `Horizontal`  
+    可选的值：`Horizontal` | `Vertical` | `Score` | `Area` | `Random`  
+    可结合 `index` 字段使用。
+
+- `index`: *int*  
+    命中第几个结果。可选，默认 0。  
+    假设共有 N 个结果，则 `index` 的取值范围为 [-N, N - 1]，其中负数使用类 Python 的规则转换为 N - index。若超出范围，则视为当前识别无结果。
+
 举例：例如画面中可能出现 猫、狗、老鼠，我们训练了支持该三分类的检测模型。  
 希望检测到 猫 或 老鼠 才点击，而识别到 狗 不点击，则相关字段为  
 

source/MaaFramework/Resource/PipelineResMgr.cpp

@@ -445,6 +445,19 @@ bool PipelineResMgr::parse_template_matcher_param(const json::value& input, MAA_
         return false;
     }
 
+    if (!parse_order_of_result(input, output.order_by, output.result_index, default_value.order_by,
+                               default_value.result_index,
+                               {
+                                   MAA_VISION_NS::ResultOrderBy::Horizontal,
+                                   MAA_VISION_NS::ResultOrderBy::Vertical,
+                                   MAA_VISION_NS::ResultOrderBy::Score,
+                                   MAA_VISION_NS::ResultOrderBy::Area,
+                                   MAA_VISION_NS::ResultOrderBy::Random,
+                               })) {
+        LogError << "failed to parse_order_of_result" << VAR(input);
+        return false;
+    }
+
     if (!get_and_check_value_or_array(input, "template", output.template_paths, default_value.template_paths)) {
         LogError << "failed to get_and_check_value_or_array templates" << VAR(input);
         return false;
@@ -492,6 +505,19 @@ bool PipelineResMgr::parse_feature_matcher_param(const json::value& input, MAA_V
         return false;
     }
 
+    if (!parse_order_of_result(input, output.order_by, output.result_index, default_value.order_by,
+                               default_value.result_index,
+                               {
+                                   MAA_VISION_NS::ResultOrderBy::Horizontal,
+                                   MAA_VISION_NS::ResultOrderBy::Vertical,
+                                   MAA_VISION_NS::ResultOrderBy::Score,
+                                   MAA_VISION_NS::ResultOrderBy::Area,
+                                   MAA_VISION_NS::ResultOrderBy::Random,
+                               })) {
+        LogError << "failed to parse_order_of_result" << VAR(input);
+        return false;
+    }
+
     if (!get_and_check_value(input, "template", output.template_path, default_value.template_path)) {
         LogError << "failed to get_and_check_value template_path" << VAR(input);
         return false;
@@ -554,6 +580,19 @@ bool PipelineResMgr::parse_ocrer_param(const json::value& input, MAA_VISION_NS::
         return false;
     }
 
+    if (!parse_order_of_result(input, output.order_by, output.result_index, default_value.order_by,
+                               default_value.result_index,
+                               {
+                                   MAA_VISION_NS::ResultOrderBy::Horizontal,
+                                   MAA_VISION_NS::ResultOrderBy::Vertical,
+                                   MAA_VISION_NS::ResultOrderBy::Area,
+                                   MAA_VISION_NS::ResultOrderBy::Length,
+                                   MAA_VISION_NS::ResultOrderBy::Random,
+                               })) {
+        LogError << "failed to parse_order_of_result" << VAR(input);
+        return false;
+    }
+
     if (!get_and_check_value(input, "model", output.model, default_value.model)) {
         LogError << "failed to get_and_check_value model" << VAR(input);
         return false;
@@ -635,6 +674,19 @@ bool PipelineResMgr::parse_nn_classifier_param(const json::value& input,
         return false;
     }
 
+    if (!parse_order_of_result(input, output.order_by, output.result_index, default_value.order_by,
+                               default_value.result_index,
+                               {
+                                   MAA_VISION_NS::ResultOrderBy::Horizontal,
+                                   MAA_VISION_NS::ResultOrderBy::Vertical,
+                                   MAA_VISION_NS::ResultOrderBy::Score,
+                                   MAA_VISION_NS::ResultOrderBy::Area,
+                                   MAA_VISION_NS::ResultOrderBy::Random,
+                               })) {
+        LogError << "failed to parse_order_of_result" << VAR(input);
+        return false;
+    }
+
     if (!get_and_check_value(input, "cls_size", output.cls_size, default_value.cls_size)) {
         LogError << "failed to get_and_check_value cls_size" << VAR(input);
         return false;
@@ -671,6 +723,19 @@ bool PipelineResMgr::parse_nn_detector_param(const json::value& input,
         return false;
     }
 
+    if (!parse_order_of_result(input, output.order_by, output.result_index, default_value.order_by,
+                               default_value.result_index,
+                               {
+                                   MAA_VISION_NS::ResultOrderBy::Horizontal,
+                                   MAA_VISION_NS::ResultOrderBy::Vertical,
+                                   MAA_VISION_NS::ResultOrderBy::Score,
+                                   MAA_VISION_NS::ResultOrderBy::Area,
+                                   MAA_VISION_NS::ResultOrderBy::Random,
+                               })) {
+        LogError << "failed to parse_order_of_result" << VAR(input);
+        return false;
+    }
+
     if (!get_and_check_value(input, "cls_size", output.cls_size, default_value.cls_size)) {
         LogError << "failed to get_and_check_value cls_size" << VAR(input);
         return false;
@@ -871,6 +936,47 @@ bool PipelineResMgr::parse_roi(const json::value& input, std::vector<cv::Rect>&
     return !output.empty();
 }
 
+bool PipelineResMgr::parse_order_of_result(const json::value& input, MAA_VISION_NS::ResultOrderBy& output,
+                                           int& output_index, const MAA_VISION_NS::ResultOrderBy& default_value,
+                                           int default_index,
+                                           const std::unordered_set<MAA_VISION_NS::ResultOrderBy>& valid_values)
+{
+    static const std::string kDefaultOrderFlag = "Default";
+    std::string order;
+    if (!get_and_check_value(input, "order", order, kDefaultOrderFlag)) {
+        LogError << "failed to get_and_check_value order" << VAR(input);
+        return false;
+    }
+
+    const std::unordered_map<std::string, MAA_VISION_NS::ResultOrderBy> kOrderMap = {
+        { kDefaultOrderFlag, default_value },
+        { "Horizontal", MAA_VISION_NS::ResultOrderBy::Horizontal },
+        { "Vertical", MAA_VISION_NS::ResultOrderBy::Vertical },
+        { "Score", MAA_VISION_NS::ResultOrderBy::Score },
+        { "Area", MAA_VISION_NS::ResultOrderBy::Area },
+        { "Length", MAA_VISION_NS::ResultOrderBy::Length },
+        { "Random", MAA_VISION_NS::ResultOrderBy::Random },
+        { "Expected", MAA_VISION_NS::ResultOrderBy::Expected },
+    };
+    auto order_iter = kOrderMap.find(order);
+    if (order_iter == kOrderMap.end()) {
+        LogError << "order not found" << VAR(order);
+        return false;
+    }
+    if (!valid_values.contains(order_iter->second)) {
+        LogError << "current recognition not support order" << VAR(order);
+        return false;
+    }
+    output = order_iter->second;
+
+    if (!get_and_check_value(input, "index", output_index, default_index)) {
+        LogError << "failed to get_and_check_value index" << VAR(input);
+        return false;
+    }
+
+    return true;
+}
+
 bool PipelineResMgr::parse_action(const json::value& input, Action::Type& out_type, Action::Param& out_param,
                                   const Action::Type& default_type, const Action::Param& default_param)
 {

source/MaaFramework/Resource/PipelineResMgr.h

@@ -3,6 +3,7 @@
 #include <filesystem>
 #include <set>
 #include <unordered_map>
+#include <unordered_set>
 
 #include <meojson/json.hpp>
 
@@ -44,7 +45,7 @@ class PipelineResMgr : public NonCopyable
     static bool parse_ocrer_param(const json::value& input, MAA_VISION_NS::OCRerParam& output,
                                   const MAA_VISION_NS::OCRerParam& default_value);
     static bool parse_custom_recognition_param(const json::value& input, MAA_VISION_NS::CustomRecognizerParam& output,
-                                              const MAA_VISION_NS::CustomRecognizerParam& default_value);
+                                               const MAA_VISION_NS::CustomRecognizerParam& default_value);
     static bool parse_nn_classifier_param(const json::value& input, MAA_VISION_NS::NeuralNetworkClassifierParam& output,
                                           const MAA_VISION_NS::NeuralNetworkClassifierParam& default_value);
     static bool parse_nn_detector_param(const json::value& input, MAA_VISION_NS::NeuralNetworkDetectorParam& output,
@@ -54,6 +55,9 @@ class PipelineResMgr : public NonCopyable
 
     static bool parse_roi(const json::value& input, std::vector<cv::Rect>& output,
                           const std::vector<cv::Rect>& default_value);
+    static bool parse_order_of_result(const json::value& input, MAA_VISION_NS::ResultOrderBy& output, int& output_index,
+                                      const MAA_VISION_NS::ResultOrderBy& default_value, int default_index,
+                                      const std::unordered_set<MAA_VISION_NS::ResultOrderBy>& valid_values);
 
     static bool parse_action(const json::value& input, Action::Type& out_type, Action::Param& out_param,
                              const Action::Type& default_type, const Action::Param& default_param);

source/MaaFramework/Task/Actuator.cpp

@@ -131,10 +131,20 @@ void Actuator::wait_freezes(const MAA_RES_NS::WaitFreezesParam& param, const cv:
     });
 
     cv::Mat pre_image = controller()->screencap();
+    if (need_to_stop()) {
+        LogInfo << "Task interrupted";
+        return;
+    }
+
     auto pre_time = std::chrono::steady_clock::now();
 
-    while (!need_to_stop()) {
+    while (true) {
         cv::Mat cur_image = controller()->screencap();
+        if (need_to_stop()) {
+            LogInfo << "Task interrupted";
+            return;
+        }
+
         auto ret = comp.analyze(pre_image, cur_image);
         if (ret.empty()) {
             pre_image = cur_image;