Skip to content

@azure/search-documents v12.1.0: Missing @search.rerankerScore in results when using queryType: "semantic" #33855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zyptai opened this issue Apr 15, 2025 · 1 comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Search Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@zyptai
Copy link

zyptai commented Apr 15, 2025

Environment Details:
SDK Version: @azure/search-documents@12.1.0
Node.js Version: v22.14.0 (Please confirm or update with your exact version: run node -v)
Operating System: Windows (Please confirm or update)
IDE: VS Code

Describe the Bug:
When performing a search using searchClient.search with queryType set to "semantic", the results returned by the SDK are missing the @search.rerankerScore field. This occurs both when attempting a hybrid search (combining semantic with vectorSearchOptions) and when attempting a pure semantic search (omitting vectorSearchOptions).
However, executing the equivalent semantic search directly in the Azure Portal Search Explorer successfully returns the @search.rerankerScore field with numeric values for the same index and query, confirming the Azure Search service and index semantic configuration are functioning correctly.

This suggests an issue within the @azure/search-documents SDK v12.1.0 related to requesting or parsing the semantic reranker score when queryType: "semantic" is specified.

Steps to Reproduce:
Have an Azure Cognitive Search index configured with:
A vector field (e.g., descriptionVector).
A valid semantic configuration (e.g., named my-semantic-config) with appropriate title, content (description), and keyword fields selected.
The Search Service tier must support Semantic Search (Basic, Standard, etc.).
Use the following Node.js code with @azure/search-documents@12.1.0:
// Assuming searchClient is initialized correctly (see setup in logs below)
// import { SearchClient, AzureKeyCredential, SearchOptions } from "@azure/search-documents";

// Example query and vector
const query = "2025 SuccessFactors date"; // Or another query known to hit relevant documents
const queryVector: number[] = [/* Your actual 1536-dimension query vector here */]; // Generated via OpenAI embedding model

// Search Options structure that compiles successfully with SDK v12 types
const searchOptions: SearchOptions = { // SearchDocument defined appropriately
select: [
"description",
"chunkindex",
"filename",
"fileUrl",
"docId",
"docTitle",
"lastmodified",
"totalChuncks"
],
top: 50,
queryType: "semantic" as const,
semanticSearchOptions: {
configurationName: "my-semantic-config", // Use your actual configuration name
answers: {
answerType: "extractive",
count: 3
},
captions: {
captionType: "extractive",
highlight: true
}
},
// NOTE: Issue persists even if vectorSearchOptions is commented out
vectorSearchOptions: {
queries: [{
kind: "vector" as const,
fields: ["descriptionVector"],
kNearestNeighborsCount: 10,
vector: queryVector,
}]
},
searchMode: "any" as const,
includeTotalCount: true
};

try {
const searchResultsIterator = await searchClient.search(query, searchOptions);
const results: any[] = []; // Using any[] for simplicity in bug report
for await (const result of searchResultsIterator.results) {
results.push(result);
}

console.log("SDK Search Results Sample:", JSON.stringify(results.slice(0, 3), null, 2));
// Check if '@search.rerankerScore' exists and has a value in the results

} catch (error) {
console.error("Search Error:", error);
}
Use code with caution.
TypeScript
Run the code and observe the output/logs.
Expected Behavior:
The results array returned by searchClient.search should contain objects where each object corresponding to a search hit includes a property named @search.rerankerScore with a numeric value (e.g., 2.4998).
Actual Behavior:
The results returned by the SDK do not contain the @search.rerankerScore property. Logging the results shows the field is missing.
Example Log Snippet from SDK Execution (Pure Semantic Search Test):
[2025-04-15T17:46:47.757Z] INFO: [4f466452][performHybridSearch] Search completed {"totalResultsFetched":50,"topResultScore":"11.4585","topResultRerankerScore":"N/A"}
[2025-04-15T17:46:47.759Z] INFO: [4f466452][performHybridSearch] Top 3 Results (Sorted by Reranker Score):
[2025-04-15T17:46:47.760Z] INFO: [1] File: Phase 2 Planning Summit Day 1 (1) (1) (1).pdf, RerankerScore: N/A, HybridScore: 11.4585, Chunk: 54
[2025-04-15T17:46:47.761Z] INFO: [2] File: Phase 2 Planning Summit Day 1 (1) (1) (1).docx, RerankerScore: N/A, HybridScore: 8.2526, Chunk: 44
[2025-04-15T17:46:47.761Z] INFO: [3] File: Fake meeting template - just a little discussion.docx, RerankerScore: N/A, HybridScore: 8.2526, Chunk: 1
Use code with caution.
(Note: The topResultScore (hybrid score) is returned correctly)
Supporting Evidence (Azure Portal):
Executing the same query via the Azure Portal Search Explorer does return the reranker score.
Portal Query JSON:
{
"search": "SuccessFactors 2025 date",
"count": true,
"queryType": "semantic",
"semanticConfiguration": "my-semantic-config", // <<< Actual config name used
"queryLanguage": "en-us",
"captions": "extractive",
"answers": "extractive|count-3",
"select": "docId,docTitle,description,filename,fileUrl,lastmodified,chunkindex,totalChuncks",
"top": 10
}
Use code with caution.
Json
Portal Result Snippet:
{
"@search.score": 8.205243,
"@search.rerankerScore": 2.4998013973236084, // <<< SCORE IS PRESENT HERE
"@search.captions": [ /* ... / ],
"docId": "01ICKOLJXEFIGVESYBIZGJX5CZUCVDEXGQ-26",
"docTitle": "Phase 2 Planning Summit Day 1 (1) (1).docx",
"description": "So yeah, that's, I guess we'll be wrapping up then...",
/
... other fields ... /
},
{
"@search.score": 11.458464,
"@search.rerankerScore": 2.4457449913024902, // <<< SCORE IS PRESENT HERE
"@search.captions": [ /
... / ],
"docId": "01ICKOLJQOQ25UN6GRBVH3XZME5S2FSB6J-54",
/
... */
},
// ... more results with reranker scores
Use code with caution.
Json
Additional Context:
Troubleshooting confirmed the issue persists even when vectorSearchOptions are removed from the SDK call (i.e., pure semantic search via SDK still lacks the score).
Various structures for searchOptions were attempted to align with TypeScript definitions, but none resulted in the reranker score being returned via the SDK.
The application relies on @search.rerankerScore for accurate relevance determination in a RAG pipeline.
Please investigate why the @search.rerankerScore might be missing when using queryType: "semantic" with the @azure/search-documents@12.1.0 SDK. Thank you!

@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Search Service Attention Workflow: This issue is responsible by Azure service team. labels Apr 15, 2025
Copy link

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @bleroy @MarkHeff @miwelsh @tjacobhi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Search Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

1 participant