IDP Connector should return response as Key/Value pair of extracted fields instead of json string

[mathieu-stennier]

What should we do?

IDP Connector should return response as Key/Value pair of extracted fields instead of json string
Full conversation here.

Response provided by a deployed IDP connector would be something like this

"response":"\n\n{\n  \"PolicyholderName\": \"Residential Builders Inc.\",\n  \"PolicyHolderAddress\": \"Parklaan 88 3500 Hasselt Belgie\",\n  \"PolicyholderEnterpriseNumber\": \"1234567890\",\n  \"PolicyholderCapacity\": \"Project Owner\",\n  \"TotalProjectValue\": \"5000000 EUR\",\n  \"ProjectDescription\": \"This project involves the construction of a new 5-story residential apartment building on a currently vacant lot. The building will have reinforced concrete construction, with one underground parking level and five above-ground floors, including a ground floor with retail spaces.\"\n}"

Which is not easily parseable as variables that can then be used in isolations in next steps of the process.

See related FEEL expression feature gap here.

Why should we do it?

I would argue we should parse the results we get from LLM already and provide variables that can be easily mapped as result for further activities in the process.

[mathieu-stennier]

@sahilbhatoacamunda,
@sbuettner recommends us to invest into a proper schema in response that allows you to add more feature in the future. ExtractionResult cotaining extracted fields might be enough for now but then at least we can extend it later on.

[mathieu-stennier]

Note: changing this output is ok before 8.7 because we don't offer support for things built with alphas.

[sahilbhatoacamunda]

@mathieu-stennier @sbuettner
The ExtractionResult will have a response which is of type Map<String, Object>. This map will be built based on the IDP extraction fields passed as taxonomy items in the request.

Use cases

1. The response from LLM is a valid JSON string (all taxonomy items available)> The Connector will parse the JSON string and build the final map by checking if all the taxonomy items are available in the response.
2. The response from LLM is a valid JSON string (some taxonomy items are missing)> The Connector will parse the JSON string and build the final map by checking if all the taxonomy items are available in the response. The system will log a warning LLM model response is missing the following keys: ('missing key 1', 'missing key 2'}).
3. The response from LLM is a valid JSON string containing nested response> The Connector will parse the JSON string promoting nested response and build the final map by checking if all the taxonomy items are available in the response.
4. The response from LLM is an invalid JSON string > The Connector will log an error and return the empty map as a response.

Questions >
From points 3 and 4. Is it okay to continue the operation even if the response from LLM is an invalid JSON string OR if some keys are missing? I'm returning an empty/partial map in that case.

I created a draft PR with the above use cases in mind.