fix: UTF-8 data in the JSON response for ansible 2.18<

[igelkotten]

SUMMARY

Fixes #692

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

This is done to solve mongodb_info module related

ADDITIONAL INFORMATION

See #692

Before output of mongodb_info

//When a user is present the userid
\"userId\": \"<E\\udc91\\udc98\\b\\udcacBq\\udcbc\\udcf4|\\udcc8\\udcfd\\r`I\",
// Including the deprecation warning in ansible-core<2.18 and in >2.18 an error "returned non UTF-8 data in the JSON response"

//When the server has a replica set it gives

         "general": {
            "$clusterTime": {
                "clusterTime": "Timestamp(1739390814, 1)",
                "signature": {
                    "hash": "\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000",

After the fix the signature looks correct.

                "signature": {
                    "hash": "AAAAAAAAAAAAAAAAAAAAAAAAAAA=",

Equivalent to what mongosh: rs.status()

    signature: {
      hash: Binary.createFromBase64('AAAAAAAAAAAAAAAAAAAAAAAAAAA=', 0),

The userid from mongodb_info output look like this:

                    "userId": "748a2493-e8c5-46b1-8733-151ca8f11690"

As the userid from mongosh show users look like:
``
userId: UUID('748a2493-e8c5-46b1-8733-151ca8f11690'),

And it works with ansible-core 2.18

[codecov]

Codecov Report

Attention: Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.

Project coverage is 73.21%. Comparing base (979ca9f) to head (3d910c9).

Files with missing lines	Patch %	Lines
plugins/module_utils/mongodb_common.py	55.55%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #693      +/-   ##
==========================================
- Coverage   73.55%   73.21%   -0.34%     
==========================================
  Files           6        6              
  Lines         934      941       +7     
  Branches      143      145       +2     
==========================================
+ Hits          687      689       +2     
- Misses        210      212       +2     
- Partials       37       40       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rhysmeister]

Not really sure about the need to test the length of the value here. What happens if there's a bytes value of <16, 17,18,19 or >20? A generic solution would probably make more sense.

[igelkotten]

It can also be solved here as I think was the intention and possibly something that worked once (https://github.com/ansible-collections/community.mongodb/blob/979ca9f72cb47e7571d89531bfe17375c011193e/plugins/modules/mongodb_info.py#L223C1-L226C1 The forloop in that function to check if isinstance(val, UUID) but I never got it to trigger on being uuid when debugging that. It can likely be omitted.

You mean we check if it is bytes and then have a length-if-section for 16(being UUID) and one for 20(signature-hash) and then possibly another catch all?

[rhysmeister]

Well.. something like that i guess, otherwise the bug is still there.

[rhysmeister]

Tests exists for both sytandalone and replicaset MongoDB Instance in https://github.com/ansible-collections/community.mongodb/tree/master/tests/integration/targets/mongodb_info/tasks

It's probably easy to add a test for this case.

[igelkotten]

[master_tasks/mongod_replicaset](https://github.com/ansible-collections/community.mongodb/blob/master/tests/integration/targets/master_tasks/mongod_replicaset.yml)

Is it possible to use the mongodb_info there?

So like...

[...]
- name: Wait for mongod to start responding
  wait_for:
    port: '{{ item }}'
  with_items: '{{ mongodb_nodes }}'

- name: Obtain mongodb info using module
  community.mongodb.mongodb_info:
         //possibly authstuff goes here
  register: mongodb_info_module_content
 
- name: debug mongocontent
  set_facts:
   mongodb_info_signaturehash: mongodb_info_module_content.general['$clusterTime'].signature.hash
   // other values we might want to compare?

- name: run shell command for obtaining signature
  shell:  mongosh <authstuff> --eval "rs.status().\$clusterTime.signature.hash | awk -F "\'" '{print $2}'
  register: shell_mongosh_signaturehash_raw
// This should give us something like this:
//Binary.createFromBase64('AAAAAAAAAAAAAAAAAAAAAAAAAAA=', 0)

- name: incorrect values from mongosh and mongodb_info signature hash values doesnt' match
  fail:
   msg: |
    shell_mongosh_signaturehash_raw: {{ shell_mongosh_signaturehash_raw }}
    mongodb_info_signaturehash: {{ mongodb_info_signaturehash }}
   when: shell_mongosh_signaturehash_raw != mongodb_info_signaturehash

are there tests running that also create users?

[rhysmeister]

[master_tasks/mongod_replicaset](https://github.com/ansible-collections/community.mongodb/blob/master/tests/integration/targets/master_tasks/mongod_replicaset.yml)

Is it possible to use the mongodb_info there?

So like...

[...]
- name: Wait for mongod to start responding
  wait_for:
    port: '{{ item }}'
  with_items: '{{ mongodb_nodes }}'

- name: Obtain mongodb info using module
  community.mongodb.mongodb_info:
         //possibly authstuff goes here
  register: mongodb_info_module_content
 
- name: debug mongocontent
  set_facts:
   mongodb_info_signaturehash: mongodb_info_module_content.general['$clusterTime'].signature.hash
   // other values we might want to compare?

- name: run shell command for obtaining signature
  shell:  mongosh <authstuff> --eval "rs.status().\$clusterTime.signature.hash | awk -F "\'" '{print $2}'
  register: shell_mongosh_signaturehash_raw
// This should give us something like this:
//Binary.createFromBase64('AAAAAAAAAAAAAAAAAAAAAAAAAAA=', 0)

- name: incorrect values from mongosh and mongodb_info signature hash values doesnt' match
  fail:
   msg: |
    shell_mongosh_signaturehash_raw: {{ shell_mongosh_signaturehash_raw }}
    mongodb_info_signaturehash: {{ mongodb_info_signaturehash }}
   when: shell_mongosh_signaturehash_raw != mongodb_info_signaturehash

are there tests running that also create users?

You can use all of the mongodb_ modules here. Tests exist for mongodb_user, if that's what you mean.

[igelkotten]

You can use all of the mongodb_ modules here. Tests exist for mongodb_user, if that's what you mean.

Yes thank you!

I am more concerned about the userid part as it will fail the entire mongodb_info module. While the signature hash just show scrambled output.

This seems useful,

mongodb_info

I have played around with assert in my playbook. This seems to work:

     - assert:
         that:
          - mongodb_info_module_content.users.admin.dummyuser.userId | regex_search('^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$')

While it fails when i change it to .dummyuser._id (and ofc it fails when .dummyuser.doesntexist)

      - result.users.rhys.user3  | regex_search('^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$')

But I guess this should be added after the fix have been applied that corrects the userid