HTTPCLIENT-2358 Implement a mutual authentication capable SPNEGO scheme

[stoty]

No description provided.

[ok2c]

@stoty I cannot contribute here much. My only wish, and I understand it is a big ask, to also create a compatibility test similar to those we have for BASIC / DIGEST with Squid and Apache HTTPD

https://github.com/apache/httpcomponents-client/tree/master/httpclient5-testing/src/test/java/org/apache/hc/client5/testing/compatibility

[stoty]

I was afraid you're going to say that...

There are two ways to go about that:

• Try to find an docker image with a KDC and web server, and model the test on the Squid test
• Initialize a local (non-dockerized) KDC with Kerby, start Jetty, and use that.

I don't know which of the two is more work, but I know where to copy the test setup from for the second one.

Do you have a preference ?

[ok2c]

I was afraid you're going to say that...

@stoty Kek.

Anyways, We already test for compatibility with Jetty in core by running it in a Docker container

https://github.com/apache/httpcomponents-core/blob/master/httpcore5-testing/src/test/java/org/apache/hc/core5/testing/compatibility/JettyCompatIT.java

I t would be my preferred option but I do not know how difficult it is to pack extra Jetty dependencies into a Docker container. I was also hoping Apache HTTPD or Ngnix might have Kerberos support.

If it is too much effort, disregard my request.

[stoty]

It is a reasonable request, and it IS helpful to catch any regressions, etc.
I will try to find and existing docker image that is close enough.

[garydgregory]

Apache Kerby implements Kerberos, you can get inspiration (copy) how they do it: https://directory.apache.org/kerby/

[stoty]

Kerby is great for in-JVM tests, but for dockerized tests it's probably easier to use MIT kerberos.

[stoty]

After poking around a bit, it seems that neither Apache Httpd, not Nginx supports SPNEGO out of the box.
While both have third party SPNEGO modules, creating a test image with a full working setup looks like a lot of work.

For now, I plan to make a local test with Kerby + Jetty, without Docker.

[michael-o]

A full coverage test contains JGSS (via Tomcat my SpnegoAuthenticator), MIT Kerberos (via mod_auth_gssapi), Microsoft Kerberos (via IIS (SSPI)). Virtually impossible to automate. I will do manual testing anyway.

I have everything in place at work to test.

[michael-o]

This is much less code review. Thanks. I already see some points.

[stoty]

Thanks.
I've made the changes to MutualKerberosConfig.

While I don't use Tomcat, testing Jgss with Jetty is doable, in fact one of the ways I tested this was via the SPNEGO test in HttpCLient->Calcite Avatica->Phoenix PQS.
It does bring in a ton of extra test dependencies, and I couldn't really find where to fit the that integration test in the project.

[stoty]

I have added tests for direct connection to httpd + mod_auth_spnego, @michael-o .
The tests have immediately found a problem, which I also fixed.

The async client cannot use the Subject set with Subject.callAs to figure out the Kerberos credentials (probably due to the auth being run from a different thread)
Is that acceptable, or should we extract and store the Subject ?

Async still works if we create and set KerbersCredentials. (as in the new tests I added)