feat: in-cluster zot registry as part of deployment

[ikhandamirov]

What this PR does / why we need it

With this PR an in-cluster instance of zot OCI registry will be deployed aside to the controllers:

• make deploy-dev makes zot accessible via http.
• make deploy makes zot accessible via https.

Which issue(s) this PR fixes

This PR is in the context of #75 and should be merged before #98

[Skarlso]

So, help me understand the purpose here please. This works for the Zot, yes. Awesome. But the actual controller is not able to start up since it's using controller:latest as image. And that will not work unless we change it to something that we already pushed. Which is fine, however, that's not the development flow, right?

For development, we are using Tilt, for rapid changes and testing the controller with local code and to make sure RBAC and things like that work, we don't just launch a local service. What I'm saying is that we don't normally use make deploy. And these changes don't help the Tilt flow at all since the Tiltfile ofc hasn't been modified.

What is the expected flow here? How would we use this during development or during a release? During a release, we use a single Install.yaml that is released as a file in the release. But for that, we just use make kustomize or something. So please help me figure out what these two targets do?

Also, I know that we can use IMG=point to local registry, but using that means you have to build the entire thing and push it and redeploy it all the time. Which is why tilt was brought it that does this flow for you.

[ikhandamirov]

So, help me understand the purpose here please...

make deploy is not invented by me. Whatever it did, it still does it. I was just looking for the simplest way to extend it with zot deployment. The reason: zot is going to be used as storage by the new snapshot controller.

Tilt was not on my radar and, I believe, is a different use case. Which however also needs to be addressed, if it is used. @frewilhelm, do you use Tilt for local development?