Add IPSW plugin

[torarnv]

Description

The IPSW data-source plugin can be used with HashiCorp Packer to fetch information about Apple device firmware, for example for building macOS images with Tart.

https://github.com/torarnv/packer-plugin-ipsw

Integration Tier

Checklist

• Has valid metadata.hcl file in plugin repository.
• Has added integration scripts packer-plugin-scaffolding to plugin repository.
• Has added top-level integration README.md file to plugin docs directory.
• All plugins components have one README.md describing their usage.
• Has a fully synced .web-docs directory ready for publishing to the integrations portal.

[torarnv]

The integration has been submitted in the past, and is listed here: https://developer.hashicorp.com/packer/integrations?components=data-source&filter=ipsw

But the docs link is broken, likely due to the new way that the docs are published.

[torarnv]

https://github.com/torarnv/packer-plugin-ipsw/actions/runs/8204818234/job/22440273928

Shows Error: [validate-hcl] Error: Organization not found for integration identifier: 'packer/torarnv/ipsw'

I assume that's because the plugin hasn't been added to the docs yet?

[torarnv]

I believe you helped me with this in the past @lbajolet-hashicorp

See #12541

[lbajolet-hashicorp]

Hi @torarnv,

It seems we have forgotten to include you in our round for integrations, apologies about that!

Good to see you've updated the docs structure, with the current docs I believe we'll be able to add them to the internal integrations (or update what exists at least), I'll see to this right now.

Thanks for signalling this to us!

[torarnv]

Thanks @lbajolet-hashicorp <3

[lbajolet-hashicorp]

Hi @torarnv,

You can try to run the integration action once again, on our side we did the updates to the documentation, this should work now

[torarnv]

Thanks @lbajolet-hashicorp, the integration now worked. Do I need to do anything else? I see https://developer.hashicorp.com/packer/plugins/datasources/ipsw is still 404, but perhaps it takes a while to go live?

Does the integration automatically publish the new docs, or does it trigger a PR or something similar on your side?

[torarnv]

Integration here: https://github.com/torarnv/packer-plugin-ipsw/actions/runs/8207527813/job/22448992866

[lbajolet-hashicorp]

Hi again! I can confirm your plugin is available, not on plugins anymore though (https://developer.hashicorp.com/packer/integrations/torarnv/ipsw), I'm surprised the link doesn't work anymore, there should be some redirect in place to avoid this problem, that's a discussion I need to have internally.
Also it seems that the documentation is not complete on the new system, I'm not sure why, I'll need to troubleshoot this. I'll make this a priority for Monday.

[torarnv]

Thanks @lbajolet-hashicorp , no rush! 😄 Have a good weekend!

[torarnv]

I'm looking at perhaps renaming this plugin to packer-plugin-appledb, as there are other releases one can fetch from https://appledb.dev/ than just IPSWs. For example Xcode releases. Github will apparently do redirects for git urls, so I assume this would work fine for someone with this in their packer config?

ipsw = {
       version = ">= 0.0.8"
       source = "github.com/torarnv/ipsw"
}
``

If that works fine, then let's hold on publishing the docs for this plugin until I've renamed it :)

[lbajolet-hashicorp]

Hi @torarnv,

No objection to you renaming the project, I'd think Github does redirect so there's a good chance users may be able to download it this way, that being said, I'm not completely certain the installation/loading logic will work if the source has one name, and the binary another (i.e. in this case the plugin source would be github.com/torarnv/ipsw and the plugin would be packer-plugin-appledb).

If you plan on using appledb for future versions of the plugin, I would encourage you instead to fork your ipsw plugin as appledb, update the ipsw plugin to state that it's deprecated/will not be maintained, and point to appledb as replacement.
This advisory message can be part of the docs and the plugin itself (though I'm not certain datasources can print messages to users, if it's in the verbose logs, chances are it won't be super visible).

In any case, the fact that the main docs and the datasources aren't visible most likely point to a problem with our configuration, so I'll take a look this way, and let you know if there's something to change in your repo, this fix will likely be applicable to both plugins.

[lbajolet-hashicorp]

Hi again @torarnv,

I just opened torarnv/packer-plugin-ipsw#1, this removes an invalid flag from the metadata. This invalid flag prevented the ingestion of the payload sent by the publication earlier, hence why we're seeing an empty documentation for the plugin at the moment.

Once merged, you'll need to run the ingestion action once more (no need to bump the release number, it can still be 0.0.12), this will re-trigger the ingestion, and provided there are no other errors, the docs should get updated.

I've also opened a PR internally to enable the redirection from the old URIs, this way links like the one you shared above should work again.

[torarnv]

Ah, awesome, thank you so much @lbajolet-hashicorp !

[torarnv]

Looks like the docs are now live at https://developer.hashicorp.com/packer/integrations/torarnv/ipsw

Thanks again @lbajolet-hashicorp ❤️

As for the invalid flag, I initially followed this guide:

https://developer.hashicorp.com/packer/docs/plugins/creation#example-add-integration-files-to-existing-plugin-repository

Which has the same problematic line in https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.web-docs/metadata.hcl#L11, so perhaps that needs to be updated?

[lbajolet-hashicorp]

Very true, the scaffolding HCL file is indeed wrong, thanks for the notice! I'll fix that immediately.

[lbajolet-hashicorp]

Oh also FYI the old URIs should work too in addition to the new one, I fixed the redirections for your plugin, so now the old links will redirect you to the new docs

[torarnv]

Awesome!! I think we can close this one then, thanks for all your help, it makes working with Packer a joy <3

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.