Use 'terminate' processor instead of 'fail'

[andrewkroh]

In some ingest pipelines we want to explicitly stop processing and jump to the next pipeline without any error. So far we accomplished this by using fail, but it had the unwanted side-effect of creating an additional error.message value.

As of 8.16.0 this is now possible with the addition of the terminate processor. So when an integration moves to requiring 8.16 or higher then it can be made to use terminate instead of fail.

This is the current list of integrations that had a fail processor conditional on error.message.

name	conditions.kibana.version	owner.github
abnormal_security	^8.13.0	elastic/security-service-integrations
authentik	^8.13.0	elastic/security-service-integrations
bitdefender	^8.14.3	elastic/security-service-integrations
carbonblack_edr	^8.13.0	elastic/security-service-integrations
checkpoint_email	^8.15.0	elastic/security-service-integrations
claroty_ctd	^8.13.0	elastic/security-service-integrations
digital_guardian	^8.13.0	elastic/security-service-integrations
first_epss	^8.14.0	elastic/security-service-integrations
forcepoint_web	^8.13.0	elastic/security-service-integrations
github	^8.13.0	elastic/security-service-integrations
jumpcloud	^8.13.0	elastic/security-service-integrations
m365_defender	^8.13.0	elastic/security-service-integrations
mattermost	^8.13.0	elastic/security-service-integrations
microsoft_sentinel	^8.14.0	elastic/security-service-integrations
mimecast	^8.14.0	elastic/security-service-integrations
mongodb_atlas	^8.13.0	elastic/obs-infraobs-integrations
qualys_vmdr	^8.13.0	elastic/security-service-integrations
servicenow	^8.14.0	elastic/security-service-integrations
spycloud	^8.13.0	elastic/security-service-integrations
sublime_security	^8.13.0	elastic/security-service-integrations
symantec_edr_cloud	^8.13.0	elastic/security-service-integrations
symantec_endpoint_security	^8.13.0	elastic/security-service-integrations
tenable_io	^8.13.0	elastic/security-service-integrations
ti_custom	^8.14.0	elastic/security-service-integrations
ti_opencti	^8.13.0	elastic/security-service-integrations
tines	^8.13.0	elastic/security-service-integrations
wiz	^8.16.0	elastic/security-service-integrations
zscaler_zia	^8.13.0	elastic/security-service-integrations

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

It seems elastic-package/package-spec is not yet aware of this.

Error: building package failed: invalid content found in built zip package: found 3 validation errors:
   1. file "…/github.com/elastic/integrations/build/packages/wiz-2.6.0.zip/data_stream/audit/elasticsearch/ingest_pipeline/default.yml" is invalid: field processors.1: Additional property terminate is not allowed
   2. file "…/github.com/elastic/integrations/build/packages/wiz-2.6.0.zip/data_stream/issue/elasticsearch/ingest_pipeline/default.yml" is invalid: field processors.1: Additional property terminate is not allowed
   3. file "…/github.com/elastic/integrations/build/packages/wiz-2.6.0.zip/data_stream/vulnerability/elasticsearch/ingest_pipeline/default.yml" is invalid: field processors.1: Additional property terminate is not allowed

[andrewkroh]

Package-spec change proposal: elastic/package-spec#846