-
-
-
+
-This tool collects data from the APIC to be used by Cisco Services in the ACI -Health Check. +This tool collects data from the APIC to be used by Cisco Services in the ACI Health Check. -Binary releases are available -[in the releases tab](https://github.com/brightpuddle/vetr-collector/releases/latest). -Please always use the latest release unless you have a known requirement to use -an earlier version. +Binary releases are available [in the releases tab](https://github.com/brightpuddle/vetr-collector/releases/latest). It's recommended to always use the latest release unless you have a known requirement to use an earlier version. -# Purpose +Purpose +======= -This tool performs data collection for the ACI health check. This tool can be -run from any computer with access to the APIC, including the APIC itself. +This tool performs data collection for the ACI health check. This tool can be run from any computer with access to the APIC, including the APIC itself. -Once the collection is complete, the tool will create an `aci-vetr-data.zip` -file. This file should be provided to the Cisco Services ACI consulting engineer -for further analysis. +Once the collection is complete, the tool will create an `aci-vetr-data.zip` file. This file should be provided to the Cisco Services ACI consulting engineer for further analysis. -The tool also creates a log file that can be reviewed and/or provided to Cisco -to troubleshoot any issues with the collection process. Note, that this file -will only be available in a failure scenario; upon successful collection this -file is bundled into the `aci-vetr-data.zip` file along with collection data. +Note that in addition to Cisco Services analysis, this file can also be read by open source third party tools to review the configuration directly. See the [Third Party Tooling](#third-party-tooling) section for more details. -# How it works +The tool also creates a log file that can be reviewed and/or provided to Cisco to troubleshoot any issues with the collection process. Note, that this file will only be available in a failure scenario; upon successful collection this file is bundled into the `aci-vetr-data.zip` file along with collection data. -The tool collects data from a number of endpoints on the APIC for configuration, -current faults, scale-related data, etc. The results of these queries are -archived in a zip file to be shared with Cisco. The tool currently has no -interaction with the switches--all data is collected from the APIC, via the API. +How it works +============ -The following file can be referenced to see the class queries performed by this -tool: +The tool collects data from a number of endpoints on the APIC for configuration, current faults, scale-related data, etc. The results of these queries are archived in a zip file to be shared with Cisco. The tool currently has no interaction with the switches--all data is collected from the APIC, via the API. + +The following file can be referenced to see the class queries performed by this tool: https://github.com/brightpuddle/vetr-collector/blob/master/pkg/req/reqs.json -**Note** that this file is part of the CI/CD process for this tool, so is always -up to date with the latest query data. +**Note** that this file is part of the CI/CD process for this tool, so is always up to date with the latest query data. -# Safety/Security +Safety/Security +=============== -- All of the queries performed by this tool are also performed by the APIC GUI, - so there is no more risk than clicking through the GUI. -- Queries to the APIC are batched and throttled as to ensure reduced load on the - APIC. Again, this results in less impact to the API than the GUI. -- The APIC has internal safeguards to protect against excess API usage -- API interaction in ACI has no impact on data forwarding behavior -- This tool is open source and can be compiled manually with the Go compiler +- All of the queries performed by this tool are also performed by the APIC GUI, so there is no more risk than clicking through the GUI. +- Queries to the APIC are batched and throttled as to ensure reduced load on the APIC. Again, this results in less impact to the API than the GUI. +- The APIC has internal safeguards to protect against excess API usage +- API interaction in ACI has no impact on data forwarding behavior +- This tool is open source and can be compiled manually with the Go compiler -This tool only collects the output of the afformentioned managed objects. -Documentation on these endpoints is available in the -[full API documentation](https://developer.cisco.com/site/apic-mim-ref-api/). -Credentials are only used at the point of collection and are not stored in any -way. +This tool only collects the output of the afformentioned managed objects. Documentation on these endpoints is available in the [full API documentation](https://developer.cisco.com/site/apic-mim-ref-api/). Credentials are only used at the point of collection and are not stored in any way. -All data provided to Cisco will be maintained under Cisco's -[data retention policy](https://www.cisco.com/c/en/us/about/trust-center/global-privacy-policy.html). +All data provided to Cisco will be maintained under Cisco's [data retention policy](https://www.cisco.com/c/en/us/about/trust-center/global-privacy-policy.html). -Lastly, the binary collector is not strictly required. The releases downloads -also include a shell script, named `vetr-collector.sh`. This file can be copied -up to the APIC using SCP, and run locally. The script uses icurl and zip to -generate the same output as the binary collector. Note, that the script will -need to be marked as executable to run on the APIC, i.e. -`chmod +x vetr-collector.sh`. This is a more involved process and doesn't -include the batching, throttling, and pagination capabilities of the binary -collector, but can be used as an alternative collection mechanism if required. +Lastly, the binary collector is not strictly required. The releases downloads also include a shell script, named `vetr-collector.sh`. This file can be copied up to the APIC using SCP, and run locally. The script uses icurl and zip to generate the same output as the binary collector. Note, that the script will need to be marked as executable to run on the APIC, i.e. `chmod +x vetr-collector.sh`. This is a more involved process and doesn't include the batching, throttling, and pagination capabilities of the binary collector, but can be used as an alternative collection mechanism if required. -# Usage +Usage +===== -All command line parameters are optional; the tool will prompt for any missing -information. Use the `--help` option to see this output from the CLI. +All command line parameters are optional; the tool will prompt for any missing information. Use the `--help` option to see this output from the CLI. -**Note** that only `apic`, `username`, and `password` are typically required. -The remainder of the options exist to work around uncommon connectivity -challenges, e.g. a long RTT or slow response from the APIC. +**Note** that only `apic`, `username`, and `password` are typically required. The remainder of the options exist to work around uncommon connectivity challenges, e.g. a long RTT or slow response from the APIC. ``` ACI vetR collector @@ -107,7 +79,8 @@ Options: ``` -## Performance and Troubleshooting +Performance and Troubleshooting +------------------------------- In general the collector is expected to run very quickly and have no issues. That said, one error sometimes encountered is a class with too much data. As an example of this, suppose a fabric has a very large number of static path bindings. The collector queries objects by class, so all static path bindings will be requested in a single query, and when a response has too much data, instead of sending the response data, the APIC will respond with an error. @@ -125,9 +98,17 @@ Again, these and othe configurable settings should not generally need to be modi Static binaries are provided for convenience and are generally preferred; however, if you'd like to run the code directly from source, e.g. for security auditing, this is also an option. -1. [Install Go](https://go.dev/doc/install) -2. Clone the repo -3. `go mod download` -4. `go run ./cmd/collector/*.go` +1. [Install Go](https://go.dev/doc/install) +2. Clone the repo +3. `go mod download` +4. `go run ./cmd/collector/*.go` If on Windows, it's recommended to use Powershell or WSL to avoid issues with ANSI escape sequences and path slash direction. + +Third Party Tooling +=================== + +The following tools can be used to visualize or analyze the vetR collection file. Note that these are not owned by Cisco Systems. + +- [vetR Summarizer](https://github.com/Tes3awy/vetr-summarizer) - Visualize and summarize vetR collection data through a web UI +- [reQuery](https://github.com/brightpuddle/requery) - Run moquery-like queries against the collection file from the CLI diff --git a/go.mod b/go.mod index 944e4e8..7e19e74 100644 --- a/go.mod +++ b/go.mod @@ -3,15 +3,15 @@ module collector go 1.22 require ( - github.com/alexflint/go-arg v1.4.3 + github.com/alexflint/go-arg v1.5.1 github.com/gofiber/fiber/v2 v2.42.0 - github.com/rs/zerolog v1.32.0 + github.com/rs/zerolog v1.33.0 github.com/segmentio/encoding v0.3.6 github.com/stretchr/testify v1.7.1 - github.com/tidwall/gjson v1.17.1 + github.com/tidwall/gjson v1.17.3 github.com/tidwall/sjson v1.2.5 - golang.org/x/sync v0.6.0 - golang.org/x/term v0.18.0 + golang.org/x/sync v0.8.0 + golang.org/x/term v0.24.0 gopkg.in/h2non/gock.v1 v1.1.2 gopkg.in/yaml.v2 v2.4.0 ) @@ -41,7 +41,7 @@ require ( github.com/valyala/bytebufferpool v1.0.0 // indirect github.com/valyala/fasthttp v1.44.0 // indirect github.com/valyala/tcplisten v1.0.0 // indirect - golang.org/x/sys v0.18.0 // indirect + golang.org/x/sys v0.25.0 // indirect gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/go.sum b/go.sum index b9c465d..a5b4b8a 100644 --- a/go.sum +++ b/go.sum @@ -1,5 +1,7 @@ github.com/alexflint/go-arg v1.4.3 h1:9rwwEBpMXfKQKceuZfYcwuc/7YY7tWJbFsgG5cAU/uo= github.com/alexflint/go-arg v1.4.3/go.mod h1:3PZ/wp/8HuqRZMUUgu7I+e1qcpUbvmS258mRXkFH4IA= +github.com/alexflint/go-arg v1.5.1 h1:nBuWUCpuRy0snAG+uIJ6N0UvYxpxA0/ghA/AaHxlT8Y= +github.com/alexflint/go-arg v1.5.1/go.mod h1:A7vTJzvjoaSTypg4biM5uYNTkJ27SkNTArtYXnlqVO8= github.com/alexflint/go-scalar v1.1.0/go.mod h1:LoFvNMqS1CPrMVltza4LvnGKhaSpc3oyLEBUZVhhS2o= github.com/alexflint/go-scalar v1.2.0 h1:WR7JPKkeNpnYIOfHRa7ivM21aWAdHD0gEWHCx+WQBRw= github.com/alexflint/go-scalar v1.2.0/go.mod h1:LoFvNMqS1CPrMVltza4LvnGKhaSpc3oyLEBUZVhhS2o= @@ -50,6 +52,8 @@ github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUc github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg= github.com/rs/zerolog v1.32.0 h1:keLypqrlIjaFsbmJOBdB/qvyF8KEtCWHwobLp5l/mQ0= github.com/rs/zerolog v1.32.0/go.mod h1:/7mN4D5sKwJLZQ2b/znpjC3/GQWY/xaDXUM0kKWRHss= +github.com/rs/zerolog v1.33.0 h1:1cU2KZkvPxNyfgEmhHAz/1A9Bz+llsdYzklWFzgp0r8= +github.com/rs/zerolog v1.33.0/go.mod h1:/7mN4D5sKwJLZQ2b/znpjC3/GQWY/xaDXUM0kKWRHss= github.com/savsgio/dictpool v0.0.0-20221023140959-7bf2e61cea94 h1:rmMl4fXJhKMNWl+K+r/fq4FbbKI+Ia2m9hYBLm2h4G4= github.com/savsgio/dictpool v0.0.0-20221023140959-7bf2e61cea94/go.mod h1:90zrgN3D/WJsDd1iXHT96alCoN2KJo6/4x1DZC3wZs8= github.com/savsgio/gotils v0.0.0-20220530130905-52f3993e8d6d/go.mod h1:Gy+0tqhJvgGlqnTF8CVGP0AaGRjwBtXs/a5PA0Y3+A4= @@ -68,6 +72,8 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= github.com/tidwall/gjson v1.17.1 h1:wlYEnwqAHgzmhNUFfw7Xalt2JzQvsMx2Se4PcoFCT/U= github.com/tidwall/gjson v1.17.1/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= +github.com/tidwall/gjson v1.17.3 h1:bwWLZU7icoKRG+C+0PNwIKC6FCJO/Q3p2pZvuP0jN94= +github.com/tidwall/gjson v1.17.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= @@ -108,6 +114,8 @@ golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ= golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ= +golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -124,11 +132,15 @@ golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4= golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.25.0 h1:r+8e+loiHxRqhXVl6ML1nO3l1+oFoWbnlu2Ehimmi34= +golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA= golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8= golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58= +golang.org/x/term v0.24.0 h1:Mh5cbb+Zk2hqqXNO7S1iTjEphVL+jb8ZWaqh/g+JWkM= +golang.org/x/term v0.24.0/go.mod h1:lOBK/LVxemqiMij05LGJ0tzNr8xlmwBRJ81PX6wVLH8= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=