tools: Accept hex representation as binary input (KhronosGroup#5870)

Sometimes when debugging or logging, SPIR-V may be dumped as a stream of
hex values.  There are tools to convert such a stream to binary
(such as [1]) but they create an inconvenient extra step when for
example the disassembly of that hex stream is needed.

[1]: https://www.khronos.org/spir/visualizer/hexdump.html

In this change, the binary reader used by the tools is enhanced to
detect when the binary is actually a hex stream, and parse that instead.
The following formats are accepted, detected based on how the SPIR-V
magic number is output:

=== Words

If the first token of the hex stream is one of 0x07230203, 0x7230203,
x07230203, or x7230203, the hex stream is expected to consist of 32-bit
hex words prefixed with 0x or x.  For example:

    0x7230203, 0x10400, 0x180001, 0x79, 0x0

is parsed as:

    0x07230203 0x00010400 0x00180001 0x00000079 0x00000000

Note that `,` is optional in the stream, but the hex values are expected
to be delimited by either `,` or whitespace.

=== Bytes With Prefix

If the first token of the hex stream is one of 0x07, 0x7, x07, x7, 0x03,
0x3, x03, or x3, the hex stream is expected to consist of 8-bit hex
bytes prefixed with 0x or x.  If the first token has a value of 7, the
stream is big-endian.  Otherwise it's little-endian.  For example:

    0x3, 0x2, 0x23, 0x7, 0x0, 0x4, 0x1, 0x0, 0x1, 0x0, 0x18, 0x0, 0x79, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0

is parsed as:

    0x07230203 0x00010400 0x00180001 0x00000079 0x00000000

Similar to "Words", `,` is optional in the stream, but the hex values
are expected to be delimited by either `,` or whitespace.

=== Bytes Without Prefix

If the first two characters of the hex stream is 07, or 03, the hex
stream is expected to consist of 8-bit hex bytes of 2 characters each.
If the first token is 07, the stream is big-endian.  Otherwise it's
little-endian.  Unlike the other modes, delimiter is optional (which
automatically handles 32-bit word streams), but no 0-padding is done.
For example, all of the following:

    03, 02, 23, 07, 00, 04, 01, 00, 01, 00, 18, 00, 79, 00, 00, 00, 00, 00, 00, 00
    03 02 23 07 00 04 01 00 01 00 18 00 79 00 00 00 00 00 00 00
    03022307 00040100 01001800 79000000 00000000
    07,23,02,03,00,01,04,00,00,18,00,01,00,00,00,79,00,00,00,00
    07230203, 00010400, 00180001, 00000079, 00000000

are parsed as:

    0x07230203 0x00010400 0x00180001 0x00000079 0x00000000

Author: ShabbyX

BUILD.bazel

@@ -447,6 +447,7 @@ cc_library(
         "tools_util",
         ":spirv_tools_internal",
         ":test_lib",
+        ":tools_io",
         "@googletest//:gtest",
         "@googletest//:gtest_main",
     ],

BUILD.gn

@@ -1377,6 +1377,7 @@ if (build_with_chromium && spvtools_build_executables) {
       "test/fix_word_test.cpp",
       "test/generator_magic_number_test.cpp",
       "test/hex_float_test.cpp",
+      "test/hex_to_text_test.cpp",
       "test/immediate_int_test.cpp",
       "test/libspirv_macros_test.cpp",
       "test/name_mapper_test.cpp",
@@ -1424,6 +1425,7 @@ if (build_with_chromium && spvtools_build_executables) {
       ":spvtools_language_header_cldebuginfo100",
       ":spvtools_language_header_debuginfo",
       ":spvtools_language_header_vkdebuginfo100",
+      ":spvtools_tools_io",
       ":spvtools_val",
       "//testing/gmock",
       "//testing/gtest",

README.md

@@ -80,6 +80,8 @@ further notice.
 * Assembler only does basic syntax checking.  No cross validation of
   IDs or types is performed, except to check literal arguments to
   `OpConstant`, `OpSpecConstant`, and `OpSwitch`.
+* Where tools expect binary input, a hex stream may be provided instead.  See
+  `spirv-dis --help`.
 
 See [`docs/syntax.md`](docs/syntax.md) for the assembly language syntax.
 

test/CMakeLists.txt

@@ -88,6 +88,7 @@ endfunction()
 set(TEST_SOURCES
   test_fixture.h
   unit_spirv.h
+  ${spirv-tools_SOURCE_DIR}/tools/io.h
 
   assembly_context_test.cpp
   assembly_format_test.cpp
@@ -110,6 +111,7 @@ set(TEST_SOURCES
   fix_word_test.cpp
   generator_magic_number_test.cpp
   hex_float_test.cpp
+  hex_to_text_test.cpp
   immediate_int_test.cpp
   libspirv_macros_test.cpp
   named_id_test.cpp
@@ -154,6 +156,7 @@ set(TEST_SOURCES
   to_string_test.cpp
 
   unit_spirv.cpp
+  ${spirv-tools_SOURCE_DIR}/tools/io.cpp
 )
 
 spvtools_pch(TEST_SOURCES pch_test)

test/diff/diff_test.cpp

@@ -20,7 +20,6 @@
 #include "source/opt/ir_context.h"
 #include "source/spirv_constant.h"
 #include "spirv-tools/libspirv.hpp"
-#include "tools/io.h"
 #include "tools/util/cli_consumer.h"
 
 #include <fstream>

test/diff/diff_test_utils.cpp

@@ -18,7 +18,6 @@
 #include "source/opt/ir_context.h"
 
 #include "spirv-tools/libspirv.hpp"
-#include "tools/io.h"
 #include "tools/util/cli_consumer.h"
 
 #include "gtest/gtest.h"