Merge branch 'main' of github.com:rangsimanketkaew/tensorflow_cpp_api into main

Author: rangsimanketkaew

README.md

@@ -5,16 +5,16 @@ https://www.tensorflow.org/api_docs/cc
 
 ## What is it exactly?
 
-The functionality of the TensorFlow (TF) is implemented in many languages and one of them is C++, 
+The functionality of TensorFlow (TF) is implemented in many languages and one of them is C++, 
 but Python is used as an interface (front-end) between the backend system and the users. 
-The TF C++ API his built on TensorFlow Session (in the old version) and on TensorFlow ClientSession (in the new version). 
-One can make use either of these to execute TensorFlow graphs that have been built using the Python API and serialized to a `GraphDef`.
+The TF C++ API is built on TensorFlow Session (in the old version) and TensorFlow ClientSession (in the new version). 
+One can make use of either of these to execute TensorFlow graphs that have been built using the Python API and serialized to a `GraphDef`.
 
 ## C++ API for TensorFlow
 
 The runtime of TensorFlow is written in C++, and mostly, C++ is connected to TensorFlow through header files in `tensorflow/cc`. 
-The C++ API is still in experimental stages of development, and also the documentation is being updated, 
-meaning that it lacks of information and tutorial about how to use TensorFlow API.
+The C++ API is still in the experimental stages of development, and also the documentation is being updated, 
+meaning that it lacks information and a tutorial about how to use TensorFlow API.
 
 ## Content
 

compile_tensorflow_cpp.md

@@ -91,7 +91,14 @@ chmod +x bazel-3.7.2-installer-linux-x86_64.sh
 
 ### 1. Compile TensorFlow C++ shared library (with optimization)
 
-Download or clone github repo to your system:
+Download a tarball of TensorFlow (I strongly prefer v.2.7 to other versions), 
+e.g., from https://github.com/tensorflow/tensorflow/releases/tag/v2.7.4
+```bash
+wget https://github.com/tensorflow/tensorflow/archive/refs/tags/v2.7.4.tar.gz
+tar -xzvf v2.7.4.tar.gz
+cd tensorflow-2.7.4
+```
+or clone github repo to your system:
 ```bash
 git clone https://github.com/tensorflow/tensorflow
 cd tensorflow
@@ -165,11 +172,15 @@ Note:
 1. Building TensorFlow can consume a lot of memory. So I prefer a small number of CPUs (`--jobs`), e.g. 4 CPUs use `--jobs=4`.
 2. Limit RAM requested by bazel with `--local_ram_resources`. The value is either integer, .e.g., 2048 use `--local_ram_resources=2048` or % of total memory, e.g., 50% use `"HOST_RAM*.50"`.
 3. The whole process can take up to 1 hour.
-4. Add `-D_GLIBCXX_USE_CXX11_ABI=0` if you use GCC 5 or higher version.
-5. Flags for optimization: `--copt="-O3"`.
-6. Flasg for both AMD and Intel chips: `--copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --copt=-mfpmath=both`.
-7. Flags for Intel: `--copt=-mavx --copt=-mavx2`.
-8. Rebuild with `--config=monolithic` if you want to compile all TensorFlow C++ code into a single shared object.
+4. If you don't want Bazel creates cache files in your local space, add [`--output_user_root`](https://docs.bazel.build/versions/main/user-manual.html#flag--output_user_root) to change the directory where output and base files will be created, e.g., 
+   ```bash
+   bazel --output_user_root=/scratch/bazel/ build ...
+   ```
+5. Add `-D_GLIBCXX_USE_CXX11_ABI=0` if you use GCC 5 or higher version.
+6. Flags for optimization: `--copt="-O3"`.
+7. Flasg for both AMD and Intel chips: `--copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --copt=-mfpmath=both`.
+8. Flags for Intel: `--copt=-mavx --copt=-mavx2`.
+9. Rebuild with `--config=monolithic` if you want to compile all TensorFlow C++ code into a single shared object.
 
 **Optional 1:** Test
 ```bash

load_model_tensorflow_cpp.md

@@ -62,6 +62,20 @@ Status status = LoadSavedModel(session_options, run_options, export_dir, {kSaved
 if (!status.ok()) {
    std::cerr << "Failed: " << status;
 }
+
+auto sig_map = model_bundle.GetSignatures();
+auto model_def = sig_map.at("serving_default");
+
+for (auto const& p : sig_map) {
+        std::cout << "key: " << p.first.c_str() << std::endl;
+}
+for (auto const& p : model_def.inputs()) {
+        std::cout << "key: " << p.first.c_str() << " " << p.second.name().c_str() << std::endl;
+}
+for (auto const& p : model_def.outputs()) {
+        std::cout << "key: " << p.first.c_str() << " " << p.second.name().c_str() << std::endl;
+}
+
    return 0;
 }
 ```

tensorflow_serving.md

@@ -3,19 +3,25 @@
 ## SavedModel
 
 **Save your (keras/tf) model:**
-```
+```python
 model.compile(...)
 model.fit(...)
 model.save("/path/to/your/model/folder/")
 ```
 
-**Get model input:**
+of you can use [`tf.saved_model.save`](https://www.tensorflow.org/api_docs/python/tf/saved_model/save) method to save object in SavedModel format at low-level:
+
+```python
+tf.saved_model.save(model, export_dir)
 ```
+
+**Get model input:**
+```bash
 $ saved_model_cli show --tag_set serve --signature_def serving_default --dir model/
 ```
 
 output:
-```
+```bash
 The given SavedModel SignatureDef contains the following input(s):
   inputs['dense_input'] tensor_info:
       dtype: DT_FLOAT
@@ -29,12 +35,50 @@ The given SavedModel SignatureDef contains the following output(s):
 Method name is: tensorflow/serving/predict
 ```
 
-The name of input and output tensors are `serving_default_dense_input:0` and `StatefulPartitionedCall:0`, respectively
+The name of input and output tensors are `serving_default_dense_input` and `StatefulPartitionedCall`, respectively
+
+You can also use `--all` argument to get all information, like this:
+
+```bash
+MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
+
+signature_def['__saved_model_init_op']:
+  The given SavedModel SignatureDef contains the following input(s):
+  The given SavedModel SignatureDef contains the following output(s):
+    outputs['__saved_model_init_op'] tensor_info:
+        dtype: DT_INVALID
+        shape: unknown_rank
+        name: NoOp
+  Method name is: 
+
+signature_def['serving_default']:
+  The given SavedModel SignatureDef contains the following input(s):
+    inputs['args_0'] tensor_info:
+        dtype: DT_FLOAT
+        shape: (-1, 394)
+        name: serving_default_args_0:0
+    inputs['args_0_1'] tensor_info:
+        dtype: DT_FLOAT
+        shape: (-1, 99)
+        name: serving_default_args_0_1:0
+  The given SavedModel SignatureDef contains the following output(s):
+    outputs['out_1'] tensor_info:
+        dtype: DT_FLOAT
+        shape: (-1, 394)
+        name: StatefulPartitionedCall:0
+    outputs['out_2'] tensor_info:
+        dtype: DT_FLOAT
+        shape: (-1, 99)
+        name: StatefulPartitionedCall:1
+  Method name is: tensorflow/serving/predict
+
+...
+```
 
 ## Run model
 
 **Load model**
-```
+```cpp
 const std::string export_dir = "/path/to/your/model/folder/";
 
 tensorflow::SavedModelBundle model_bundle;