Merge pull request #13 from ygorelik/0.8.6.2

0.8.6.2

Author: ygorelik

CHANGES.md

@@ -1,3 +1,24 @@
+### 2022-03-31 version 0.8.6.2
+
+#### New features and enhancements
+  * Improved installation script to allow use of Python system installation
+  * Corrected few Codacy issues
+  * Added support for MacOS Big Sur
+
+#### Resolved GitHub issues
+  * Fixed Python 3.5 code compatibility issue for Ubuntu:xenial
+  * install_ydk.sh tries to create a new venv ([#1034](https://github.com/CiscoDevNet/ydk-gen/issues/1034))
+  * getting YCodecError when leaf value has string 'sftp://' ([#1056](https://github.com/CiscoDevNet/ydk-gen/issues/1056))
+  * vrf route-target leaf stitching(bool) fails as invalid ([#1059](https://github.com/CiscoDevNet/ydk-gen/issues/1059))
+  
+### Resolved non-GitHub issues
+  * Addressed segmentation fault error when required modules are not loaded.
+  * Patched catch.hpp for Linux distributions that do not define SIGSTKSZ
+
+#### Bundle improvements
+  * Cleaned up older profiles
+
+
 ### 2021-09-30 version 0.8.6
 
 #### New features and enhancements
@@ -8,6 +29,8 @@
 
 #### Resolved GitHub issues
   * Go function EntityEqual result depends on parameters order ([#1053](https://github.com/CiscoDevNet/ydk-gen/issues/1053))
+  * Failed to get "ietf-netconf" module with Executor Service and get-schema RPC ([#1047](https://github.com/CiscoDevNet/ydk-gen/issues/1047))
+  * The Executor Service fails to return module content back to user ([#1048](https://github.com/CiscoDevNet/ydk-gen/issues/1048))
 
 #### Bundle improvements
   * Updated cisco-ios-xr bundle (profile cisco-ios-xr_6_7_3.json) to support Cisco IOS XR 6.7.3
@@ -399,7 +422,7 @@ The [backward incompatible](http://ydk.cisco.com/py/docs/backward_compatibility.
  * Add support for choosing either a per-device or a common cache for storing downloaded yang models ([#502](https://github.com/CiscoDevNet/ydk-gen/pull/502))
  * Introduced encoding/decoding subtree XML in `CodecService` and changed `CRUDService` to use XML subtree filtering to create filters for the `read` operation ([#489](https://github.com/CiscoDevNet/ydk-gen/pull/489))
  * Added support for non-standard RPCs as well in path API ([#498](https://github.com/CiscoDevNet/ydk-gen/pull/498))
- * Add ability to generate Linux and Mac OSX packages in cmake using [cpack](https://cmake.org/Wiki/CMake:Packaging_With_CPack) ([#466](https://github.com/CiscoDevNet/ydk-gen/pull/466))
+ * Add ability to generate Linux and MacOS packages in cmake using [cpack](https://cmake.org/Wiki/CMake:Packaging_With_CPack) ([#466](https://github.com/CiscoDevNet/ydk-gen/pull/466))
 
 #### ydk-gen
  * Updated [`cisco-ios-xr`](https://github.com/CiscoDevNet/ydk-gen/blob/master/profiles/bundles/cisco-ios-xr_6_2_2.json) to support Cisco IOS XR 6.2.2 release
@@ -473,7 +496,7 @@ The [backward incompatible](http://ydk.cisco.com/py/docs/backward_compatibility.
   * RestconfServiceProvider
   * OpenDaylightServiceProvider
 * Introduced support for netconf service ([#341](https://github.com/CiscoDevNet/ydk-gen/pull/341), [#352](https://github.com/CiscoDevNet/ydk-gen/pull/352))
-* Released ydk-cpp for OSX platform (on [Homebrew](https://github.com/CiscoDevNet/homebrew-ydk)) and on Ubuntu platform (on [Lauchpad](https://launchpad.net/~ydk)) ([#362](https://github.com/CiscoDevNet/ydk-gen/pull/362), [#322](https://github.com/CiscoDevNet/ydk-gen/pull/322))
+* Released ydk-cpp for MacOS platform (on [Homebrew](https://github.com/CiscoDevNet/homebrew-ydk)) and on Ubuntu platform (on [Lauchpad](https://launchpad.net/~ydk)) ([#362](https://github.com/CiscoDevNet/ydk-gen/pull/362), [#322](https://github.com/CiscoDevNet/ydk-gen/pull/322))
 * Added support for generated CRUD model tests based on bundles ([#354](https://github.com/CiscoDevNet/ydk-gen/pull/354))
 * Improved negative test cases and added support for netconf operations on leafs and leaf-lists ([#324](https://github.com/CiscoDevNet/ydk-gen/pull/324))
 

README.md

@@ -49,12 +49,11 @@ YANG Development Kit
   - [Building from source](#building-from-source)
   - [Adding gNMI service](#adding-gnmi-service)
 - [Generate YDK Components](#generate-ydk-components)
-  - [Generate deployment packages](#generate-deployment-packages)
   - [Build model bundle profile](#build-model-bundle-profile)
   - [Generate and install model bundle](#generate-and-install-model-bundle)
   - [Writing your first app](#writing-your-first-app)
-  - [Documentation](#documentation)
-- [Generating an "Adhoc" YDK-Py Bundle](#generating-an-adhoc-ydk-py-bundle)
+  - [Generating Documentation](#generating-documentation)
+  - [Generating an "Adhoc" YDK-Py Bundle](#generating-an-adhoc-ydk-py-bundle)
 - [Notes](#notes)
   - [Python requirements](#python-requirements)
   - [C++ requirements](#c-requirements)
@@ -120,7 +119,7 @@ docker run -it ydksolutions/ydk-gen
 The YDK is currently supported on the following platforms including native installations, virtual machines, and docker images:
  - Linux Ubuntu Xenial (16.04 LTS), Bionic (18.04 LTS), and Focal (20.04 LTS)
  - Linux CentOS/RHEL versions 7 and 8
- - MacOS up to 10.14.6 (Mojave)
+ - MacOS up to 11.6.2 (Big Sur)
 
 On Windows 10 the Linux virtual machine can run using Windows Subsystem for Linux (WSL);
 check [this](https://www.windowscentral.com/install-windows-subsystem-linux-windows-10) for virtual machine installation procedure.
@@ -133,7 +132,7 @@ For both the methods the user must install `git` package prior to the installati
 
 All YDK core components are based on C and C++ code. These components compiled using default compilers for the supported platform.
 Corresponding binaries, libraries, and header files are installed in default locations,
-which are `/usr/local/bin` ,`/usr/local/lib`, and `/usr/local/include`.
+which are `/usr/local/bin`, `/usr/local/lib`, and `/usr/local/include`.
 The user must have sudo access in order to install YDK core components to these locations.
 
 # Core Installation
@@ -142,11 +141,11 @@ The user must have sudo access in order to install YDK core components to these
 
 For YDK installation it is recommended to use script `install_ydk.sh` from `ydk-gen` git repository.
 The script detects platform OS, installs all the dependencies and builds complete set of YDK components for specified language.
-The user must have sudo access to these locations.
 
 The YDK extensively uses Python scripts for building its components and model API packages (bundles).
-In order to isolate YDK Python environment from system installation, the script builds Python virtual environment.
-The user must manually activate virtual environment when generating model bundles and/or running YDK based application.
+By default the YDK uses Python system installation.
+In order to isolate YDK Python environment from system installation, the script can build Python3 virtual environment.
+If built, the user must manually activate virtual environment when generating model bundles and/or running YDK based application.
 By default the Python virtual environment is installed under `$HOME/venv` directory.
 For different location the PYTHON_VENV environment variable should be set to that location.
 
@@ -166,11 +165,14 @@ Full set of script capabilities could be viewed like this:
 
 ```
 ./install_ydk.sh --help
-usage: install_ydk [-l [cpp, py, go]] [-s gnmi] [-h] [-n]
+usage: install_ydk [ {--cpp|--py|--go|--all} ] [-c] [-s gnmi] [-h] [-n] [-v]
 Options and arguments:
-  -l [cpp, py, go, all] installation language; if not specified Python is assumed
-                        'all' corresponds to all available languages
-  -c|--core             install YDK core package
+  --cpp                 install YDK for C++ programming language
+  --go                  install YDK for Go programming language
+  --py|--python         install YDK for Python programming language (default)
+  --all                 install YDK for all supported programming languages
+  -v|--venv             create python virtual environment
+  -c|--core             install YDK core packages
   -s|--service gnmi     install gNMI service package
   -n|--no-deps          skip installation of dependencies
   -h|--help             print this help message and exit
@@ -188,17 +190,33 @@ C_INCLUDE_PATH      location of C include files;
                     if not set, /usr/local/include is assumed
 CPLUS_INCLUDE_PATH  location of C++ include files;
                     if not set, /usr/local/include is assumed
+CMAKE_LIBRARY_PATH  Location of Python shared libraries;
+                    if not set, default system library location is assumed
 ```
 
 If user environment is different from the default one (different Python installation or different
 location of libraries) then building from source method should be used.
 
 ## Building from source
 
+### Environment variables
+
+In some OS configurations during YDK package installation the cmake fails to find C/C++ headers for previously installed YDK libraries.
+In this case the header files location must be specified explicitly (in below commands the default location is shown):
+
+```
+  export C_INCLUDE_PATH=/usr/local/include
+  export CPLUS_INCLUDE_PATH=/usr/local/include
+```
+
+When non-standard Python installation is used or there are multiple installations of Python on the platform,
+the PATH and CMAKE_LIBRARY_PATH environment variables must be set accordingly in order for the installation scripts
+to pick up correct Python binaries and shared libraries.
+
 ### Installing third party dependencies
 
 If user platform is supported one, it is recommended to use `ydk-gen/install_ydk.sh` script. 
-The script will also install Python virtual environment in default or specified location.
+The script will also install Python virtual environment in default or specified location, when '--venv' is specified.
 
 ```
 # Clone ydk-gen from GitHub
@@ -208,30 +226,22 @@ cd ydk-gen
 # Define optional environment variables and install dependencies
 export YDKGEN_HOME=`pwd`  
 export PYTHON_VENV=$HOME/ydk_venv
-./install_ydk.sh   # also builds Python virtual environment
+./install_ydk.sh -v   # also builds Python virtual environment
 ```
 
 For unsupported platforms it is recommended to follow logic of `ydk-gen/test/dependencies-*` scripts.
 
-### Environment variables
-
-In some OS configurations during YDK package installation the cmake fails to find C/C++ headers for previously installed YDK libraries.
-In this case the header location must be specified explicitly (in below commands the default location is shown)::
-
-  export C_INCLUDE_PATH=/usr/local/include
-  export CPLUS_INCLUDE_PATH=/usr/local/include
-
 ### Installing core components
 
 ```
-# Activate Python virtual environment
+# If created, activate Python virtual environment
 source $PYTHON_VENV/bin/activate
 
 # Generate and install YDK core library
 ./generate.py -is --core --cpp
 
 # For Python programming language add
-./generate.py -i --core 
+./generate.py -i --core --py
 
 # For Go programming language add
 ./generate.py -i --core --go
@@ -248,7 +258,7 @@ Here is simple example how gNMI service package for Python could be added:
 
 ```
 cd ydk-gen
-./install_ydk.sh -l py --service gnmi
+./install_ydk.sh --py --service gnmi -v
 ```
 
 ### Runtime environment
@@ -316,28 +326,6 @@ and [ydk-cpp](https://github.com/CiscoDevNet/ydk-cpp).
 The script [create_ydk_sdk_for_github.sh](create_ydk_sdk_for_github.sh) can be used to generate the `ydk-py`, 
 `ydk-cpp` and `ydk-go` repositories after having generated all the bundles and core packages using `generate.py`.
 
-## Generate deployment packages
-
-First the core installation procedure must be executed. Refer to [Core Installation](#core-installation) section for details.
-
-To create the `libydk` binary package run the below commands:
-
-```
-./generate.py --libydk
-cd gen-api/cpp/ydk/build
-make
-[sudo] make package
-```
-
-To create the `libydk_gnmi` binary package run the below commands:
-
-```
-./generate.py --cpp --service profiles/services/gnmi-0.4.0.json
-cd gen-api/cpp/ydk-service-gnmi/build
-make
-[sudo] make package
-```
-
 ## Build model bundle profile
 
 The first step in using ydk-gen is either using one of the already built 
@@ -456,24 +444,23 @@ Also refer to the [documentation for python](http://ydk.cisco.com/py/docs/develo
 [Go](http://ydk.cisco.com/go/docs/developer_guide.html) and
 [C++](http://ydk.cisco.com/cpp/docs/developer_guide.html).
 
-## Documentation
+## Generating Documentation
 
 In order to generate YDK core and bundles documentation, the `--generate-doc` option of `generate.py` script is used, 
 while generating core package. Therefore the user should generate all the bundles without the `--generate-doc` 
 option prior to the documentation generation.
 For example, the below sequence of commands will generate the documentation for the three python bundles and the python core
 (for C++, use `--cpp`; for Go, use `--go`).
 
-**Note.** The documentation generation for bundles can take few hours due to their sizes.
-
 ```
 ./generate.py --python --bundle profiles/bundles/ietf_0_1_1.json
 ./generate.py --python --bundle profiles/bundles/openconfig_0_1_1.json
 ./generate.py --python --bundle profiles/bundles/cisco_ios_xr_6_1_1.json
 ./generate.py --python --core --generate-doc
 ```
 
-If you have previously generated documentation using the `--cached-output-dir --output-directory <dir>` option,
+**Note.** The documentation generation for bundles can take few hours due to their size. If you have previously 
+generated documentation using the `--cached-output-dir --output-directory <dir>` option,
 the add-on documentation generation time can be reduced. Adding cisco-ios-xr documentation as an example:
 
 ```
@@ -484,17 +471,14 @@ mv gen-api/python gen-api/cache
 ./generate.py --python --core --generate-doc --output-directory gen-api --cached-output-dir
 ```
 
-Pre-generated documentation is available on the web for [Python](http://ydk.cisco.com/py/docs/), 
-[C++](http://ydk.cisco.com/cpp/docs/) and [Go](http://ydk.cisco.com/go/docs/).
-
-# Generating an "Adhoc" YDK-Py Bundle
+## Generating an "Adhoc" YDK-Py Bundle
 
 When YANG models available on the hard drive, there is capability to generate small model bundles, which include
 just few models. It is called an "adhoc" bundle. Such a bundle generated without profile directly from command line.
 Here is simple example:
 
 ```
-./generate.py --adhoc-bundle-name test --adhoc-bundle \
+./generate.py -i --adhoc-bundle-name test --adhoc-bundle \
     /opt/git-repos/clean-yang/vendor/cisco/xr/621/Cisco-IOS-XR-ipv4-bgp-oper*.yang \
     /opt/git-repos/clean-yang/vendor/cisco/xr/621/Cisco-IOS-XR-types.yang
     /opt/git-repos/clean-yang/vendor/cisco/xr/621/Cisco-IOS-XR-ipv4-bgp-datatypes.yang
@@ -510,13 +494,15 @@ and easily let the user see if dependencies have been satisfied.
 
 ## Python requirements
 
-Starting from release 0.8.6 YDK supports only Python3 version.  At least Python 3.5 along with corresponding pip3 utility 
-must be installed on your the system. It is also required for Python installation to include corresponding shared 
+Starting from release 0.8.5 YDK supports only Python3 version.  At least Python 3.5 along with corresponding pip3 utility 
+must be installed on your system. It is also required for Python installation to include corresponding shared 
 library. As example:
 
- - python3.5m - /usr/lib/x86_64-linux-gnu/libpython3.5m.so
+ - python3.6m - /usr/lib/x86_64-linux-gnu/libpython3.6m.so
+
+It is recommended to follow [Core Installation](#core-installation) procedure to ensure presence of shared Python libraries.
 
-Please follow [Core Installation](#core-installation) procedures to assure presence of shared Python libraries.
+**NOTE.** Due to GitHub issue [#1050](https://github.com/CiscoDevNet/ydk-gen/issues/1050) YDK is not supported with Python 3.9.x.
 
 ## C++ requirements
 
@@ -549,7 +535,7 @@ generate_bundles.sh          - convinience script to generate core and bundle pa
 generate.py     - script used to generate SDK for YANG models
 install_ydk.sh  - YDK core components installation script
 requirements.txt- python dependencies used during installation
-README          - installation and usage notes
+README.md       - installation and usage notes
 ```
 
 ## Troubleshooting
@@ -564,7 +550,7 @@ Other times, when the problem is not so evident, it is recommended to try runnin
 ```
 
 Also, it may be a good idea to obtain a local copy of the YANG models and compile them using `pyang` to ensure
-the validity of the models:
+the validity of the models and completeness of the model package:
 
 ```
 cd /path/to/yang/models
@@ -654,4 +640,5 @@ Please refer [here](https://github.com/ygorelik/ydk-gen/blob/master/sdk/go/core/
 # Release Notes
 
 The current YDK release version is 0.8.6.
+
 YDK-Gen is licensed under the Apache 2.0 License.