SOVERSION does not indicate binary compatibility accurately

[mgorny]

Currently, SOVERSION on the shared library is set to match two version components:

cpp-httplib/CMakeLists.txt

Line 196 in 996acc5

SOVERSION "${${PROJECT_NAME}_VERSION_MAJOR}.${${PROJECT_NAME}_VERSION_MINOR}"

This indicates that all 0.18.x releases should be backwards compatible. However, this is not the case. The most recent example is #2022 adding is_connection_closed field in middle of Request class. This caused all field offsets to change, causing software linked to the previous version of the library to start accessing wrong fields.

Please either update SOVERSION to correctly indicate binary compatible, or — even better — set stronger binary compatibility guarantees. This library is used by some huge stuff (for example, LLVM) and people need to spend hours rebuilding it whenever ABI of cpp-httplib changes.

[yhirose]

@mgorny thanks for the report and sorry for causing the inconvenience.

I have been just focusing on the source level and behavior compatibility. Whenever I change an existing public method signature or remove an existing public method, or making a behavior change of it, I bump up the minor version. (I don't bump the version when I just add a new public method.)

I do refactoring the code base once in a while, and it doesn't cause any behavior change and public method signature change, but it may add/remove private members (and private methods). In this case, I personally don't want to bump up the minor version. But I am ok to change my version update strategy if the current way causes problems in many users. Please explain your expectations in more detail. Thanks!

[mgorny]

Well, my first suggestion would be to look at abi-compliance-checker tool. If you build two versions of the shared library with debug symbols, you can use it like this:

abi-dumper -lver 0.18.5 path/to/libcpp-httplib.so.0.18.5 -o old.dump
abi-dumper -lver 0.18.6 path/to/libcpp-httplib.so.0.18.6 -o new.dump
abi-compliance-checker -l cpp-httplib -old old.dump -new new.dump

…and it will generate a nice HTML report that tells if you anything ABI-breaking happened, and explain why it breaks ABI. Long story short, if it points out any backwards compatibility breakage, you need to raise library SOVERSION.

Of course, anytime that happens, it means that all consumers linking to the shared library need to rebuilt. This is especially problematic for LLVM since it takes a few hours to build, and an average modern Gentoo system needs 2-3 versions of LLVM simultaneously, so basically a cpp-httplib ABI change means whole day spent rebuilding LLVM.

Unfortunately, changes to private fields of public classes also count as ABI changes. There are some ways to avoid that, though. If the class is never allocated by the caller, you could add new fields to the very end — in that case ABI is preserved. If the class is allocated by the caller, you can still avoid ABI changes by using the PImpl idiom.

[yhirose]

@mgorny could you explain more about the "shared library" such as 'libcpp-httplib.so.0.18.5'? (I am not building and maintaining such binary static library files, since cpp-httplib is "A C++11 single-file header-only cross platform HTTP/HTTPS library". In all my projects, I simply drop httplib.h into my project folder and include it in the .cpp files. 😄)

[mgorny]

It's enabled by -DHTTPLIB_COMPILE=ON -DBUILD_SHARED_LIBS=ON, and apparently people are relying on it.

[yhirose]

@mgorny thanks for the info. In fact, I am actually not maintaining CMake related files (and meson) at all, but I just accept such contributions as they are and ask them to maintain. That's why the README doesn't mention anything about such build systems. So if it creates such extra work every time I bump the version, it's too much to me to be honest...

@sum01 @abouvier @jimmy-park @Tachi107 as maintainers for CMake and meson, do you have any idea to check the binary compatibility easily on my side when releasing a new version with out any manual work?

[yhirose]

@mgorny another idea. You can simply use the original single header version of httplib.h in this repo.

[mgorny]

LLVM is not my project. I'm merely downstream for it.

I can imagine them not wanting to use the header directly, since HTTP is a dangerous business and vendoring cpp-httplib is a responsibility. In fact, that would also explain why they prefer a shared library — it gives at least some chance of upgrading without having to rebuild everything.

That said, vendoring the header doesn't buy us anything. It just means that with every version we need to spend the whole day rebuilding everything. Which is probably the case anyway. On top of that, since old LLVM releases are abandoned roughly half a year after a release, but are required by other projects for years, this would mean that I would end up having to update vendored cpp-httplib there.

[Tachi107]

Hi all.

One thing we do in a project I maintain is having a CI job which builds the project and runs abidiff on it. This way, if we introduce ABI-breaking changes, we can notice it and fix it promptly.

Making cpp-httplib's ABI more stable would benefit lots of people which prefer using the compiled variant of the library, and I'd be glad to help some way. It also interests me personally as handling ABI changes in Debian is always a bit of an involved process.

[sum01]

@yhirose I don't think it's really something to solve in Cmake. The compatibility is already set to SameMinorVersion there. Otherwise all you could do is set it to ExactVersion and set the patch version on the solib too. But that's just telling the user "any and all version bumps should be considered breaking".

@Tachi107's recommendation sounds good for a way to automate it. Haven't used it myself though.

[yhirose]

@mgorny I tested with v0.18.4 and v0.18.7 on Ubuntu 22.04. Unfortunately, abi-compliance-checker doesn't detect the ABI change. It seems like the tool doesn't work as advertised... Do you have any clue? Or is there a easier way to detect the change?

#!/usr/bin/env bash
set -e
set -x

rm -rf build
mkdir -p build/out
cd build

OLD_VER=0.18.4
NEW_VER=0.18.7
BUILD_TYPE=Debug

cmake \
  -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
  -DCMAKE_CXX_FLAGS=-Og \
  -DBUILD_SHARED_LIBS=ON \
  -DHTTPLIB_COMPILE=ON \
  -DCMAKE_INSTALL_PREFIX=./out \
  ..

git checkout "v0.18.4"
cmake --build . --target install

git checkout "v0.18.7"
cmake --build . --target install

tree out/

abi-dumper -lver ${OLD_VER} out/lib/libcpp-httplib.so.${OLD_VER} -o ${OLD_VER}.dump
abi-dumper -lver ${NEW_VER} out/lib/libcpp-httplib.so.${NEW_VER} -o ${NEW_VER}.dump

abi-compliance-checker -l cpp-httplib -old ${OLD_VER}.dump -new ${NEW_VER}.dump

Preparing, please wait ...
Comparing ABIs ...
Comparing APIs ...
Creating compatibility report ...
Binary compatibility: 100%
Source compatibility: 100%
Total binary compatibility problems: 0, warnings: 0
Total source compatibility problems: 0, warnings: 0
Report: compat_reports/cpp-httplib/0.18.4_to_0.18.7/compat_report.html

[mgorny]

That's weird, because it worked for me with your script:

Preparing, please wait ...
Comparing ABIs ...
Comparing APIs ...
Creating compatibility report ...
Binary compatibility: 97.7%
Source compatibility: 100%
Total binary compatibility problems: 2, warnings: 1
Total source compatibility problems: 0, warnings: 1
Report: compat_reports/cpp-httplib/0.18.4_to_0.18.7/compat_report.html

I only need to adjust the path to use lib64/ since this is what CMake uses on Gentoo amd64.

[falbrechtskirchinger]

@yhirose Since you release quite frequently for bug fixes, have you considered moving major work to a development branch? That way, ABI-breaking changes could sit there for a while, before you release and bump the minor version. Bug fixes could still go into master directly.

[yhirose]

@mgorny I just wrote a simple shell script which tries to detect ABI changes (remove functions, add/remove variables), and I confirmed that it works on both macOS and Ubuntu. As you can see in the script, I only use output from nm command, but it seems it's enough to detect such ABI changes. Here is the usage.

Usage: check-abi-compatibility.sh old_library.so new_library.so

Could you try it on your environment when you have time? Thanks a lot!

[mgorny]

Could you try it on your environment when you have time?

Well, it does detect something but I'm not sure whether it's the thing it should detect (change in Request class) or some false positive.

[DarthGandalf]

I've just tried to check this script on a simple change:

 struct MultipartFormData {
-  std::string name;
   std::string content;
+  std::string name;
   std::string filename;
   std::string content_type;
 };

This is breaking ABI.
But the result is ABI compatibility check passed: No variable symbols were removed or added, and no function symbols were removed.

[Tachi107]

abidiff exists already, and its likely way more reliable than a shell script using nm :) <https://sourceware.org/libabigail/manual/abidiff.html> Il 12 febbraio 2025 00:54:55 CET, yhirose ***@***.***> ha scritto:

…

yhirose left a comment (yhirose/cpp-httplib#2043) @mgorny I just wrote [a simple shell script](https://github.com/yhirose/cpp-httplib/blob/master/test/check-abi-compatibility.sh) which tries to detect ABI changes (remove functions, add/remove variables), and I confirmed that it works on both macOS and Ubuntu. As you can see in the script, I only use output from `nm` command, but it seems it's enough to detect such ABI changes. Here is the usage. ```bash Usage: check-abi-compatibility.sh old_library.so new_library.so ``` Could you try it on your environment when you have time? Thanks a lot!

[yhirose]

abidiff exists already, and its likely way more reliable than a shell script using nm :)

Thanks for the information, but I don't find it in apt or homebrew. Does it meant that it should be built from source? If so, it's painful... Do you know an easy way to try it out?

[eli-schwartz]

Thanks for the information, but I don't find it in apt or homebrew. Does it meant that it should be built from source? If so, it's painful... Do you know an easy way to try it out?

It is part of https://packages.debian.org/bookworm/abigail-tools and https://formulae.brew.sh/formula/libabigail

In both cases the software name is "libabigail", Debian splits it into "libabigail-dev" and "abigail-tools" but that is a debianism for confusing names. If you want to find software on Debian, do NOT try to look for a package named the same as the thing you are trying to find... use the web page at https://www.debian.org/distrib/packages and fill out the form under "Search the contents of packages". Or use https://wiki.debian.org/apt-file

[Tachi107]

I know this is a bit off topic, but I can't resist defending my little precious Debian :)

In both cases the software name is "libabigail", Debian splits it into "libabigail-dev" and "abigail-tools" but that is a debianism for confusing names.

Not exactly, "libabigail" a library part of the Abigail framework: <https://sourceware.org/libabigail/manual/libabigail-overview.html>. The framework also provides some _tools_ based on libabigail, hence "abigail-tools" makes sense as a name if you want to split things in different packages (like Fedora does as well).

If you want to find software on Debian, do NOT try to look for a package named the same as the thing you are trying to find... use the web page at https://www.debian.org/distrib/packages and fill out the form under "Search the contents of packages". Or use https://wiki.debian.org/apt-file

Or just use `apt search`: $ apt search abigail abigail-doc/testing,unstable 2.6-2 all ABI Generic Analysis and Instrumentation Library (documentation) abigail-tools/testing,unstable 2.6-2 amd64 ABI Generic Analysis and Instrumentation Library (tools) libabigail-dev/testing,unstable 2.6-2 amd64 ABI Generic Analysis and Instrumentation Library (development files) libabigail5/testing,unstable 2.6-2 amd64 ABI Generic Analysis and Instrumentation Library (runtime)

[eli-schwartz]

The framework also provides some tools based on libabigail, hence "abigail-tools" makes sense as a name if you want to split things in different packages (like Fedora does as well).

Proud user of multiple distros that don't split things in different packages, for many many years now. :P

Or just use apt search:

And why should I assume that the search term to use is "abigail" rather than "libabigail", if the project name is "libabigail"? Very easy to accidentally miss the package providing the tools.

After all, it is https://sourceware.org/libabigail/ which tells you how to find the "source code of Libabigail" and "hang out with libabigail developers and users" on IRC, and where to find the "documentations of the libabigail tools", and basic instructions on how "To compile libabigail" (including its tools).

And even the "Libabigail manual", describes the "abigail framework" / "libabigail library" / "abidiff program".

If apt search were to e.g. return results for string matches on the source package name too, that could be solved...

[yhirose]

I've just tried to check this script on a simple change:

struct MultipartFormData {

• std::string name;
  std::string content;

• std::string name;
  std::string filename;
  std::string content_type;
  };
  This is breaking ABI. But the result is ABI compatibility check passed: No variable symbols were removed or added, and no function symbols were removed.

@DarthGandalf, Thanks for the report! Could you try with abi-compliance-checker that @mgorny recommend? (Somehow, the tool doesn't work on my Ubuntu and macOS installed with apt and homebrew. So I am not able to test it...)

[yhirose]

@Tachi107 thanks for the great job at #2054, also thanks all for valuable comments and suggestions! 😄