Build process failed due to a COMPILER BUG DETECTED #465

coolbeevip · 2025-01-04T08:08:20Z

coolbeevip
Jan 4, 2025

The error message and related version information are as follows. Do I need to upgrade the gcc version?

cargo install --path router -F candle-cuda -F http --no-default-features
  Installing text-embeddings-router v1.6.0 (/work/text-embeddings-inference/router)
    Updating git repository `https://github.com/OlivierDehaene/candle`
    Updating git repository `https://github.com/coreylowman/cudarc`
    Updating crates.io index
    Updating git repository `https://github.com/huggingface/candle-cublaslt`
    Updating git repository `https://github.com/huggingface/candle-layer-norm`
    Updating git repository `https://github.com/huggingface/candle-rotary`
   Compiling backtrace v0.3.74
   Compiling regex v1.11.1
   Compiling fancy-regex v0.13.0
   Compiling serde_derive v1.0.217
   Compiling synstructure v0.13.1
   Compiling zerocopy-derive v0.7.35
   Compiling futures-macro v0.3.31
   Compiling tokio-macros v2.4.0
   Compiling tracing-attributes v0.1.28
   Compiling thiserror-impl v1.0.69
   Compiling zerovec-derive v0.10.3
   Compiling displaydoc v0.2.5
   Compiling icu_provider_macros v1.5.0
   Compiling bytemuck_derive v1.8.1
   Compiling async-trait v0.1.84
   Compiling openssl-macros v0.1.1
   Compiling pin-project-internal v1.1.7
   Compiling async-stream-impl v0.3.6
   Compiling darling_core v0.20.10
   Compiling num_enum_derive v0.7.3
   Compiling clap_derive v4.5.18
   Compiling monostate-impl v0.1.13
   Compiling veil-macros v0.1.7
   Compiling utoipa-gen v4.3.1
   Compiling bindgen_cuda v0.1.5
   Compiling aws-lc-sys v0.24.1
   Compiling ring v0.17.8
   Compiling onig_sys v69.8.1
   Compiling rayon-cond v0.3.0
   Compiling esaxx-rs v0.1.10
   Compiling simsimd v4.4.0
   Compiling rust-embed-impl v8.5.0
   Compiling onig v6.4.0
   Compiling anyhow v1.0.95
The following warnings were emitted during compilation:

warning: aws-lc-sys@0.24.1: Building with: CC
warning: aws-lc-sys@0.24.1: Symbol Prefix: Some("aws_lc_0_24_1")
warning: aws-lc-sys@0.24.1: Compilation of 'c11.c' succeeded - Ok(["/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out/out-c11/7dfda64fdf5a526c-c11.o"]).
warning: aws-lc-sys@0.24.1: Using flag: -ffile-prefix-map=/home/cb/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.1=
warning: aws-lc-sys@0.24.1: Compilation of 'stdalign_check.c' succeeded - Ok(["/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out/out-stdalign_check/7dfda64fdf5a526c-stdalign_check.o"]).
warning: aws-lc-sys@0.24.1: Compilation of 'builtin_swap_check.c' succeeded - Ok(["/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out/out-builtin_swap_check/7dfda64fdf5a526c-builtin_swap_check.o"]).

error: failed to run custom build command for `aws-lc-sys v0.24.1`

Caused by:
  process didn't exit successfully: `/work/text-embeddings-inference/target/release/build/aws-lc-sys-d3b79c524310840d/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
  cargo:rerun-if-env-changed=AWS_LC_SYS_PREGENERATING_BINDINGS
  cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
  cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
  cargo:rerun-if-env-changed=AWS_LC_SYS_CFLAGS
  cargo:rerun-if-env-changed=AWS_LC_SYS_PREBUILT_NASM
  cargo:rerun-if-env-changed=AWS_LC_SYS_C_STD
  cargo:rustc-cfg=x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
  cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
  default_for Target: 'x86_64-unknown-linux-gnu'
  cargo:rerun-if-env-changed=CARGO_FEATURE_SSL
  cargo:warning=Building with: CC
  cargo:warning=Symbol Prefix: Some("aws_lc_0_24_1")
  default_for Target: 'x86_64-unknown-linux-gnu'
  cargo:rerun-if-env-changed=CARGO_FEATURE_SSL
  cargo:root=/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out
  default_for Target: 'x86_64-unknown-linux-gnu'
  OPT_LEVEL = Some(3)
  TARGET = Some(x86_64-unknown-linux-gnu)
  HOST = Some(x86_64-unknown-linux-gnu)
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  RUSTC_WRAPPER = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some(false)
  CARGO_CFG_TARGET_FEATURE = Some(adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,pclmulqdq,popcnt,rdrand,rdseed,sha,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves)
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  CARGO_ENCODED_RUSTFLAGS = Some(-Ctarget-cpu=native)
  OPT_LEVEL = Some(3)
  TARGET = Some(x86_64-unknown-linux-gnu)
  HOST = Some(x86_64-unknown-linux-gnu)
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  RUSTC_WRAPPER = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some(false)
  CARGO_CFG_TARGET_FEATURE = Some(adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,pclmulqdq,popcnt,rdrand,rdseed,sha,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves)
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  CARGO_ENCODED_RUSTFLAGS = Some(-Ctarget-cpu=native)
  cargo:warning=Compilation of 'c11.c' succeeded - Ok(["/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out/out-c11/7dfda64fdf5a526c-c11.o"]).
  cargo:rerun-if-env-changed=CC
  cargo:rerun-if-env-changed=CXX
  OUT_DIR = Some(/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out)
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  TARGET = Some(x86_64-unknown-linux-gnu)
  CARGO_CFG_TARGET_FEATURE = Some(adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,pclmulqdq,popcnt,rdrand,rdseed,sha,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves)
  HOST = Some(x86_64-unknown-linux-gnu)
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:warning=Using flag: -ffile-prefix-map=/home/cb/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.1=
  OPT_LEVEL = Some(3)
  TARGET = Some(x86_64-unknown-linux-gnu)
  HOST = Some(x86_64-unknown-linux-gnu)
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  RUSTC_WRAPPER = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some(false)
  CARGO_CFG_TARGET_FEATURE = Some(adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,pclmulqdq,popcnt,rdrand,rdseed,sha,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves)
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  CARGO_ENCODED_RUSTFLAGS = Some(-Ctarget-cpu=native)
  cargo:warning=Compilation of 'stdalign_check.c' succeeded - Ok(["/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out/out-stdalign_check/7dfda64fdf5a526c-stdalign_check.o"]).
  OPT_LEVEL = Some(3)
  TARGET = Some(x86_64-unknown-linux-gnu)
  HOST = Some(x86_64-unknown-linux-gnu)
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  RUSTC_WRAPPER = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some(false)
  CARGO_CFG_TARGET_FEATURE = Some(adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,pclmulqdq,popcnt,rdrand,rdseed,sha,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves)
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  CARGO_ENCODED_RUSTFLAGS = Some(-Ctarget-cpu=native)
  cargo:warning=Compilation of 'builtin_swap_check.c' succeeded - Ok(["/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out/out-builtin_swap_check/7dfda64fdf5a526c-builtin_swap_check.o"]).
  OPT_LEVEL = Some(3)
  OUT_DIR = Some(/work/text-embeddings-inference/target/release/build/aws-lc-sys-f97534cc628d42bc/out)
  TARGET = Some(x86_64-unknown-linux-gnu)
  HOST = Some(x86_64-unknown-linux-gnu)
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  RUSTC_WRAPPER = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some(false)
  CARGO_CFG_TARGET_FEATURE = Some(adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,pclmulqdq,popcnt,rdrand,rdseed,sha,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves)
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  CARGO_ENCODED_RUSTFLAGS = Some(-Ctarget-cpu=native)

  --- stderr
  thread 'main' panicked at /home/cb/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.1/builder/cc_builder.rs:355:13:
  ### COMPILER BUG DETECTED ###
  Your compiler (cc) is not supported due to a memcmp related bug reported in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189. We strongly recommend against using this compiler.
  EXECUTED: true
  ERROR:
  OUTPUT:

  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `text-embeddings-router v1.6.0 (/work/text-embeddings-inference/router)`, intermediate artifacts can be found at `/work/text-embeddings-inference/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

Driver Info

$ nvidia-smi
Sat Jan  4 15:53:12 2025
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A800-SXM4-80GB          Off | 00000000:3D:00.0 Off |                    0 |
| N/A   32C    P0              63W / 400W |  47846MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA A800-SXM4-80GB          Off | 00000000:42:00.0 Off |                    0 |
| N/A   29C    P0              66W / 400W |   6950MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   2  NVIDIA A800-SXM4-80GB          Off | 00000000:61:00.0 Off |                    0 |
| N/A   28C    P0              66W / 400W |  37006MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   3  NVIDIA A800-SXM4-80GB          Off | 00000000:67:00.0 Off |                    0 |
| N/A   32C    P0              61W / 400W |  34011MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   4  NVIDIA A800-SXM4-80GB          Off | 00000000:AD:00.0 Off |                    0 |
| N/A   32C    P0              65W / 400W |  35922MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   5  NVIDIA A800-SXM4-80GB          Off | 00000000:B1:00.0 Off |                    0 |
| N/A   27C    P0              61W / 400W |   3559MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   6  NVIDIA A800-SXM4-80GB          Off | 00000000:D0:00.0 Off |                    0 |
| N/A   26C    P0              54W / 400W |      5MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   7  NVIDIA A800-SXM4-80GB          Off | 00000000:D3:00.0 Off |                    0 |
| N/A   31C    P0              64W / 400W |  10116MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

GCC

$ gcc --version
gcc (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2)
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

CPU

$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                128
On-line CPU(s) list:   0-127
Thread(s) per core:    2
Core(s) per socket:    32
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 106
Model name:            Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz
Stepping:              6
CPU MHz:               800.000
CPU max MHz:           2601.0000
CPU min MHz:           800.0000
BogoMIPS:              5200.00
Virtualization:        VT-x
L1d cache:             48K
L1i cache:             32K
L2 cache:              1280K
L3 cache:              49152K
NUMA node0 CPU(s):     0-31,64-95
NUMA node1 CPU(s):     32-63,96-127
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 invpcid_single ssbd mba rsb_ctxsw ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq md_clear pconfig spec_ctrl intel_stibp flush_l1d arch_capabilities

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Build process failed due to a COMPILER BUG DETECTED #465

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Build process failed due to a COMPILER BUG DETECTED #465

Uh oh!

Uh oh!

coolbeevip Jan 4, 2025

Replies: 0 comments

coolbeevip
Jan 4, 2025