Unable to install psycopg

[frankiedrake]

I can install psycopg in my notebook instance when it's already started without any problems.
Now I wanted to create a lifecycle configuration so that all required dependencies are already installed when notebook instance is started.
Here's my script:

su ec2-user

source /home/ec2-user/anaconda3/etc/profile.d/conda.sh

conda activate pytorch_p310

pip install clean-text vllm unidecode "psycopg[c]" psycopg-binary fschat accelerate tenacity

But psycopg dependencies not installing with error

Collecting psycopg-c==3.1.17 (from psycopg[c])
  Downloading psycopg-c-3.1.17.tar.gz (561 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 561.4/561.4 kB 55.5 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      running dist_info
      creating /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info
      writing /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/top_level.txt
      writing manifest file '/tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/SOURCES.txt'
      couldn't run 'pg_config' --includedir: [Errno 2] No such file or directory: 'pg_config'
      error: [Errno 2] No such file or directory: 'pg_config'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

Full LifeCycleConfig logs look like:

Collecting clean-text Downloading clean_text-0.6.0-py3-none-any.whl (11 kB) Collecting vllm Downloading vllm-0.2.7-cp310-cp310-manylinux1_x86_64.whl.metadata (6.8 kB) Collecting unidecode Downloading Unidecode-1.3.8-py3-none-any.whl.metadata (13 kB) Collecting psycopg-binary Downloading psycopg_binary-3.1.17-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.8 kB) Collecting fschat Downloading fschat-0.2.35-py3-none-any.whl.metadata (19 kB) Collecting accelerate Downloading accelerate-0.26.1-py3-none-any.whl.metadata (18 kB) Requirement already satisfied: tenacity in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (8.2.3) Collecting psycopg[c] Downloading psycopg-3.1.17-py3-none-any.whl.metadata (4.2 kB) Collecting emoji<2.0.0,>=1.0.0 (from clean-text) Downloading emoji-1.7.0.tar.gz (175 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.4/175.4 kB 2.4 MB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting ftfy<7.0,>=6.0 (from clean-text) Downloading ftfy-6.1.3-py3-none-any.whl.metadata (6.2 kB) Collecting ninja (from vllm) Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl.metadata (5.3 kB) Requirement already satisfied: psutil in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from vllm) (5.9.5) Collecting ray>=2.5.1 (from vllm) Downloading ray-2.9.1-cp310-cp310-manylinux2014_x86_64.whl.metadata (13 kB) Collecting sentencepiece (from vllm) Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 20.7 MB/s eta 0:00:00 Requirement already satisfied: numpy in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from vllm) (1.26.1) Collecting torch==2.1.2 (from vllm) Downloading torch-2.1.2-cp310-cp310-manylinux1_x86_64.whl.metadata (25 kB) Collecting transformers>=4.36.0 (from vllm) Downloading transformers-4.37.0-py3-none-any.whl.metadata (129 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.4/129.4 kB 14.9 MB/s eta 0:00:00 Collecting xformers==0.0.23.post1 (from vllm) Downloading xformers-0.0.23.post1-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.0 kB) Requirement already satisfied: fastapi in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from vllm) (0.95.2) Requirement already satisfied: uvicorn[standard] in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from vllm) (0.22.0) Requirement already satisfied: pydantic==1.10.13 in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from vllm) (1.10.13) Collecting aioprometheus[starlette] (from vllm) Downloading aioprometheus-23.12.0-py3-none-any.whl.metadata (9.8 kB) Requirement already satisfied: typing-extensions>=4.2.0 in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from pydantic==1.10.13->vllm) (4.8.0) Requirement already satisfied: filelock in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from torch==2.1.2->vllm) (3.12.4) Requirement already satisfied: sympy in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from torch==2.1.2->vllm) (1.12) Requirement already satisfied: networkx in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from torch==2.1.2->vllm) (3.2) Requirement already satisfied: jinja2 in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from torch==2.1.2->vllm) (3.1.2) Requirement already satisfied: fsspec in /home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages (from torch==2.1.2->vllm) (2023.10.0) Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.1.2->vllm) Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 25.3 MB/s eta 0:00:00 Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.1.2->vllm) Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 12.2 MB/s eta 0:00:00 Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.1.2->vllm) Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 37.7 MB/s eta 0:00:00 Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.1.2->vllm) Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.1.2->vllm) Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 4.2 MB/s eta 0:00:00 Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.1.2->vllm) Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 19.4 MB/s eta 0:00:00 Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.1.2->vllm) Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 13.4 MB/s eta 0:00:00 Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.1.2->vllm) Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 6.5 MB/s eta 0:00:00 Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.1.2->vllm) Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 4.6 MB/s eta 0:00:00 Collecting nvidia-nccl-cu12==2.18.1 (from torch==2.1.2->vllm) Downloading nvidia_nccl_cu12-2.18.1-py3-none-manylinux1_x86_64.whl (209.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 MB 4.5 MB/s eta 0:00:00 Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.1.2->vllm) Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 18.0 MB/s eta 0:00:00 Collecting triton==2.1.0 (from torch==2.1.2->vllm) Downloading triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.3 kB) Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.1.2->vllm) Downloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) Collecting psycopg-c==3.1.17 (from psycopg[c]) Downloading psycopg-c-3.1.17.tar.gz (561 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 561.4/561.4 kB 55.5 MB/s eta 0:00:00 Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'error' error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
running dist_info
creating /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info
writing /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/top_level.txt
writing manifest file '/tmp/pip-modern-metadata-5gamrb8j/psycopg_c.egg-info/SOURCES.txt'
couldn't run 'pg_config' --includedir: [Errno 2] No such file or directory: 'pg_config'
error: [Errno 2] No such file or directory: 'pg_config'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.