TransWikia.com

psycopg2 on elastic beanstalk - can't deploy app

Stack Overflow Asked by shartshooter on November 4, 2021

I’m deploying my first app(Flask) to EB and am hitting a snag. Using PostgreSQL requires that I get psycopg2 installed but I can’t figure out how to get it running.

I saw in this post (Psycopg2 on Amazon Elastic Beanstalk) as well as this post(https://realpython.com/blog/python/deploying-a-django-app-to-aws-elastic-beanstalk/) that I need to add a file name .elasticbeanstalk/01_packages.config with the following code:

packages:
  yum:
    postgresql-devel: []

I’ve tried different variations with postgresql93-devel: [] and still no luck. Attached below are my error logs.

[2015-11-16T21:03:24.749Z] INFO  [2680]  - [Application update/AppDeployStage0/AppDeployPreHook/03deploy.py] : Starting activity...
[2015-11-16T21:03:27.024Z] INFO  [2680]  - [Application update/AppDeployStage0/AppDeployPreHook/03deploy.py] : Activity execution failed, because: Collecting psycopg2==2.6.1 (from -r /opt/python/ondeck/app/requirements.txt (line 1))
    Using cached psycopg2-2.6.1.tar.gz
      Complete output from command python setup.py egg_info:
      running egg_info
      creating pip-egg-info/psycopg2.egg-info
      writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
      writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
      writing pip-egg-info/psycopg2.egg-info/PKG-INFO
      writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
      warning: manifest_maker: standard file '-c' not found

      Error: pg_config executable not found.

      Please add the directory containing pg_config to the PATH
      or specify the full executable path with the option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      ----------------------------------------
  Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-z20gvx7_/psycopg2
  2015-11-16 21:03:27,010 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
  Traceback (most recent call last):
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
      install_dependencies()
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
    File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
      raise CalledProcessError(retcode, cmd)
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (ElasticBeanstalk::ExternalInvocationError)
caused by: Collecting psycopg2==2.6.1 (from -r /opt/python/ondeck/app/requirements.txt (line 1))
    Using cached psycopg2-2.6.1.tar.gz
      Complete output from command python setup.py egg_info:
      running egg_info
      creating pip-egg-info/psycopg2.egg-info
      writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
      writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
      writing pip-egg-info/psycopg2.egg-info/PKG-INFO
      writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
      warning: manifest_maker: standard file '-c' not found

      Error: pg_config executable not found.

      Please add the directory containing pg_config to the PATH
      or specify the full executable path with the option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      ----------------------------------------
  Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-z20gvx7_/psycopg2
  2015-11-16 21:03:27,010 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
  Traceback (most recent call last):
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
      install_dependencies()
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
    File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
      raise CalledProcessError(retcode, cmd)
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (Executor::NonZeroExitStatus)

It seems pretty clear that I need to get psycopg2 installed but I’m not sure what other steps I can take to make this work.

amazon ec2amazon elastic beanstalkamazon web servicespostgresqlpython

3 Answers

Funny enough, the solution for me for Amazon Linux 2 ended up being what the original question was - adding a 01_package.conf with the contents of

packages:
  yum:
    postgresql-devel: []

Furthmore, this was contrary to almost all answers I found online which directed to use postgresql96-devel or some other number. I found out I needed to make the package name postgresql-devel after SSHing onto the EC2 instance and viewing all available Yum packages.

[ec2-user@ip-xxx-xxx-xxx-xxx ~]$ yum list available | grep 'postgresql'
freeradius-postgresql.x86_64           3.0.13-10.amzn2                amzn2-core
pcp-pmda-postgresql.x86_64             3.12.2-5.amzn2                 amzn2-core
postgresql.x86_64                      10.4-5.amzn2.0.2               amzn2extra-postgresql10
postgresql-contrib.x86_64              10.4-5.amzn2.0.2               amzn2extra-postgresql10
postgresql-devel.x86_64                10.4-5.amzn2.0.2               @amzn2extra-postgresql10
postgresql-docs.x86_64                 10.4-5.amzn2.0.2               amzn2extra-postgresql10
...

and I saw it was just postgres-devel. Once I changed it to that it deployed alright. Make sure you commit your change if you're using eb with git before running eb deploy - or at least add the changes to your git staging area and run with eb deploy --staged as well. That one cost me a run around thinking I had already tried the package name when I hadn't ?

I found the answers on this stackoverflow post here to be helpful as well. Good luck!

I ended up needing a newer psql client (11.x) than what was default using the above answer (9.2 at the time of posting). This answer here was the only way I could make it work. Answer pasted here for posterity...

packages:
    yum:
        amazon-linux-extras: []

commands:
    01_postgres_activate:
        command: sudo amazon-linux-extras enable postgresql11
    02_postgres_install:
        command: sudo yum install -y postgresql-devel

Answered by Nick Brady on November 4, 2021

Even after installing postgresql93-devel, I kept getting an error "no module named psycopg2". To solve this I had to connect to my enviroment with eb ssh and then execute sudo pip install psycopg2 to solve this.

Answered by joalquipe on November 4, 2021

I solved it, you need to open you EC2 instance first, then install like this

sudo yum install postgresql-devel

Then deploy your EB

Answered by Muke on November 4, 2021

Add your own answers!

Ask a Question

Get help from others!

Recent Questions

Recent Answers

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP