Error executing action run on resource bash migrate gitlab rails database

I’ve installed gitlab before and it was a pretty straight forward process, for the past week however the process outlines on the website fails at the sudo EXTERNAL_URL="http://gitlab.example.com" apt-get install gitlab-ee step.

The error seen:

    ================================================================================
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of "bash"  "/tmp/chef-script20171024-7894-1sl10lo" ----
    STDOUT: rake aborted!
    PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => gitlab:db:configure
    (See full trace by running task with --trace)
    STDERR: 
    ---- End output of "bash"  "/tmp/chef-script20171024-7894-1sl10lo" ----
    Ran "bash"  "/tmp/chef-script20171024-7894-1sl10lo" returned 1

    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

     51: bash "migrate gitlab-rails database" do
     52:   code <<-EOH
     53:     set -e
     54:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
     55:     umask 077
     56:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
     57:     STATUS=${PIPESTATUS[0]}
     58:     echo $STATUS > #{db_migrate_status_file}
     59:     exit $STATUS
     60:   EOH
     61:   environment env_variables unless env_variables.empty?
     62:   notifies :run, 'execute[enable pg_trgm extension]', :before if omnibus_helper.service_enabled?('postgresql')
     63:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
     64:   dependent_services.each do |svc|
     65:     notifies :restart, svc, :immediately
     66:   end
     67:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
     68:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
     69: end

    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:51:in `from_file'

    bash("migrate gitlab-rails database") do
      action [:run]
      updated true
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      command "migrate gitlab-rails database"
      backup 5
      returns 0
      code "    set -en    log_file="/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"n    umask 077n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}n    STATUS=${PIPESTATUS[0]}n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-3a712c3n    exit $STATUSn"
      interpreter "bash"
      declared_type :bash
      cookbook_name "gitlab"
      recipe_name "database_migrations"
      not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-3a712c3) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-3a712c3 | grep -Fx 0)"
      only_if { #code block }
    end

    Platform:
    ---------
    x86_64-linux


Running handlers:
Running handlers complete
Chef Client failed. 5 resources updated in 45 seconds
dpkg: error processing package gitlab-ee (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 gitlab-ee
E: Sub-process /usr/bin/dpkg returned an error code (1)

This is on Ubuntu 16.04LTS. How to resolve this and install gitlab?

I’ve also tried using older ways of installing gitlab (where gitlab was first installed then configured) however it still failed at the sudo gitlab-ctl reconfigure step.

Upgrading gitlab-ce 11.9.8 fails with database_migrations failure and unicorn segfaults

Upgrading my omnibus installation from gitlab-ce 11.9.8 to upper versions fails with the following error during the database-migrations step of gitlab-ctl reconfigure ;

At which point the web UI returns a 502. gitlab appears to successfully starts but sidekiq reports a failing state afterwhat. Logs shows unicorn segfaulting indefinitely, sidekiq outputs stacktraces about redis, redis appears fine.

Rolling back to the previous dpkg fails because of supposedly malformed nodename.json, then succeed after removing the offending file;

Rollback process

• postgresql (0.1.0)
• redis (0.1.0)
• registry (0.1.0)
• mattermost (0.1.0)
• consul (0.1.0)
• letsencrypt (0.1.0)
• nginx (0.1.0)
• gitlab (0.0.1)
• gitaly (0.1.0)
• runit (4.3.0)
• package (0.1.0)
• crond (0.1.0)
• compat_resource (12.19.1)
• acme (3.1.0) Installing Cookbook Gems: Compiling Cookbooks. Converging 1 resources Recipe: postgresql::bin

• ruby_block[Link postgresql bin files to the correct version] action run (skipped due to only_if)

Running handlers: Running handlers complete Chef Client finished, 0/1 resources updated in 07 seconds Checking PostgreSQL executables: OK Found /etc/gitlab/skip-auto-reconfigure, exiting.

Upgrade complete! If your GitLab server is misbehaving try running sudo gitlab-ctl restart

at which point the service is up, without dataloss.

Источник

Migration error on upgrading from 14.10 to 15.0 using Docker image

Summary

I can’t upgrade from Gitlab (EE) 14.10 to 15.0 through Docker image because of database migrations. Command gitlab-ctl reconfigure fails with rails migration error (see below). Thanks for your help.

Steps to reproduce

From a Docker installation with gitlab/gitlab-ee:14.10.3-ee.0 , I replace my configuration with gitlab/gitlab-ee:15.0.3-ee.0 and recreate the container.

Docker command I use:

What is the current bug behavior?

The container tries to reconfigure, fails and restart continuously.

What is the expected correct behavior?

The container is reconfigured successfully and starts with the new Docker image (gitlab/gitlab-ee:15.0.3-ee.0).

Relevant logs

runit_service[mailroom] action restart

================================================================================ Error executing action restart on resource ‘runit_service[mailroom]’

Mixlib::ShellOut::ShellCommandFailed

Expected process to exit with [0], but received ‘1’ —- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/mailroom —- STDOUT: fail: /opt/gitlab/service/mailroom: unable to change to service directory: file does not exist STDERR: —- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/mailroom —- Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/mailroom returned 1

Cookbook Trace:

/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in tap’ /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in safe_sv_shellout!’ /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in restart_service’ /opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:358:in block in class:RunitService’

Resource Declaration:

In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/mailroom.rb

Compiled Resource:

Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/mailroom.rb:34:in `from_file’

runit_service(«mailroom») do action [:enable] default_guard_interpreter :default declared_type :runit_service cookbook_name «gitlab» recipe_name «mailroom» service_name «mailroom» options <:env_dir=>«/opt/gitlab/sv/mailroom/env», :user=>»git», :groupname=>»git», :log_directory=>»/var/log/gitlab/mailroom», :mail_room_config=>»/opt/gitlab/embedded/service/gitlab-rails/config/mail_room.yml», :exit_log_format=>»plain», :working_dir=>»/var/opt/gitlab/gitlab-rails/working»> log_options <«svlogd_size»=>209715200, «svlogd_num»=>30, «svlogd_timeout»=>86400, «svlogd_filter»=>»gzip», «svlogd_udp»=>nil, «svlogd_prefix»=>nil, «udp_log_shipping_host»=>nil, «udp_log_shipping_hostname»=>nil, «udp_log_shipping_port»=>514, «logrotate_frequency»=>»daily», «logrotate_maxsize»=>nil, «logrotate_size»=>nil, «logrotate_rotate»=>30, «logrotate_compress»=>»compress», «logrotate_method»=>»copytruncate», «logrotate_postrotate»=>nil, «logrotate_dateformat»=>nil, «enable»=>true, «ha»=>false, «log_directory»=>»/var/log/gitlab/mailroom», «exit_log_format»=>»plain», «incoming_email_auth_token»=>nil, «service_desk_email_auth_token»=>nil> run_template_name «mailroom» finish true log_template_name «mailroom» check_script_template_name «mailroom» finish_script_template_name «mailroom» control_template_names <> end

System Info:

chef_version=15.17.4 platform=ubuntu platform_version=20.04 ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux] program_name=/opt/gitlab/embedded/bin/cinc-client executable=/opt/gitlab/embedded/bin/cinc-client

• execute[clear the gitlab-rails cache] action run
  • execute /opt/gitlab/bin/gitlab-rake cache:clear

Running handlers: There was an error running gitlab-ctl reconfigure:

Multiple failures occurred:

• Mixlib::ShellOut::ShellCommandFailed occurred in Cinc Client run: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
• Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: runit_service[mailroom] (gitlab::mailroom line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘1’ —- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/mailroom —- STDOUT: fail: /opt/gitlab/service/mailroom: unable to change to service directory: file does not exist STDERR: —- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/mailroom —- Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/mailroom returned 1

Running handlers complete Cinc Client failed. 154 resources updated in 02 minutes 21 seconds

Источник

ruby crashes with SIGSEGV in omnibus 10.6.0-ce.0 package on debian 9.4 due to libjemalloc.so in LD_PRELOAD

Summary

Updating gitlab-ce omnibus debian package 10.5.4-ce.0 -> 10.6.0-ce.0 gives:

Error executing action `run` on resource ‘bash[migrate gitlab-rails database]’

Relevant logs are attached below.

Manual attempts to execute failed db migration action gives SIGSEGV :

Further investigation shows that:

• ruby crashes on early stages after clone() syscall(see strace output and gdb backtrace below)
• crash related to libjemalloc.so loaded by /opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD

Removing /opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD fixes everything, so manually running gitlab-rake gitlab:db:configure afterwards gives usable installation.

I am not sure if the problem is with debian’s libc6 2.24-11+deb9u3 or with omnibus package’s libjemalloc.so/ruby or with something else, but some combination of those definitely fails to work on my installation.

Steps to reproduce

> sudo apt-get install gitlab-ce on Debian 9.4 x64 with libc6 version: 2.24-11+deb9u3

What is the current bug behavior?

What is the expected correct behavior?

Working ruby installation is expected.

Relevant logs

Results of GitLab environment info

Suggested output of gitlab-rake gitlab:env:info is obviously Segmentation fault as well. If libjemalloc.so is removed from LD_PRELOAD, however, the output is following

Expand for output related to GitLab environment info

• default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Suggested output of gitlab-rake gitlab:check SANITIZE=true is obviously Segmentation fault as well. If libjemalloc.so is removed from LD_PRELOAD, however, the output is following

Expand for output related to the GitLab application check

Possible fixes

fixes the problem with crashing ruby. So manual running of db migration

succeeds and gitlab is finally running after restart.

Источник

Bash[migrate gitlab-rails database] error when i do “gitlab-crl reconfigure” — Gitlab

I have a problem with my Gitlab and i cannot resolve it :frowning: I tried to backup with sudo gitlab-rake gitlab:backup:create STRATEGY=copy, i did that many times and everything was fine, but the last time there was some error.

This is the whole error when i was tried to do “gitlab-ctl reconfigure”. Now my git server doesn’t work and i don’t know how to fix this error. I have 8 GB RAM, so i guess that it’s not a problem with memory. I cannot do command like gitlab:rake info etc. All services in gitlab-ctl status are working (RUN) Tried to restart gitlab-ctl but unsuccess. Also, cannot yum install gitlab-ce newer version before i fix gitlab-ctl reconfigure

Running handlers: There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database] (gitlab::database_migrations line 54) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘1’ —- Begin output of «bash» «/tmp/chef-script20191024-2422-c0yhb3» —- STDOUT: rake aborted! TypeError: Internal is not a module /opt/gitlab/embedded/service/gitlab-rails/lib/api/internal/base.rb:5:in ‘ /opt/gitlab/embedded/service/gitlab-rails/lib/api/internal/base.rb:3:in ‘ /opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:121:in ‘ /opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:4:in ‘ /opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:3:in ‘ /opt/gitlab/embedded/service/gitlab-rails/config/initializers/rack_attack.rb:12:in ‘ /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:6:in ‘ /opt/gitlab/embedded/bin/bundle:23:in load’ /opt/gitlab/embedded/bin/bundle:23:in `’ Tasks: TOP => gitlab:db:configure => environment (See full trace by running task with —trace) STDERR: —- End output of «bash» «/tmp/chef-script20191024-2422-c0yhb3» —- Ran «bash» «/tmp/chef-script20191024-2422-c0yhb3» returned 1

Running handlers complete Chef Client failed. 5 resources updated in 01 minutes 53 seconds

STDOUT: rake aborted! TypeError: Internal is not a module

What does mean this error, Internal is not a module, i didn’t find any Topis whit this error :frowning: I will be gratefull if someone help me with this. Thanks a lot!*

Источник

Gitlab-CE reconfigure time out and gitlab-rake had no response

Hi, Any guys can help me? I hava been bored by this problem for two weeks.

I had installed Gitlab-CE 9.0.4 in Ubuntu 14.04_x86_64 by using apt-get install gitlab-ce command. And it had been ok for me for about 2 months. But suddently a day, when I pushed a commit to my gitlab-ce, I found that gitlat-ce returned 502 error. So I use lslf -i:8082 command to see whether unicorn service is OK. Because I used 8082 port for unicorn in /etc/gitlab/gitlab.rb. For this command’s result, I found that unicorn was not bind its port. And it didn’t work for me to change this port in gitlab.rb and reconfigure.

So I use sudo gitlab-rake gitlab:check SANITIZE=true command to see if there were any wrong things. But it had no response for a long time, even that I had been waiting for it for a whole night.

So I use apt-get update and apt-get upgrade to update Gitlab-CE. but failed. I found the update failed in follow steps:

bash[migrate gitlab-rails database] action run

================================================================================ Error executing action run on resource ‘bash[migrate gitlab-rails database]’

Mixlib::ShellOut::CommandTimeout

Command timed out after 3600s: Command exceeded allowed execution time, process terminated —- Begin output of «bash» «/tmp/chef-script20170814-104859-fk1812» —- STDOUT: STDERR: —- End output of «bash» «/tmp/chef-script20170814-104859-fk1812» —- Ran «bash» «/tmp/chef-script20170814-104859-fk1812» returned

Resource Declaration:

In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

51: bash «migrate gitlab-rails database» do

And then I removed gitlab-ce followed gitlab official steps from gitlab.com. And downloaded gitlab-ce by using apt-get download gitlab-ce , and used dpkg -i gitlab-ce_9.4.3xxxxx.deb to install. It was OK but when I use gitlab-ctl reconfigure it failed for the same thing: database migrate timeout.

So I looked at gitlab-rake code, I found it use «exec /opt/gitlab/embedded/bin/chpst -e /opt/gitlab/etc/gitlab-rails/env -u git -U git /opt/gitlab/embedded/bin/bundle exec rake «$@». So I use «exec /opt/gitlab/embedded/bin/chpst -e /opt/gitlab/etc/gitlab-rails/env -u git -U git /opt/gitlab/embedded/bin/bundle exec rake «gitlab:env:info» to test. But I found it has no response for a long long long long long time.

I guess that the timeout may had some relations with db migrate timeout.

This is my os infomation:

14.04.1-Ubuntu SMP Fri Jan 6 18:02:02 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@PEK1000114103:/etc/gitlab# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 45 model name : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz stepping : 7 microcode : 0x710 cpu MHz : 2900.042 cache size : 20480 KB physical id : 0 siblings : 16 core id : 0 cpu cores : 16 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt bugs : bogomips : 5800.08 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management:

processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 45 model name : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz stepping : 7 microcode : 0x710 cpu MHz : 2900.042 cache size : 20480 KB

root@PEK1000114103:/etc/gitlab# cat /proc/meminfo MemTotal: 32921716 kB MemFree: 28081740 kB MemAvailable: 31385988 kB Buffers: 640272 kB Cached: 2939556 kB SwapCached: 0 kB Active: 2265948 kB Inactive: 1978444 kB Active(anon): 742188 kB Inactive(anon): 187336 kB Active(file): 1523760 kB Inactive(file): 1791108 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 3905532 kB SwapFree: 3905532 kB Dirty: 120 kB Writeback: 0 kB AnonPages: 664908 kB Mapped: 366752 kB Shmem: 264968 kB Slab: 450744 kB SReclaimable: 390132 kB SUnreclaim: 60612 kB KernelStack: 7568 kB PageTables: 20240 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 20366388 kB Committed_AS: 10105540 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 497664 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 67580 kB DirectMap2M: 33609728 kB

Источник

Recipe: gitlab::database_migrations

• bash[migrate gitlab-rails database] action run

  ================================================================================
  Error executing action run on resource ‘bash[migrate gitlab-rails database]’

  Mixlib::ShellOut::ShellCommandFailed

  Expected process to exit with [0], but received ‘137’
  —- Begin output of «bash» «/tmp/chef-script20170225-13894-1tm6qkl» —-
  STDOUT:
  STDERR:
  —- End output of «bash» «/tmp/chef-script20170225-13894-1tm6qkl» —-
  Ran «bash» «/tmp/chef-script20170225-13894-1tm6qkl» returned 137

  Resource Declaration:

  In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

  51: bash «migrate gitlab-rails database» do
  52: code <<-EOH
  53: set -e
  54: log_file=»#{node[‘gitlab’][‘gitlab-rails’][‘log_directory’]}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log»
  55: umask 077
  56: /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
  57: STATUS=${PIPESTATUS[0]}
  58: echo $STATUS > #{db_migrate_status_file}
  59: exit $STATUS
  60: EOH
  61: environment env_variables unless env_variables.empty?
  62: notifies :run, ‘execute[enable pg_trgm extension]’, :before unless omnibus_helper.not_listening?(«postgresql») || !node[‘gitlab’][‘postgresql’][‘enable’]
  63: notifies :run, «execute[clear the gitlab-rails cache]», :immediately unless omnibus_helper.not_listening?(«redis») || !node[‘gitlab’][‘gitlab-rails’][‘rake_cache_clear’]
  64: dependent_services.each do |svc|
  65: notifies :restart, svc, :immediately
  66: end
  67: not_if «(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)»
  68: only_if { node[‘gitlab’][‘gitlab-rails’][‘auto_migrate’] }
  69: end

  Compiled Resource:

  Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:51:in `from_file’

  bash(«migrate gitlab-rails database») do
  action [:run]
  updated true
  retries 0
  retry_delay 2
  default_guard_interpreter :default
  command «migrate gitlab-rails database»
  backup 5
  returns 0
  code » set -en log_file=»/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log»n umask 077n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}n STATUS=${PIPESTATUS[0]}n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-77254abn exit $STATUSn»
  interpreter «bash»
  declared_type :bash
  cookbook_name «gitlab»
  recipe_name «database_migrations»
  not_if «(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-77254ab) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-77254ab | grep -Fx 0)»
  only_if { #code block }
  end

  Platform:

  x86_64-linux(Ubuntu/precise)

My system is running Ubuntu 18.04.2 LTS and Docker version 18.09.5, build e8ff056, Gitlab version gitlab-ee=12.0.3-ee.0.

I am trying to spin up a docker container with the following run command:

docker run -i 
  --hostname gitlab2.michael-thompson.net 
  --name gitlab 
  --restart unless-stopped 
  --volume /docker/gitlab/config:/etc/gitlab:Z 
  --volume /docker/gitlab/log:/var/log/gitlab:Z 
  --volume /docker/gitlab/data:/var/opt/gitlab:Z 
  --net=bridge 
  --env TZ=Australia/Perth 
  gitlab/gitlab-ee:latest

The container boots for 20 seconds and then restarts (and continues in a never ending loop). Everything seems to run well until Gitlab attempts to «migrate» the database, and spits out the following error:

Recipe: gitlab::database_migrations
  * bash[migrate gitlab-rails database] action run
    [execute] rake aborted!
              StandardError: An error has occurred, this and all later migrations canceled:

              PG::DuplicateTable: ERROR:  relation "audit_events" already exists
              : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'

              Caused by:
              ActiveRecord::StatementInvalid: PG::DuplicateTable: ERROR:  relation "audit_events" already exists
              : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'

              Caused by:
              PG::DuplicateTable: ERROR:  relation "audit_events" already exists
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'
              Tasks: TOP => db:migrate
              (See full trace by running task with --trace)
              == 20141118150935 AddAuditEvent: migrating ====================================
              -- adapter_name()
                 -> 0.0000s
              -- adapter_name()
                 -> 0.0000s
              -- create_table(:audit_events, {:id=>:integer})

    ================================================================================
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of "bash"  "/tmp/chef-script20190704-26-1gi3jcd" ----
    STDOUT: rake aborted!
    StandardError: An error has occurred, this and all later migrations canceled:

    PG::DuplicateTable: ERROR:  relation "audit_events" already exists
    : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'

    Caused by:
    ActiveRecord::StatementInvalid: PG::DuplicateTable: ERROR:  relation "audit_events" already exists
    : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'

    Caused by:
    PG::DuplicateTable: ERROR:  relation "audit_events" already exists
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => db:migrate
    (See full trace by running task with --trace)
    == 20141118150935 AddAuditEvent: migrating ====================================
    -- adapter_name()
       -> 0.0000s
    -- adapter_name()
       -> 0.0000s
    -- create_table(:audit_events, {:id=>:integer})
    STDERR:
    ---- End output of "bash"  "/tmp/chef-script20190704-26-1gi3jcd" ----
    Ran "bash"  "/tmp/chef-script20190704-26-1gi3jcd" returned 1

    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

     53: bash "migrate gitlab-rails database" do
     54:   code <<-EOH
     55:     set -e
     56:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
     57:     umask 077
     58:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
     59:     STATUS=${PIPESTATUS[0]}
     60:     echo $STATUS > #{db_migrate_status_file}
     61:     exit $STATUS
     62:   EOH
     63:   environment env_variables unless env_variables.empty?
     64:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
     65:   dependent_services.each do |svc|
     66:     notifies :restart, svc, :immediately
     67:   end
     68:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
     69:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
     70: end

    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:53:in `from_file'

    bash("migrate gitlab-rails database") do
      action [:run]
      default_guard_interpreter :default
      command nil
      backup 5
      returns 0
      user nil
      interpreter "bash"
      declared_type :bash
      cookbook_name "gitlab"
      recipe_name "database_migrations"
      code "    set -en    log_file="/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"n    umask 077n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}n    STATUS=${PIPESTATUS[0]}n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93n    exit $STATUSn"
      domain nil
      not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93 | grep -Fx 0)"
      only_if { #code block }
    end

    System Info:
    ------------
    chef_version=13.6.4
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Recipe: gitlab::gitlab-rails
  * execute[clear the gitlab-rails cache] action run

I’ve spent the best part of a day on this and can’t figure out why it is crashing. I’m not using an external database engine / container, so I expect that that should be brand new with my container, and don’t understand how it could be that the table already exists.

I have run the suggested docker exec -it gitlab update-permissions however given my suspicions it is database related, I had low hopes to which I seemed to be correct when trying to start the container again.

I have also tried updating my system via apt (with no luck, to no surprise), stopping and starting my container, to find the same error. I am already running 2 docker containers at this very moment on the same box, without a problem. I am also running another instance of Gitlab elsewhere (not inside a container) with the same config (although tweaked for the new system), also without a problem. It was my intention to migrate into the docker Gitlab if I could get it working.

Thanks

    ================================================================================
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of "bash"  "/tmp/chef-script20171024-7894-1sl10lo" ----
    STDOUT: rake aborted!
    PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => gitlab:db:configure
    (See full trace by running task with --trace)
    STDERR: 
    ---- End output of "bash"  "/tmp/chef-script20171024-7894-1sl10lo" ----
    Ran "bash"  "/tmp/chef-script20171024-7894-1sl10lo" returned 1

    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

     51: bash "migrate gitlab-rails database" do
     52:   code <<-EOH
     53:     set -e
     54:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
     55:     umask 077
     56:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
     57:     STATUS=${PIPESTATUS[0]}
     58:     echo $STATUS > #{db_migrate_status_file}
     59:     exit $STATUS
     60:   EOH
     61:   environment env_variables unless env_variables.empty?
     62:   notifies :run, 'execute[enable pg_trgm extension]', :before if omnibus_helper.service_enabled?('postgresql')
     63:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
     64:   dependent_services.each do |svc|
     65:     notifies :restart, svc, :immediately
     66:   end
     67:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
     68:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
     69: end

    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:51:in `from_file'

    bash("migrate gitlab-rails database") do
      action [:run]
      updated true
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      command "migrate gitlab-rails database"
      backup 5
      returns 0
      code "    set -en    log_file="/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"n    umask 077n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}n    STATUS=${PIPESTATUS[0]}n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-3a712c3n    exit $STATUSn"
      interpreter "bash"
      declared_type :bash
      cookbook_name "gitlab"
      recipe_name "database_migrations"
      not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-3a712c3) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-3a712c3 | grep -Fx 0)"
      only_if { #code block }
    end

    Platform:
    ---------
    x86_64-linux


Running handlers:
Running handlers complete
Chef Client failed. 5 resources updated in 45 seconds
dpkg: error processing package gitlab-ee (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 gitlab-ee
E: Sub-process /usr/bin/dpkg returned an error code (1)

2 ответа

Лучший ответ

grep -i memory /var/log/messages