Error job failed exit code 1 gitlab

Overview

This drove me crazy and I’m still not sure what the appropriate answer is. I just ran into this issue myself and sunk hours into this issue. I think GitLab messed something up with command substitution (shows a new release yesterday), although I could be wrong about the issue or its timing. It also seems to only occur for some command substitutions and not others, I initially suspected it might be related to outputting to /dev/null, but wasn’t going to dive too deep. It always failed immediately after the command substitution was initiated.

My code

I had code similar to yours (reduced version below), tried manipulating it multiple ways, but each use of command substitution yielded the same failure message:

Cleaning up file based variables          00:01
ERROR: Job failed: exit code 1

Attempts I’ve made include the following:

- folders=$(find .[^.]* * -type d -maxdepth 0 -exec echo {} ; 2>/dev/null)
- >
  while read folder; do
      echo "$folder"
  done <<< "$folders"

And …

- >
  while read folder; do
      echo "$folder"
  done <<< $(find .[^.]* * -type d -maxdepth 0 -exec echo {} ; 2>/dev/null)

Both those versions succeeded on my local machine, but failed in GitLab (I might have typos in above — please don’t scrutinize, it’s reduced version of my actual program).

How I fixed it

Rather than using command substitution $(...), I instead opted for process substitution <(...) and it seems to be working without issue.

- >
  while read folder; do
      echo "$folder"
  done < <(find .[^.]* * -type d -maxdepth 0 -exec echo {} ; 2>/dev/null)

I would try to substitute the same in your code if possible:

- >
  while read dir; do
      # the rest goes here
  done < <(git log -m -1 --name-only -r --pretty="format:" "$CI_COMMIT_SHA")

The issue might also be the line inside the if statement (the echo), you can replace that with the following:

read SERVICE < <(echo "$dir")

Again, not exactly sure this will fix the issue for you as I’m still unsure what the cause is, but it resolved my issue. Best of luck.

GitLab CI Shell Executor failing builds with ERROR: Job failed: exit status 1

Tonight I’ve been playing around with GitLab CI’s shell executor, and my builds have been failing with this error:

After some searching online, it appeared that similar to when receiving No Such Directory , this comment noted that it’s an issue with SKEL — the solution was to delete .bash_logout from the gitlab-runner user’s home, but I also removed .bashrc and .profile .

How to work around `ERROR: Job failed: exit status 1` errors with GitLab CI’s shell executor.

Written by Jamie Tanna on Wed, 03 Jun 2020 21:13:41 BST , and last updated on Wed, 02 Mar 2022 13:34:19 UTC .

Content for this article is shared under the terms of the Creative Commons Attribution Non Commercial Share Alike 4.0 International, and code is shared under the Apache License 2.0.

Has this content helped you? Did it solve that difficult-to-resolve issue you’ve been chasing for weeks? Or has it taught you something new you’ll be able to re-use daily?

Please consider supporting me so I can continue to create content like this!

This post was filed under articles.

Interactions with this post

Below you can find the interactions that this page has had using WebMention.

Have you written a response to this post? Let me know the URL:

Do you not have a website set up with WebMention capabilities? You can use Comment Parade.

Источник

Upload artifact fails but jobs succeeds

Summary

Occasionally, after one of our build jobs is finished, it will start uploading the artifacts, and silently fail: the job succeeds but the artifacts are not uploaded and no errors are reported.

• Jobs that depend on these artifacts then fail, which requires re-running the whole job again (as opposed to just re-running the upload)

Steps to reproduce

• I have managed to reproduce the behavior (the helper crashing but job succeeding)
  • More details in this project https://gitlab.com/jpsamper/runner-helper-reproducer
  • In production, we usually see it when there are a lot of jobs running/uploading artifacts at the same time
    • We’ve seen it with as low as 10 jobs uploading 1.5GB zipped (4-5GB unzipped) concurrently
• By using a custom gitlab-runner-helper with a lot more print statements, we have found that the logs stop after invoking r.client.Do (i.e. if we add a print statement right before and right after, the one right after never appears)
  • Naturally, this is the behavior when something goes wrong, when the artifact is uploaded correctly, we see both print statements

Actual behavior

If I understand correctly, the function call linked above is invoking Do from the net/http package, and that call seems to be crashing the gitlab-runner-helper with no additional error message/return code/etc.

Expected behavior

• If an artifact upload fails, the job fails or retries
• An informative error message too, ideally

Relevant logs and/or screenshots

When everything works as expected:

And when it doesn’t:

Environment description

• Gitlab Runner with docker executor kubernetes executor
• Latest docker version
• Default config.toml

Used GitLab Runner version

We’re currently on gitlab-runner 13.3.0 but have been seeing this since at least 12.9.0

Источник

Container killed while running / Output logs missing in GitLab console

Summary

The job output is not completely displayed in GitLab console. That happened for successful and failed jobs.

Context

The script’s job simply call a bash script:

Output

The GitLab job console output the following:

As one can see, the final echo Job completed with code $exit_code did not show up. I have also seen this behavior with a successful job. When following the container with git logs -f , the output also stops in the middle of nowhere.

So, I’m not sure if sometimes the container gets killed too early, resulting in a exit code 1, or if it is a job failure in my script , since I have no way to access the error logs. Also it is possible that some flush is missing because sometime the job is still successful even if I don’t see the end of the logs.

Reproducibility

This happens with gitlab-runner 12.10.2 but:

• it happens always on the same job (we have multiple jobs of the same size in the same stage that are running concurrently).
• it happens only on one runner. If the job is executed on another runner (by pausing the problematic one), the job completes normally.
• on the problematic runner, the crash occurs almost always for that job, but sometimes it passes.
• the crash happen even if it is the only job running on the server (we have 4 executors on this runner).

I could see this happen directly: I was following the executor with docker logs -f . Suddenly, the logs stops and I get back to the shell. Then a docker logs just tell that no such container exists, meaning the runner just deleted it directly after he crashed.

More information

• We are running GitLab 12.10.3, with 2 runners version 12.10.2.
• The problem has also been seen on gitlab-runner 12.10.1 and 12.9.0 (but with a GitLab 12.10.3 instance)

It looks like related to our job that could be flaky, but that’s not the case (as said, this kind of error did never occured with previous GitLab versions and even with a flaky code, we should see the last echo line). Also, the same job on a stable branch (which had no modification since we updated GitLab) fails with the exact same behavior.

We updated the system’s runner and rebooted it, pruned all the containers, but nevertheless the error is still present.

At this point, it is very hard to get a deeper understanding of this problem because:

• we have multiple other jobs with similar, even longer tests that run without any problem.
• this same behaviour has been seen on both runners. If I shut down the runner and the job is executed by the other one, it succeeds, even with partial logs.
• it really looks like either gitlab-runner «returns» too early, omitting some log from the container, or either kills it prematurely.
• could it be related to #25348 (closed) ? 🤔

Configuration

We use Docker version 18.09.1, build 4c52b90 and docker-compose version 1.21.0 with the following configuration:

Источник

Make it possible to control build status using exit codes

Update

We’ve open a separate issue #273157 (closed) to explore additional ways on how to solve this issue — Please add your comments to this issue only if the proposed solution does not fit your needs

Description

It may be an interesting feature to be able to control build status using .gitlab-ci.yml and exit codes.

Proposal

Implement control for exit codes as follows, based on the list of possible ones that are supported (failed, warning, success):

script_exit_codes are applied to the job based on the first non-zero exit code encountered in the script, or the last zero if all exit codes were zero.

• If success: is not defined, it is evaluated as success: [0] . If it is defined, and you want 0 to be a success, it must be included in your array.
• If warning: is not defined, it is evaluated as warning: []
• If canceled: is not defined, it is evaluated as canceled: []
• If a job is set to allow_failure: true , all failure states will be also treated as warnings.

Invalid configurations:

The following configuration should be considered invalid and a syntax error should be returned because of duplicate values ( 1 in this case). Both success and warning should have mutually exclusive values.

References

Idea from @ayufan during Slack conversation 🙂

Sequencing

1. Add script_exit_codes to the YAML keywords
2. Include script_exit_codes in the /request jobs response
3. Add support for canceled state. This should already be done as part of gitlab-runner#4843
4. The runner should compare the exit code with the ones defined in script_exit_codes (if available) before sending the status.

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Источник

gitlab runner выдает в конце сообщение «Очистка файловых переменных 00:01 ERROR: Job failed: exit code 1»

Несмотря на то, что все мои шаги проходят успешно, Gitlab CI показывает это — «Очистка файловых переменных 00:01 ОШИБКА: задание не выполнено: код выхода 1»

И терпит неудачу в самом конце. Также интересно, что это происходит только для моей основной ветки. Он успешно работает в других ветках. Кто-нибудь сталкивался с этой проблемой и нашел решение?

4 ответа

Обзор

Это свело меня с ума, и я до сих пор не знаю, какой ответ правильный. Я сам столкнулся с этой проблемой и потратил на нее часы. Я думаю, что GitLab что-то испортил с подстановкой команд (показывает новый выпуск вчера), хотя я могу ошибаться насчет проблемы или ее сроков. Кроме того, похоже, что это происходит только для некоторых замен команд, а не для других. Я изначально подозревал, что это может быть связано с выводом в /dev/null , но не собирался углубляться. Он всегда выходил из строя сразу после того, как была инициирована подстановка команды.

Мой код

У меня был код, похожий на ваш (сокращенная версия ниже), я пытался манипулировать им несколькими способами, но каждое использование подстановки команд приводило к одному и тому же сообщению об ошибке:

Я предпринял следующие попытки:

Обе эти версии успешно работали на моем локальном компьютере, но не работали в GitLab (у меня могут быть опечатки выше — пожалуйста, не разбирайтесь, это уменьшенная версия моей реальной программы).

Как я это исправил

Вместо того, чтобы использовать подстановку команд $(. ) , я выбрал замену процесса , и, похоже, она работает без проблем.

Я бы попытался заменить то же самое в вашем коде, если это возможно:

Проблема также может заключаться в строке внутри оператора if (эхо), вы можете заменить ее следующим:

Опять же, не совсем уверен, что это решит проблему для вас, поскольку я все еще не уверен, в чем причина, но это решило мою проблему. Удачи.

В моем случае мой сценарий завершился командой curl для URL-адреса, который вернул бы 403 Forbidden и, вероятно, повесил трубку

. если это кому-то поможет 🙂

Мы столкнулись с той же проблемой в GitLab v13.3.6-ee со следующей строкой скрипта, который мы используем для открытого нового запроса на слияние:

И, как заявил @ctwheels, изменив эту строку на это:

Источник

Запускаю karma тесты для Angular в gitlab CI.

.gitlab-ci.yml:

image: node:14.0.0

cache:
  paths:
    - node_modules/

before_script:
  - npm install
  - apt-get update -qq
  - apt-get install -y -qq rsync
  - apt-get install -y -qq sshpass
  - wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
  - apt install ./google-chrome-stable_current_amd64.deb -y

stages:
  - testing_deploy


build_and_deploy_testing:
  stage: testing_deploy
  script:
    - echo "============== ADD DEVELOP SSH KEY =============="
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add - > /dev/null
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config
    - echo "============== START BUILD AND DEPLOY TESTING =============="
    - ./node_modules/@angular/cli/bin/ng test --progress false --watch=false

Failed job сообщение:

12 01 2021 16:25:31.828:ERROR [launcher]: Cannot start Chrome
    [2172:2172:0112/162531.552480:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

12 01 2021 16:25:31.829:ERROR [launcher]: Chrome stdout: 
12 01 2021 16:25:31.829:ERROR [launcher]: Chrome stderr: [2172:2172:0112/162531.552480:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

12 01 2021 16:25:31.840:ERROR [launcher]: Chrome failed 2 times (cannot start). Giving up.
Cleaning up file based variables
ERROR: Job failed: exit code 1

Как это исправить?

446 votes

1 answers

Get the solution ↓↓↓

checking for PHP prefix... /usr/local
checking for PHP includes... -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib
checking for PHP extension directory... /usr/local/lib/php/extensions/no-debug-non-zts-20190902
checking for PHP installed headers prefix... /usr/local/include/php
checking if debug is enabled... no
checking if zts is enabled... no
checking for gawk... no
checking for nawk... nawk
checking if nawk is broken... no
checking for zip archive read/write support... yes, shared
checking for libzip >= 0.11 libzip != 1.3.1 libzip != 1.7.0... no
configure: error: Package requirements (libzip >= 0.11 libzip != 1.3.1 libzip != 1.7.0) were not met:
No package 'libzip' found
No package 'libzip' found
No package 'libzip' found
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.
Alternatively, you may set the environment variables LIBZIP_CFLAGS
and LIBZIP_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
ERROR: Job failed: exit code 1

Write your answer

457

votes

Answer

Solution:

you are missing libzip package, so you need to include libzip-dev package as well.

In your .gitlab-ci.yml file istead of:

  # Install dependencies
  - apt-get install libonig-dev git nodejs libcurl4-gnutls-dev libicu-dev libmcrypt-dev libvpx-dev libjpeg-dev libpng-dev libxpm-dev zlib1g-dev libfreetype6-dev libxml2-dev libexpat1-dev libbz2-dev libgmp3-dev libldap2-dev unixodbc-dev libpq-dev libsqlite3-dev libaspell-dev libsnmp-dev libpcre3-dev libtidy-dev -yqq

You need to have:

  # Install dependencies
  - apt-get install libonig-dev git nodejs libcurl4-gnutls-dev libicu-dev libmcrypt-dev libvpx-dev libjpeg-dev libpng-dev libxpm-dev zlib1g-dev libfreetype6-dev libxml2-dev libexpat1-dev libbz2-dev libgmp3-dev libldap2-dev unixodbc-dev libpq-dev libsqlite3-dev libaspell-dev libsnmp-dev libpcre3-dev libtidy-dev libzip-dev -yqq

Then this error should not happen anymore.

Share solution ↓

Additional Information:

Link To Source

Link To Answer
People are also looking for solutions of the problem: use the option —with-all-dependencies (-w) to allow upgrades, downgrades and removals for packages currently locked to specific versions.

Similar questions

Find the answer in similar questions on our website.