Here’s systemctl status auditd.service
:
● auditd.service - Security Auditing Service
Loaded: loaded (/lib/systemd/system/auditd.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2016-10-28 19:31:48 UTC; 4s ago
Process: 17438 ExecStartPost=/sbin/auditctl -R /etc/audit/audit.rules (code=exited, status=0/SUCCESS)
Process: 17437 ExecStart=/sbin/auditd -n (code=exited, status=6)
Main PID: 17437 (code=exited, status=6)
Oct 28 19:31:48 hostname systemd[1]: Stopped Security Auditing Service.
Oct 28 19:31:48 hostname systemd[1]: Starting Security Auditing Service...
Oct 28 19:31:48 hostname auditctl[17438]: The audit system is in immutable mode, no rule changes allowed
Oct 28 19:31:48 hostname systemd[1]: auditd.service: Main process exited, code=exited, status=6/NOTCONFIGURED
Oct 28 19:31:48 hostname systemd[1]: Failed to start Security Auditing Service.
Oct 28 19:31:48 hostname systemd[1]: auditd.service: Unit entered failed state.
Oct 28 19:31:48 hostname systemd[1]: auditd.service: Failed with result 'exit-code'.
and journalctl -xe
:
Oct 28 19:31:48 hostname systemd[1]: Starting Security Auditing Service...
-- Subject: Unit auditd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit auditd.service has begun starting up.
Oct 28 19:31:48 hostname auditd[17437]: The audit daemon is exiting.
Oct 28 19:31:48 hostname auditctl[17438]: The audit system is in immutable mode, no rule changes allowed
Oct 28 19:31:48 hostname systemd[1]: auditd.service: Main process exited, code=exited, status=6/NOTCONFIGURED
Oct 28 19:31:48 hostname systemd[1]: Failed to start Security Auditing Service.
-- Subject: Unit auditd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit auditd.service has failed.
--
-- The result is failed.
Oct 28 19:31:48 hostname systemd[1]: auditd.service: Unit entered failed state.
Oct 28 19:31:48 hostname audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=auditd comm="systemd" exe="/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Oct 28 19:31:48 hostname systemd[1]: auditd.service: Failed with result 'exit-code'.
Oct 28 19:31:48 hostname kernel: audit_printk_skb: 30 callbacks suppressed
Oct 28 19:31:48 hostname kernel: audit: type=1130 audit(1477683108.658:9065): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=auditd comm="systemd" exe="/lib/systemd/systemd" hostname=? addr=? termin
Auditd was working on my system (Ubuntu 18.04LTS, kernel 4.15.0-1065-aws) until recently. But after splitting off /var into a new filesystem it fails to launch.
running ‘/sbin/auditd -f’ as root indicates a problem writing the pid file (no file exists even when it says one does) Post config load command output:
Started dispatcher: /sbin/audispd pid: 16927
type=DAEMON_START msg=audit(1587280022.692:2019): op=start ver=2.8.2 format=raw kernel=4.15.0-1065-aws auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=success
config_manager init complete
Error setting audit daemon pid (File exists)
type=DAEMON_ABORT msg=audit(1587280022.692:2020): op=set-pid auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=failed
Unable to set audit pid, exiting
The audit daemon is exiting.
Error setting audit daemon pid (Permission denied)
/var/run is a symlink to /run
/var/run permissions are 777 root:root
/run permissions are 755f root:root
no /run/auditd.pid and subsiquently no /var/run/auditd.pid exists (even though the error incorrectly reports otherwise.
/var/log/audit/audit.log output
type=DAEMON_START msg=audit(1587278222.942:5617): op=start ver=2.8.2 format=raw kernel=4.15.0-1065-aws auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconf
ined res=success
type=DAEMON_ABORT msg=audit(1587278222.943:5618): op=set-pid auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconfined res=failed
I have been pulling my hair out over this one. So I ran ‘strace /sbin/auditd -f’ and found the following line in the output.
«openat(AT_FDCWD, «/var/run/auditd.pid», O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW, 0644) = 4″
I am grasping at straws, but suspect that the O_NOFOLLOW option is causing a failure in creating the pid file since /var/run is a symlink. I could be wrong but I can’t find anything else to suspect.
Since it is best practice to split/var into a separate file system to prevent filling the root filesystem in case of an unexpected increase in log collection I suspect this is a bug. So either the system needs to be able to follow symlinks or an option such as pid_file=[filepath] needs to be available in /etc/audit/auditd.conf.
Skip to navigation
Skip to main content
Infrastructure and Management
- Red Hat Enterprise Linux
- Red Hat Virtualization
- Red Hat Identity Management
- Red Hat Directory Server
- Red Hat Certificate System
- Red Hat Satellite
- Red Hat Subscription Management
- Red Hat Update Infrastructure
- Red Hat Insights
- Red Hat Ansible Automation Platform
Cloud Computing
- Red Hat OpenShift
-
Red Hat CloudForms
- Red Hat OpenStack Platform
- Red Hat OpenShift Container Platform
- Red Hat OpenShift Data Science
- Red Hat OpenShift Online
- Red Hat OpenShift Dedicated
- Red Hat Advanced Cluster Security for Kubernetes
- Red Hat Advanced Cluster Management for Kubernetes
- Red Hat Quay
- OpenShift Dev Spaces
-
Red Hat OpenShift Service on AWS
Storage
- Red Hat Gluster Storage
- Red Hat Hyperconverged Infrastructure
- Red Hat Ceph Storage
- Red Hat OpenShift Data Foundation
Runtimes
- Red Hat Runtimes
- Red Hat JBoss Enterprise Application Platform
- Red Hat Data Grid
- Red Hat JBoss Web Server
- Red Hat Single Sign On
-
Red Hat support for Spring Boot
- Red Hat build of Node.js
- Red Hat build of Thorntail
- Red Hat build of Eclipse Vert.x
- Red Hat build of OpenJDK
- Red Hat build of Quarkus
Integration and Automation
- Red Hat Process Automation
- Red Hat Process Automation Manager
- Red Hat Decision Manager
All Products
Issue
- The auditd service does not start when the server is booted, the server is unable to boot till auditd is manually disabled in single user mode. Errors seen are:
Feb 08 15:34:50 server.com auditd[1131]: Unable to set initial audit startup state to 'enable', exiting
Feb 08 15:34:50 server.com systemd[1]: Started Security Auditing Service.
Feb 08 15:34:50 server.com auditd[1131]: The audit daemon is exiting.
Feb 08 15:34:50 server.com systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Feb 08 15:34:50 server.com auditd[1131]: Error setting audit daemon pid (Permission denied)
Feb 08 15:34:50 server.com systemd[1]: auditd.service: main process exited, code=exited, status=1/FAILURE
Feb 08 15:34:50 server.com systemd[1]: Unit auditd.service entered failed state.
Feb 08 15:34:50 server.com systemd[1]: auditd.service failed.
Environment
- Red Hat Enterprise Linux 7.x
- audit-2.6.*
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
Current Customers and Partners
Log in for full access
Log In
On a Redhat Linux 8 system, the /var/log/audit is mounted on a dedicated partition. Somehow the size of it was allocated quite small. And we needed to increase it.
After it was remounted to a bigger partition, auditd would not start. Tried to manually start the service and got the exit status code 6
root@joetest01:/var/log# systemctl start auditd
Job for auditd.service failed because the control process exited with error code.
See "systemctl status auditd.service" and "journalctl -xe" for details.
root@joetest01:/var/log# systemctl status auditd
● auditd.service - Security Auditing Service
Loaded: loaded (/usr/lib/systemd/system/auditd.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2021-12-02 20:22:49 EST; 4min 4s ago
Docs: man:auditd(8)
https://github.com/linux-audit/audit-documentation
Process: 1695914 ExecStart=/sbin/auditd (code=exited, status=6)
Dec 02 20:22:49 joetest01 systemd[1]: Starting Security Auditing Service…
Dec 02 20:22:49 joetest01 systemd[1]: auditd.service: Control process exited, code=exited status=6
Dec 02 20:22:49 joetest01 systemd[1]: auditd.service: Failed with result 'exit-code'.
Dec 02 20:22:49 joetest01 systemd[1]: Failed to start Security Auditing Service.
I Then tried to start the audit daemo from the command line to see if it would output something that was more useful.
/sbin/auditd -f
The command above will put it in debug mode where it write more info to stdout.
Also used “journalctl -xe” to see logs and saw “Permission denied” error.
root@joetest01:/var/log# journalctl -xe
Dec 02 20:22:49 joetest01 systemd[1]: Starting Security Auditing Service…
-- Subject: Unit auditd.service has begun start-up
-- Defined-By: systemd-- Unit auditd.service has begun starting up.
Dec 02 20:22:49 joetest01 kernel: kauditd_printk_skb: 1 callbacks suppressed
Dec 02 20:22:49 joetest01 kernel: audit: type=1400 audit(1638494569.921:9146): avc: denied { read } for pid=1695914 comm="auditd" name="/" dev="sda7" ino=128 scontext=system_u:system_r:auditd_t:s0 tconte>
Dec 02 20:22:49 joetest01 auditd[1695914]: Could not open dir /var/log/audit (Permission denied)
Dec 02 20:22:49 joetest01 auditd[1695914]: The audit daemon is exiting.
Dec 02 20:22:49 joetest01 systemd[1]: auditd.service: Control process exited, code=exited status=6
Dec 02 20:22:49 joetest01 systemd[1]: auditd.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- The unit auditd.service has entered the 'failed' state with result 'exit-code'.
Dec 02 20:22:49 joetest01 systemd[1]: Failed to start Security Auditing Service.
-- Subject: Unit auditd.service has failed
-- Defined-By: systemd
-- Unit auditd.service has failed.
-- The result is failed.
So the error gave me a clue that it might be related to SELinux security context. Check with “-Z” option of “ls” and it showed “unlabeled_t” instead of “auditd_log_t“
root@aheicsporaprd01:/var/log# ls -Zd /var/log/audit
system_u:object_r:unlabeled_t:s0 /var/log/audit
Restored the context of SELinux with the command “restorecon” and confirmed again:
root@aheicsporaprd01:/var/log# restorecon /var/log/audit
root@aheicsporaprd01:/var/log# ls -Zd /var/log/audit
system_u:object_r:auditd_log_t:s0 /var/log/audit
The service has started to work again:
root@joetest01:/var/log# systemctl start auditd
root@joetest01:/var/log# systemctl status auditd
● auditd.service - Security Auditing Service
Loaded: loaded (/usr/lib/systemd/system/auditd.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2021-12-02 20:27:28 EST; 5s ago
Docs: man:auditd(8)
https://github.com/linux-audit/audit-documentation
Process: 1701973 ExecStartPost=/sbin/augenrules --load (code=exited, status=1/FAILURE)
Process: 1701966 ExecStart=/sbin/auditd (code=exited, status=0/SUCCESS)
Main PID: 1701967 (auditd)
Tasks: 5 (limit: 1645960)
Memory: 2.9M
CGroup: /system.slice/auditd.service
├─1701967 /sbin/auditd
├─1701969 /sbin/audisp-syslog LOG_LOCAL6
└─1701970 /sbin/audisp-syslog LOG_LOCAL6
Dec 02 20:27:28 joetest01 audispd[1701970]: node=joetest01 type=SYSCALL msg=audit(1638494848.444:9360): arch=c000003e syscall=44 success=yes exit=1064 a0=3 a1=7ffd86465de0 a2=428 a3=0 items=0 ppid=170>
Dec 02 20:27:28 joetest01 audispd[1701969]: node=joetest01 type=SYSCALL msg=audit(1638494848.444:9360): arch=c000003e syscall=44 success=yes exit=1064 a0=3 a1=7ffd86465de0 a2=428 a3=0 items=0 ppid=170>
Dec 02 20:27:28 joetest01 audispd[1701970]: node=joetest01 type=SOCKADDR msg=audit(1638494848.444:9360): saddr=100000000000000000000000 SADDR={ saddr_fam=netlink nlnk-fam=16 nlnk-pid=0 }
Dec 02 20:27:28 joetest01 audispd[1701969]: node=joetest01 type=SOCKADDR msg=audit(1638494848.444:9360): saddr=100000000000000000000000 SADDR={ saddr_fam=netlink nlnk-fam=16 nlnk-pid=0 }
Dec 02 20:27:28 joetest01 audispd[1701970]: node=joetest01 type=PROCTITLE msg=audit(1638494848.444:9360): proctitle=2F7362696E2F617564697463746C002D52002F6574632F61756469742F61756469742E72756C6573
Dec 02 20:27:28 joetest01 audispd[1701969]: node=joetest01 type=PROCTITLE msg=audit(1638494848.444:9360): proctitle=2F7362696E2F617564697463746C002D52002F6574632F61756469742F61756469742E72756C6573
Dec 02 20:27:28 joetest01 audispd[1701970]: node=joetest01 type=EOE msg=audit(1638494848.444:9360):
Dec 02 20:27:28 joetest01 audispd[1701969]: node=joetest01 type=EOE msg=audit(1638494848.444:9360):
Dec 02 20:27:28 joetest01 audispd[1701970]: node=joetest01 type=SERVICE_START msg=audit(1638494848.446:9361): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=audi>
Dec 02 20:27:28 joetest01 audispd[1701969]: node=joetest01 type=SERVICE_START msg=audit(1638494848.446:9361): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=audi>
lines 1-25/25 (END)