Erst error record serialization table erst support is initialized

/var/log/messages reports ERST: Error Record Serialization Table (ERST) support is initialized (Doc ID 2012603.1)

Last updated on APRIL 26, 2021

Applies to:

Symptoms

A Platinum fault is reported:

Logfile Monitor found matches in (/var/log/messages) for pattern (kernel:.*(error|crit|fatal)) last count (5)

A search for such patterns shows following:

egrep -i «kernel:.*(error|crit|fatal)» /var/log/messages*

May 12 13:51:34 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 12 15:55:15 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 12 16:11:52 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 11 14:59:37 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 11 15:43:59 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.

Checking the var/log/messages file shows following:

May 12 13:51:34 localhost kernel: Booting paravirtualized kernel on bare hardware

Cause

To view full details, sign in with your My Oracle Support account.

Don’t have a My Oracle Support account? Click to get started!

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.

Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit oracle.com. пїЅ Oracle | Contact and Chat | Support | Communities | Connect with us | | | | Legal Notices | Terms of Use

Источник

CentOS

The Community ENTerprise Operating System

Server will not boot after reboot

Server will not boot after reboot

Post by python » 2017/01/14 17:34:05

When rebooting server does not boot, finding these errors do not know if that is why it does not boot.

Kernel: ERST: Error Record Serialization Table (ERST) is initialized.
Kernel: tpm_tis 00:06: TPM error (6) occurred attempting to read a pcr value
Kernel: tpm_tis 00: 0a: TPM error (6) occurred while attempting to read a pcr value

It is a centos 7.3

I’m new to linux, maybe I should do some testing or something?

Re: Server will not boot after reboot

Post by toracat » 2017/01/14 17:45:21

Re: Server will not boot after reboot

Post by python » 2017/01/14 22:36:02

Re: Server will not boot after reboot

Post by toracat » 2017/01/15 01:02:00

Re: Server will not boot after reboot

Post by python » 2017/01/15 10:27:08

Hello, how do I start the recovery mode with a live cd? I can not update the kernel, I see that I have installed another version of the one that uses. What is happening I do not understand. How should I upgrade?

] # uname -r
2.6.32-431.11.2.el6.x86_64
root @ server:

] # rpm -q kernel
Kernel-3.10.0-327.el7.x86_64
Kernel-3.10.0-327.10.1.el7.x86_64
Kernel-3.10.0-327.13.1.el7.x86_64
Kernel-3.10.0-327.36.2.el7.x86_64
Kernel-3.10.0-514.2.2.el7.x86_64
[Root @ cartamasajes

Re: Server will not boot after reboot

Post by toracat » 2017/01/16 01:21:50

Re: Server will not boot after reboot

Post by TrevorH » 2017/01/16 01:26:11

Re: Server will not boot after reboot

Post by python » 2017/01/16 17:09:08

]# uname -r
3.10.0-327.el7.x86_64
[root@server

]# rpm -q kernel
kernel-3.10.0-327.el7.x86_64
kernel-3.10.0-327.10.1.el7.x86_64
kernel-3.10.0-327.13.1.el7.x86_64
kernel-3.10.0-327.36.2.el7.x86_64
kernel-3.10.0-514.2.2.el7.x86_64
[root@server

]# cat /proc/mdstat
Personalities :
unused devices:
[root@server

]# uname -a
Linux server-1 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I’m afraid to restart maybe not boot
I should do all the steps I need, I’m a newbie.
thank you for your help

Re: Server will not boot after reboot

Post by python » 2017/01/21 22:59:45

Источник

Thread: Kubuntu keeps freezing

Thread Tools

Display

Kubuntu keeps freezing

Hi, recently, apparently one of the hard drives of my work machine failed (I had asked about that in this thread: https://ubuntuforums.org/showthread.php?t=2412024).
I now received back the machine from our IT, with a fresh Kubuntu 18.04.2 installed. Previously, I was using Xubuntu 16.04. So far Kubuntu is running very unstable. At seemingly random actions, sometimes opening a new tab in either Firefox or Chromium, sometimes upon starting VS Code, the whole GUI freezes, except for the cursor. I can freely move it around but everything else is completely frozen in time. I let it stay like that for half an hour once and nothing happened at all, the mouse reacts and the cursor looks normal, but the keyboard does nothing, only a hard reset helps.
After the last time I tried to search the logs for warnings and errors using this handy command recommended by a user in the above linked thread:

Here is the most recent log output:

The last entry is from 10:34 but the last freeze happened at about 15:10, so apparently nothing was logged. Seeing some earlier errors containing ata7 and the device name of my only hard drive sda1, could this freezing again stem from the disk? Unfortunately the IT guy is on holiday today so I can’t ask if he used a new drive or formatted the old one or whatever. But maybe this is some known Kubuntu quirk?

Re: Kubuntu keeps freezing

Whenever the gui seems to freeze, I don’t assume the entire OS is frozen, just the 1 program or the display subsystem. Check this by using ssh into the machine the next time it appears to freeze, then run ‘top’ to see how the system is doing. Be certain to setup ssh access BEFORE you need it. If you cannot ssh in while it is frozen, then that leads to the entire OS being locked up as a strong possibility.

Usually, when I’ve had slow response with a desktop, it is because that desktop is using all the RAM and all the virtual memory. The top command will show that, so will ‘free -hm’ and vmstat. When memory gets fully utilized, slowing everything down is how our OSes let us know we’ve asked too much from it. In general, a swap partition/file for a desktop should be 4.1GB in size unless you hibernate.

You still have disk and/or cable and/or hba controller problems to sort.

Источник

Common Problems Reported by Platinum Monitoring and Recommended Actions for Exalogic Systems (Doc ID 1985576.1)

Last updated on OCTOBER 10, 2022

Applies to:

Exalogic Elastic Cloud X6-2 Hardware — Version X6 to X6 [Release X6]
Exalogic Elastic Cloud X3-2 Eighth Rack — Version X3 to X3 [Release X3]
Exalogic Elastic Cloud X4-2 Quarter Rack — Version X4 to X4 [Release X4]
Exalogic Elastic Cloud X3-2 Hardware — Version X6 to X6 [Release X6]
Exalogic Elastic Cloud X4-2 Half Rack — Version X6 to X6 [Release X6]
Linux x86-64
Oracle Solaris on x86-64 (64-bit)
Oracle Virtual Server x86-64

Purpose

This Note provides list of commonly reported Platinum Fault and Alert messages by Platinum monitoring setup and provides recommended actions for addressing those Platinum Faults.

Scope

This note focuses on common Platinum alerts and solutions. For more information on Oracle Platinum Services including a full list of Fault Monitoring that is done, visit the following note:

Details

To view full details, sign in with your My Oracle Support account.

Don’t have a My Oracle Support account? Click to get started!

In this Document

PLATINUM FAULT: LogFileMonitor:sys_occurrence_count Scanned /var/log/messages from line xxxxx to yyyyy. Found 1 occurence of the pattern [kernel:.* (error|crit|fatal)].. 1 crossed warning ( ) or critical (0) threshold.

PLATINUM FAULT: LogFileMonitor:sys_occurrence_count Scanned /var/adm/messages from line xxxxx to yyyyy. Found 2 occurences of the pattern [svc.startd.*failed].. 2 crossed warning (0) or critical (1) threshold.

1. kernel: Error: Driver ‘pcspkr’ is already registered, aborting.

2. kernel: sdp_process_tx_wc:261 sdp_sock( 4551:14 58027:10280): Send completion with error. wr_id 0x400000002 Status 12

3. kernel: uce_agent.bin[23006]: segfault at f6eac05c ip 00000000082639a1 sp 00000000f6eac060 error 6 in uce_agent.bin[8048000+6e8000]

4. kernel: xs_tcp_setup_socket: connect returned unhandled error -107

5. kernel: bonding: bond1: Error: Unable to enslave eth326_2 because it is already up

6. yum-updatesd: error getting update info: Cannot retrieve repository metadata (repomd.xml) for repository: ol5_UEK_latest. Please verify its path and try again

7. kernel: cgrep[562]: segfault at 0 ip 000000004e7f7a3c sp 00000000ffc0ac4c error 4 in libc-2.5.so[4e79f000+154000]

8. kernel: ponu_ge_lstdat1[16101]: segfault at 28 ip 00007f5002b0a33d sp 00007fffe0fb6ee0 error 4 in libkitd.so[7f5002aea000+3f000]

9.В kernel: FNDLOAD[1296]: segfault at c ip 000000000805689a sp 00000000ff8340ec error 4 in FNDLOAD[8048000+111000]

10. kernel: ERST: Error Record Serialization Table (ERST) support is initialized

11. svc.startd[13]: [ID 652011 daemon.warning] svc:/application/pkg/system-repository:default: Method «/lib/svc/method/svc-pkg-sysrepo refresh» failed with exit status 95.
В В В В svc.startd[13]: [ID 748625 daemon.error] application/pkg/system-repository:default failed fatally: transitioned to maintenance (see ‘svcs -xv’ for details)

12. kernel: JSL[23982]: Segfault at 8 ip 000000000040e995 sp 00007ffd0047b880 error 4 in JSL[400000+20000]

13. kernel: ipmitool[10165]: segfault at 421 ip 000000000044a066 sp 00007ffd6ac4e2c0 error 4 in ipmitool[400000+78000]

14. kernel: Buffer I/O error on device dm-xx, logical block xx

15. kernel: tmipcrm[19586]: segfault at 7fe3ce38504e ip 00007xxxxx80633 sp 000xxxxx35b1abc0 error 4 in libtux.so[7fe3d761b000+276000]

PLATINUM FAULT: adrAlertLogIncidentError:accessViolationErrStack An access violation detected in /u01/app/oracle/diag/rdbms/elctrldb/elctrldb/alert/log.xml at time/line number: /

PLATINUM FAULT: adrAlertLogIncidentError:genericIncidentErrStack Incident (ORA 445) detected in /u01/app/oracle/diag/rdbms/elctrldb/elctrldb/alert/log.xml at time/line number: /

PLATINUM FAULT: adrAlertLogIncidentError:genericIncidentErrStack Incident (ORA 240) detected in /u01/app/oracle/diag/rdbms/elctrldb/elctrldb/alert/log.xml at time/line number: /

PLATINUM FAULT: Ilom Sensor Alerts: SensorAlerts:PowerSupplyStatus Power supply sensor(s) at level — CRITICAL

PLATINUM FAULT: ZFSProblem:ProblemSeverity AK-8003-Y6 : The device configuration for JBOD 1111FMD00X

PLATINUM FAULT: ZFSProblem:ProblemSeverity AK-8002-9M : The cable between the Ethernet ports of each controller is down

PLATINUM FAULT: ZFSProblem:ProblemSeverity ZFS-8000-D3 : ZFS device id1 sd@SATA_____TOSHIBA_THN

PLATINUM FAULT: ZFSProblem:ProblemSeverity USB-8000-GT : A hardware fault within the device or its interface was detected in the USB device. The driver has failed to initialize the device and the device is in an invalid state.

PLATINUM FAULT: ZFSProblem:ProblemSeverity DISK-8000-CY : There have been non-recovered ZFS checksum errors on this disk

PLATINUM FAULT: ZFSAlert:ProblemType All communication with the cluster peer has been lost

PLATINUM: An Integrated I/O (II0) fatal error in downstream PCIE device has occurred

PLATINUM FAULT: ZFSProblem:ProblemSeverity SUNOS-8000-KL : The system has rebooted after a kernel panic. Severity: Major Message ID: xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx

Exalogic Virtual: «cacao: Error: Fail to start cacao agent» Error Message Seen In /var/log/messages of EC Control vServer

Exalogic Virtual: Troubleshooting ORA-240 Errors

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.

Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit oracle.com. пїЅ Oracle | Contact and Chat | Support | Communities | Connect with us | | | | Legal Notices | Terms of Use

Источник

/var/log/messages reports ERST: Error Record Serialization Table (ERST) support is initialized (Doc ID 2012603.1)

Last updated on APRIL 26, 2021

Applies to:

Symptoms

A Platinum fault is reported:

Logfile Monitor found matches in (/var/log/messages) for pattern (kernel:.*(error|crit|fatal)) last count (5)

A search for such patterns shows following:

egrep -i «kernel:.*(error|crit|fatal)» /var/log/messages*

May 12 13:51:34 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 12 15:55:15 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 12 16:11:52 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 11 14:59:37 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.
May 11 15:43:59 localhost kernel: ERST: Error Record Serialization Table (ERST) support is initialized.

Checking the var/log/messages file shows following:

May 12 13:51:34 localhost kernel: Booting paravirtualized kernel on bare hardware

Cause

To view full details, sign in with your My Oracle Support account.

Don’t have a My Oracle Support account? Click to get started!

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.

Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit oracle.com. пїЅ Oracle | Contact and Chat | Support | Communities | Connect with us | | | | Legal Notices | Terms of Use

Источник

CentOS

The Community ENTerprise Operating System

Server will not boot after reboot

Server will not boot after reboot

Post by python » 2017/01/14 17:34:05

When rebooting server does not boot, finding these errors do not know if that is why it does not boot.

Kernel: ERST: Error Record Serialization Table (ERST) is initialized.
Kernel: tpm_tis 00:06: TPM error (6) occurred attempting to read a pcr value
Kernel: tpm_tis 00: 0a: TPM error (6) occurred while attempting to read a pcr value

It is a centos 7.3

I’m new to linux, maybe I should do some testing or something?

Re: Server will not boot after reboot

Post by toracat » 2017/01/14 17:45:21

Re: Server will not boot after reboot

Post by python » 2017/01/14 22:36:02

Re: Server will not boot after reboot

Post by toracat » 2017/01/15 01:02:00

Re: Server will not boot after reboot

Post by python » 2017/01/15 10:27:08

Hello, how do I start the recovery mode with a live cd? I can not update the kernel, I see that I have installed another version of the one that uses. What is happening I do not understand. How should I upgrade?

] # uname -r
2.6.32-431.11.2.el6.x86_64
root @ server:

] # rpm -q kernel
Kernel-3.10.0-327.el7.x86_64
Kernel-3.10.0-327.10.1.el7.x86_64
Kernel-3.10.0-327.13.1.el7.x86_64
Kernel-3.10.0-327.36.2.el7.x86_64
Kernel-3.10.0-514.2.2.el7.x86_64
[Root @ cartamasajes

Re: Server will not boot after reboot

Post by toracat » 2017/01/16 01:21:50

Re: Server will not boot after reboot

Post by TrevorH » 2017/01/16 01:26:11

Re: Server will not boot after reboot

Post by python » 2017/01/16 17:09:08

]# uname -r
3.10.0-327.el7.x86_64
[root@server

]# rpm -q kernel
kernel-3.10.0-327.el7.x86_64
kernel-3.10.0-327.10.1.el7.x86_64
kernel-3.10.0-327.13.1.el7.x86_64
kernel-3.10.0-327.36.2.el7.x86_64
kernel-3.10.0-514.2.2.el7.x86_64
[root@server

]# cat /proc/mdstat
Personalities :
unused devices:
[root@server

]# uname -a
Linux server-1 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I’m afraid to restart maybe not boot
I should do all the steps I need, I’m a newbie.
thank you for your help

Re: Server will not boot after reboot

Post by python » 2017/01/21 22:59:45

Источник

Thread: Kubuntu keeps freezing

Thread Tools

Display

Kubuntu keeps freezing

Hi, recently, apparently one of the hard drives of my work machine failed (I had asked about that in this thread: https://ubuntuforums.org/showthread.php?t=2412024).
I now received back the machine from our IT, with a fresh Kubuntu 18.04.2 installed. Previously, I was using Xubuntu 16.04. So far Kubuntu is running very unstable. At seemingly random actions, sometimes opening a new tab in either Firefox or Chromium, sometimes upon starting VS Code, the whole GUI freezes, except for the cursor. I can freely move it around but everything else is completely frozen in time. I let it stay like that for half an hour once and nothing happened at all, the mouse reacts and the cursor looks normal, but the keyboard does nothing, only a hard reset helps.
After the last time I tried to search the logs for warnings and errors using this handy command recommended by a user in the above linked thread:

Here is the most recent log output:

The last entry is from 10:34 but the last freeze happened at about 15:10, so apparently nothing was logged. Seeing some earlier errors containing ata7 and the device name of my only hard drive sda1, could this freezing again stem from the disk? Unfortunately the IT guy is on holiday today so I can’t ask if he used a new drive or formatted the old one or whatever. But maybe this is some known Kubuntu quirk?

Re: Kubuntu keeps freezing

Whenever the gui seems to freeze, I don’t assume the entire OS is frozen, just the 1 program or the display subsystem. Check this by using ssh into the machine the next time it appears to freeze, then run ‘top’ to see how the system is doing. Be certain to setup ssh access BEFORE you need it. If you cannot ssh in while it is frozen, then that leads to the entire OS being locked up as a strong possibility.

Usually, when I’ve had slow response with a desktop, it is because that desktop is using all the RAM and all the virtual memory. The top command will show that, so will ‘free -hm’ and vmstat. When memory gets fully utilized, slowing everything down is how our OSes let us know we’ve asked too much from it. In general, a swap partition/file for a desktop should be 4.1GB in size unless you hibernate.

You still have disk and/or cable and/or hba controller problems to sort.

Источник

Common Problems Reported by Platinum Monitoring and Recommended Actions for Exalogic Systems (Doc ID 1985576.1)

Last updated on OCTOBER 10, 2022

Applies to:

Exalogic Elastic Cloud X6-2 Hardware — Version X6 to X6 [Release X6]
Exalogic Elastic Cloud X3-2 Eighth Rack — Version X3 to X3 [Release X3]
Exalogic Elastic Cloud X4-2 Quarter Rack — Version X4 to X4 [Release X4]
Exalogic Elastic Cloud X3-2 Hardware — Version X6 to X6 [Release X6]
Exalogic Elastic Cloud X4-2 Half Rack — Version X6 to X6 [Release X6]
Linux x86-64
Oracle Solaris on x86-64 (64-bit)
Oracle Virtual Server x86-64

Purpose

This Note provides list of commonly reported Platinum Fault and Alert messages by Platinum monitoring setup and provides recommended actions for addressing those Platinum Faults.

Scope

This note focuses on common Platinum alerts and solutions. For more information on Oracle Platinum Services including a full list of Fault Monitoring that is done, visit the following note:

Details

To view full details, sign in with your My Oracle Support account.

Don’t have a My Oracle Support account? Click to get started!

In this Document

PLATINUM FAULT: LogFileMonitor:sys_occurrence_count Scanned /var/log/messages from line xxxxx to yyyyy. Found 1 occurence of the pattern [kernel:.* (error|crit|fatal)].. 1 crossed warning ( ) or critical (0) threshold.

PLATINUM FAULT: LogFileMonitor:sys_occurrence_count Scanned /var/adm/messages from line xxxxx to yyyyy. Found 2 occurences of the pattern [svc.startd.*failed].. 2 crossed warning (0) or critical (1) threshold.

1. kernel: Error: Driver ‘pcspkr’ is already registered, aborting.

2. kernel: sdp_process_tx_wc:261 sdp_sock( 4551:14 58027:10280): Send completion with error. wr_id 0x400000002 Status 12

3. kernel: uce_agent.bin[23006]: segfault at f6eac05c ip 00000000082639a1 sp 00000000f6eac060 error 6 in uce_agent.bin[8048000+6e8000]

4. kernel: xs_tcp_setup_socket: connect returned unhandled error -107

5. kernel: bonding: bond1: Error: Unable to enslave eth326_2 because it is already up

6. yum-updatesd: error getting update info: Cannot retrieve repository metadata (repomd.xml) for repository: ol5_UEK_latest. Please verify its path and try again

7. kernel: cgrep[562]: segfault at 0 ip 000000004e7f7a3c sp 00000000ffc0ac4c error 4 in libc-2.5.so[4e79f000+154000]

8. kernel: ponu_ge_lstdat1[16101]: segfault at 28 ip 00007f5002b0a33d sp 00007fffe0fb6ee0 error 4 in libkitd.so[7f5002aea000+3f000]

9.В kernel: FNDLOAD[1296]: segfault at c ip 000000000805689a sp 00000000ff8340ec error 4 in FNDLOAD[8048000+111000]

10. kernel: ERST: Error Record Serialization Table (ERST) support is initialized

11. svc.startd[13]: [ID 652011 daemon.warning] svc:/application/pkg/system-repository:default: Method «/lib/svc/method/svc-pkg-sysrepo refresh» failed with exit status 95.
В В В В svc.startd[13]: [ID 748625 daemon.error] application/pkg/system-repository:default failed fatally: transitioned to maintenance (see ‘svcs -xv’ for details)

12. kernel: JSL[23982]: Segfault at 8 ip 000000000040e995 sp 00007ffd0047b880 error 4 in JSL[400000+20000]

13. kernel: ipmitool[10165]: segfault at 421 ip 000000000044a066 sp 00007ffd6ac4e2c0 error 4 in ipmitool[400000+78000]

14. kernel: Buffer I/O error on device dm-xx, logical block xx

15. kernel: tmipcrm[19586]: segfault at 7fe3ce38504e ip 00007xxxxx80633 sp 000xxxxx35b1abc0 error 4 in libtux.so[7fe3d761b000+276000]

PLATINUM FAULT: adrAlertLogIncidentError:accessViolationErrStack An access violation detected in /u01/app/oracle/diag/rdbms/elctrldb/elctrldb/alert/log.xml at time/line number: /

PLATINUM FAULT: adrAlertLogIncidentError:genericIncidentErrStack Incident (ORA 445) detected in /u01/app/oracle/diag/rdbms/elctrldb/elctrldb/alert/log.xml at time/line number: /

PLATINUM FAULT: adrAlertLogIncidentError:genericIncidentErrStack Incident (ORA 240) detected in /u01/app/oracle/diag/rdbms/elctrldb/elctrldb/alert/log.xml at time/line number: /

PLATINUM FAULT: Ilom Sensor Alerts: SensorAlerts:PowerSupplyStatus Power supply sensor(s) at level — CRITICAL

PLATINUM FAULT: ZFSProblem:ProblemSeverity AK-8003-Y6 : The device configuration for JBOD 1111FMD00X

PLATINUM FAULT: ZFSProblem:ProblemSeverity AK-8002-9M : The cable between the Ethernet ports of each controller is down

PLATINUM FAULT: ZFSProblem:ProblemSeverity ZFS-8000-D3 : ZFS device id1 sd@SATA_____TOSHIBA_THN

PLATINUM FAULT: ZFSProblem:ProblemSeverity USB-8000-GT : A hardware fault within the device or its interface was detected in the USB device. The driver has failed to initialize the device and the device is in an invalid state.

PLATINUM FAULT: ZFSProblem:ProblemSeverity DISK-8000-CY : There have been non-recovered ZFS checksum errors on this disk

PLATINUM FAULT: ZFSAlert:ProblemType All communication with the cluster peer has been lost

PLATINUM: An Integrated I/O (II0) fatal error in downstream PCIE device has occurred

PLATINUM FAULT: ZFSProblem:ProblemSeverity SUNOS-8000-KL : The system has rebooted after a kernel panic. Severity: Major Message ID: xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx

Exalogic Virtual: «cacao: Error: Fail to start cacao agent» Error Message Seen In /var/log/messages of EC Control vServer

Exalogic Virtual: Troubleshooting ORA-240 Errors

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.

Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit oracle.com. пїЅ Oracle | Contact and Chat | Support | Communities | Connect with us | | | | Legal Notices | Terms of Use

Источник

The Linux kernel’s pstore error log capturing system, and ACPI ERST

January 25, 2019

In response to my entry yesterday on enabling reboot on panic on
your servers, a commentator left the succinct
suggestion of ‘setup pstore’. I had never heard of pstore before,
so this sent me searching and what I found is actually quite
interesting and surprising, with direct relevance to quite a few
of our servers.

Pstore itself is a kernel feature that dates to 2011. It provides
a generic interface to storage that persists across reboots and
gets used to save kernel messages during a crash, as covered in
LWN’s Persistent storage for a kernel’s «dying breath» and the kernel documentation. Your
kernel very likely has pstore built in and your Linux probably
mounts the pstore filesystem at /sys/fs/pstore.

(The Ubuntu 16.04 and 18.04 kernels, the CentOS 7 kernel, and the
Fedora kernel all have it built in. If in doubt, check your kernel’s
configuration, which is often found in /boot/conf-*; you’re looking
for CONFIG_PSTORE and associated things.)

By itself, pstore does nothing for you because it needs a chunk of
storage that persists across reboots, and that’s up to your system
to provide in some way. One such source of this storage is in an
optional part of ACPI
called the Error Record Serialization Table (ERST). Not all machines
have an ERST (it’s apparently most common in servers), but if you
do have one, pstore will probably automatically use it.
If you have ERST at all, it will normally show up in the kernel’s
boot time messages about ACPI:

ACPI: ERST 0x00000000BF7D6000 000230 (v01 DELL   PE_SC3   00000000 DELL 00040000)

If pstore is using ERST, you will get some additional kernel messages:

ERST: Error Record Serialization Table (ERST) support is initialized.
pstore: using zlib compression
pstore: Registered erst as persistent store backend

Some of our servers have ACPI ERST and some of them have crashed,
so out of idle curiosity I went and looked at /sys/fs/pstore on
all of them. This led to a big surprise, which is that there may
be nothing in your Linux distribution that checks /sys/fs/pstore
to see if there are captured kernel crash logs. Pstore is
persistent storage, and so it does what it says on the can; if
you don’t move things out of /sys/fs/pstore, they stay there,
possibly for a very long time (one of our servers turned out to
have pstore ERST captures from a year ago). This is especially
important because things like ERST only have so much space, so
lingering old crash logs may keep you from saving new ones, ones
that you may discover you very much would like records of.

(The year-old pstore ERST captures are especially ironic because
the machine’s current incarnation was reinstalled this September,
so they are from its previous life as something else entirely,
making them completely useless to us.)

Another pstore backend that you may have on some machines is one
that uses UEFI variables. Unfortunately, you need to have booted
your system using UEFI in order to have access to UEFI services,
including UEFI variables (as I found out the hard way once), so even on a UEFI-capable system you may
not be able to use this backend because you’re still using MBR
booting. It’s possible that using UEFI variables for pstore is
disabled by some Linux distributions, since actually using UEFI
variables has caused UEFI BIOS problems in the past.

(This makes it somewhat more of a pity that I failed to migrate
to UEFI booting, since I would actually
potentially get something out of it on my workstations. Also,
although many of our servers are probably UEFI capable, they all
use MBR booting today.)

Given that nothing in our Ubuntu 18.04 server installs seems to
notice /sys/fs/pstore and we have some machines with things in
it, we’re probably going to put together some shell scripting of
our own to at least email us if something shows up.

(Additional references: Matthew Garrett’s A use for EFI, CoreOS’s Collecting
crash logs,
which mentions the need to clear out /sys/fs/pstore, and abrt’s
pstore oops wiki page,
which includes a list of pstore backends.)

PS: The awkward, brute force way to get pstore space is with the
ramoops backend,
which requires fencing off some section of your RAM from your kernel
(it should be RAM that your BIOS won’t clear on reboot for whatever
reason). This is beyond my enthusiasm level on my machines, despite
some recent problems, and I
have the impression that ramoops is usually used on embedded ARM
hardware where you have little or no other options.