Error in open session response message insufficient resources for session

Running v1.99.3-32-g05b7eba (05b7eba), I ran the following. I was told that we need to restart the ipmi network daemon due to #1110, so I did that, and it still didn't work. The full journal is...

Running v1.99.3-32-g05b7eba (05b7eba), I ran the following. I was told that we need to restart the ipmi network daemon due to #1110, so I did that, and it still didn’t work.

The full journal is attached.

$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis status
Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis on
Error in open session response message : insufficient resources for session

Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis on
Error in open session response message : insufficient resources for session

Error: Unable to establish IPMI v2 / RMCP+ session
joel@ka1 ~ 
$ ipmitool -H palm5-bmc -I lanplus -P PASSW0RD chassis on
Error in open session response message : insufficient resources for session
Mar 03 00:01:53 palmetto netipmid[810]: I> Channel::Read : DataIn Fd[3] Req[17] Recv[17]
Mar 03 00:01:53 palmetto netipmid[810]: >> GetChannelCapabilities
Mar 03 00:01:53 palmetto netipmid[810]: << GetChannelCapabilities
Mar 03 00:01:53 palmetto netipmid[810]: I> Channel::Read : DataIn Fd[3] Req[30] Recv[30]
Mar 03 00:01:53 palmetto netipmid[810]: >> openSession
Mar 03 00:01:53 palmetto netipmid[810]: E> No free sessions left: Active: 5 Allowed: 5
Mar 03 00:01:53 palmetto netipmid[810]: E> Active Session: 0x00000000
Mar 03 00:01:53 palmetto netipmid[810]: E> Active Session: 0x100a3391
Mar 03 00:01:53 palmetto netipmid[810]: E> Active Session: 0x2f33acb3
Mar 03 00:01:53 palmetto netipmid[810]: E> Active Session: 0x41ff21ed
Mar 03 00:01:53 palmetto netipmid[810]: E> Active Session: 0x4a556d18
Mar 03 00:01:53 palmetto netipmid[810]: E> Active Session: 0x75682194
Mar 03 00:01:53 palmetto netipmid[810]: No free sessions left
Mar 03 00:01:53 palmetto netipmid[810]: openSession : Problem opening a session

I’ve set 3 items for my server to get sensor data each 30 seconds but it Frequently changed to not supported
zabbix_server.log:
8042:20111025:103124.814 Enabling IPMI host [zproxy01]
8282:20111025:103134.311 IPMI Host [zproxy01]: first network error, wait for 15 seconds
8281:20111025:103141.545 Disabling IPMI host [zproxy01]
8130:20111025:103225.023 Enabling IPMI host [zabbix]
8290:20111025:103242.032 IPMI Host [zabbix]: first network error, wait for 15 seconds
8295:20111025:103336.089 IPMI Host [zabbix]: first network error, wait for 15 seconds
8291:20111025:103342.064 Disabling IPMI host [zabbix]
8123:20111025:103456.984 Enabling IPMI host [zabbix]
8102:20111025:103501.839 Enabling IPMI host [zproxy01]
8287:20111025:103539.066 IPMI Host [zproxy01]: first network error, wait for 15 seconds
8289:20111025:103540.094 IPMI Host [zproxy01]: another network error, wait for 15 secon

when it changed to not supported i had check with manual «ipmitool -U zabbix -H 192.168.XX.XX -I lanplus -L ADMINISTRATOR sensor»
i’ve got the following message:
Error in open session response message : insufficient resources for session

Error: Unable to establish IPMI v2 / RMCP+ session

also the result of «ipmitool -U zabbix -H 192.168.XX.XX -I lanplus -L ADMINISTRATOR session info all» maybe usefull (at the next possible time)
session handle : 1
slot count : 4
active sessions : 60
user id : 2
privilege level : ADMINISTRATOR
session type : IPMIv2/RMCP+
channel number : 0x02
console ip : 192.168.XX.XX
console mac : 00:00:00:00:00:00
console port : 4040

session handle : 2
slot count : 4
active sessions : 60
user id : 2
privilege level : ADMINISTRATOR
session type : IPMIv2/RMCP+
channel number : 0x02
console ip : 192.168.XX.XX
console mac : 00:00:00:00:00:00
console port : 59029

session handle : 0
slot count : 4
active sessions : 60

session handle : 4
slot count : 4
active sessions : 60
user id : 2
privilege level : ADMINISTRATOR
session type : IPMIv2/RMCP+
channel number : 0x02
console ip : 192.168.XX.XX
console mac : 00:00:00:00:00:00
console port : 25072

  • Summary

  • Files

  • Reviews

  • Support

  • Wiki

  • Mailing Lists

  • Tickets ▾

    • Support Requests
    • Patches
    • Feature Requests
    • Bugs
  • News

  • Discussion

  • Code-svn

  • Code-git

Menu

Minor Bug in 3.14 and Error -1 Meaning /Diagnosis?


Created:

2019-12-10

Updated:

2019-12-12

  • Patrick Schmidt

    Hello Andy 

    First i just wanted to inform you, that i found a possible bug in the latest (3.14) version of ipmiutil. Could not test the new 3.15 . On Windows if you use the driver type lan2 it crashes (the process). If I use 3.12 lan2 works perfect. Just want to let you know.

    Now I got a question regarding “error -1”. What are the most common causes of this? I attached an report of a remote system where no data is returned, instead with all drv types (lan2, ms , etc) i just get an error -1.
    I attached an debug output with –x.

    Perhaps you have an idea what the cause could be!! (It is an Intel BMC by the way)

    THANK YOU :)

  • Andy Cress

    This is the key message from the lanplus code:
    Error in open session response message : insufficient resources for session
    I’m thinking there may be something wrong with the linkage to openssl in 3.14.

  • Patrick Schmidt

    The error i get with -1 is also happening in version 3.12 .

  • Andy Cress

    The lan2/lanplus logic is the only place where openssl is used, not in lan 1.x.
    Are those 3.14 and 3.12 errors on different systems, or the same system?
    I ran a similar test just now with 3.15 just fine, but that has new openssl DLLs.
    I’m wondering if somehow there are more than one set of the openssl DLLs when this occurs:
    (ssleay32.dll, libeay32.dll). It is possible that some other program includes/installs openssl also.
    I will run some additonal tests with 3.14 and 3.12.

  • Patrick Schmidt

    Oh sorry i mixed seperate issues together. So 3.14 crashed. I guess you are right, this could be due to a bad openssl linking.

    The other question was asked version independent. I got one esx system which Intel BMC is returning this -1 error and the log is attached in the start post. Nothing to do with the 3.14 crash.

    3.12 version works fine for nearly every system. I just wanted to ask if you have any idea where the problem could lie with the -1 and the -x log. Bad configuration? Network Problems?

    :)

  • Andy Cress

    For the resource error, the log message it shows indicates that the remote firmware returned status code 0x01. Ipmiutil reports the error message, and the -1 error status.
    From section 13.24, table 13 of the IPMI Spec, it gives this info:
    Status | Description |
    01h | Insufficient resources to create a session |

    So the ‘resources’ here would be firmware resources. I believe you said the target is an ESX server, so it may be difficult to talk to the firmware from a local/driver interface. The firmware should also have a web interface accessible remotely. I’m guessing that you cannot log into that system firmware from its web interface either (similar error message).
    If so, the firmware has somehow consumed its available resources (odd that it didn’t clean itself up?), and we need to restart the firmware. Since we don’t have a local interface via ESX, the only way to restart the firmware would be to pull input power for 10 seconds.

    • Patrick Schmidt

      Hey :) The Intel BMC has its own web interface and from what i have heard, the status is shown there perfectly fine! I will keep that issue (with resources) in mind and will report it back. Thank you :)

  • Andy Cress

    You did mention looking at the IPMI LAN configuration for that system firmware. I don’t think this error message relates to a configuration problem, but without local access, it is difficult to look at that anyway. In fact, for an ESX system, without IPMI LAN access, it would need to be examined from either boot media or from the BIOS menus (scheduled outage).

  • Andy Cress

    Interesting. So from the BMC web interface, you should be able to view some of the IPMI LAN configuration info. Perhaps it needs to be configured for more lanplus resources.

  • Patrick Schmidt

    Okay i will try to look further into this!


Log in to post a comment.

While testing with current ironic master branch, Upon attempting to deploy, utilizing ironic to deploy a number of nodes (ten to thirty five nodes), on a high density hardware chassis (HP Moonshot) that utilizes a dual bridge IPMI bus where the only difference between the nodes is the MAC address and the bridging target number. We observed a number of nodes that failed to deploy due to ipmitool returning «Error in open session response message : insufficient resources for session»

Apparent root cause:

_exec_ipmitool does not have retry logic logic wrapped around the execution of the script and instead the parameters are passed to the command line of ipmitool. In essence, ipmitool and Ironic presently fail hard when the BMC says «give me a minute».

How to reproduce:

Have ironic running in a stand-alone environment with no other openstack services.
Attempt to deploy, in-mass, to a number of nodes on that utilize dual ipmi_bridging. This is being orchustrated by python code utilizing the python-ironicclient library setting the node instance_info, and then setting the node to an active state, in series, however not waiting for any of the operations or nodes to reach an active state.

Attempting to tune the ipmi settings, specifically increasing the min_command_interval resulted in the deployments to fail to obtain node locks as tasks were taking longer to complete. Increasing retries did not appear to remedy this issue.

Ironic conductor log output for one of the nodes that failed to deploy in this manor:
2015-03-12 19:20:30.699 4307 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): ipmitool -I lanplus -H 10.0.1.5 -L ADMINISTRATOR -U administrator -B 0 -T 160 -b 7 -t 114 -R 12 -N 5 -f /tmp/tmpgaadM
r chassis bootdev disk options=persistent execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:199
2015-03-12 19:20:30.766 4307 DEBUG oslo_concurrency.processutils [-] CMD «ipmitool -I lanplus -H 10.0.1.5 -L ADMINISTRATOR -U administrator -B 0 -T 158 -b 7 -t 114 -R 12 -N 5 -f /tmp/tmp3uQDcS chassis bootdev dis
k options=persistent» returned: 0 in 0.568s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:225
2015-03-12 19:20:30.769 4307 DEBUG ironic.common.utils [-] Execution completed, command line is «ipmitool -I lanplus -H 10.0.1.5 -L ADMINISTRATOR -U administrator -B 0 -T 158 -b 7 -t 114 -R 12 -N 5 -f /tmp/tmp3uQ
DcS chassis bootdev disk options=persistent» execute /usr/local/lib/python2.7/dist-packages/ironic/common/utils.py:83
2015-03-12 19:20:30.772 4307 DEBUG ironic.common.utils [-] Command stdout is: «Set Boot Device to disk
» execute /usr/local/lib/python2.7/dist-packages/ironic/common/utils.py:84
2015-03-12 19:20:30.775 4307 DEBUG ironic.common.utils [-] Command stderr is: «» execute /usr/local/lib/python2.7/dist-packages/ironic/common/utils.py:85
2015-03-12 19:20:30.885 4307 DEBUG oslo_concurrency.processutils [-] CMD «ipmitool -I lanplus -H 10.0.1.5 -L ADMINISTRATOR -U administrator -B 0 -T 160 -b 7 -t 114 -R 12 -N 5 -f /tmp/tmpgaadMr chassis bootdev dis
k options=persistent» returned: 1 in 0.185s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:225
2015-03-12 19:20:30.888 4307 DEBUG oslo_concurrency.processutils [-] u’ipmitool -I lanplus -H 10.0.1.5 -L ADMINISTRATOR -U administrator -B 0 -T 160 -b 7 -t 114 -R 12 -N 5 -f /tmp/tmpgaadMr chassis bootdev disk o
ptions=persistent’ failed. Not Retrying. execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:258
2015-03-12 19:20:30.891 4307 WARNING ironic.drivers.modules.ipmitool [-] IPMI set boot device failed for node a8cb6624-0d9f-c882-affc-046ebb96ec16 when executing «ipmitool chassis bootdev disk options=persistent». Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 10.0.1.5 -L ADMINISTRATOR -U administrator -B 0 -T 160 -b 7 -t 114 -R 12 -N 5 -f /tmp/tmpgaadMr chassis bootdev disk options=persistent
Exit code: 1
Stdout: u»
Stderr: u’Error in open session response message : insufficient resources for sessionnnError: Unable to establish IPMI v2 / RMCP+ sessionnError setting Chassis Boot Parameter 0nError setting Chassis Boot Parameter 4n’
2015-03-12 19:20:30.897 4307 ERROR ironic.drivers.modules.agent_base_vendor [-] Asynchronous exception for node a8cb6624-0d9f-c882-affc-046ebb96ec16: Node failed to move to active state. exception: IPMI call failed: chassis bootdev disk options=persistent.
ed: chassis bootdev disk options=persistent.
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor Traceback (most recent call last):
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor File «/usr/local/lib/python2.7/dist-packages/ironic/drivers/modules/agent_base_vendor.py», line 169, in heartbeat
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor self.reboot_to_instance(task, **kwargs)
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor File «/usr/local/lib/python2.7/dist-packages/ironic/drivers/modules/agent.py», line 378, in reboot_to_instance
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor manager_utils.node_set_boot_device(task, ‘disk’, persistent=True)
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor File «/usr/local/lib/python2.7/dist-packages/ironic/conductor/task_manager.py», line 128, in wrapper
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor return f(*args, **kwargs)
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor File «/usr/local/lib/python2.7/dist-packages/ironic/conductor/utils.py», line 44, in node_set_boot_device
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor persistent=persistent)
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor File «/usr/local/lib/python2.7/dist-packages/ironic/conductor/task_manager.py», line 128, in wrapper
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor return f(*args, **kwargs)
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor File «/usr/local/lib/python2.7/dist-packages/ironic/drivers/modules/ipmitool.py», line 745, in set_boot_device
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor raise exception.IPMIFailure(cmd=cmd)
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor IPMIFailure: IPMI call failed: chassis bootdev disk options=persistent.
2015-03-12 19:20:30.897 4307 TRACE ironic.drivers.modules.agent_base_vendor

ironic node-list:
+—————————————+—————+————-+———————+————-+
| UUID | Instance UUID | Power State | Provisioning State | Maintenance |
+—————————————+—————+————-+———————+————-+
| a8cb6624-0d9f-c882-affc-046ebb96ec10 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec11 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec12 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec13 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec14 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec15 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec16 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec17 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec18 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec19 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec20 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec21 | None | power on | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec22 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec23 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec24 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec25 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec26 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec27 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec28 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec29 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec30 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec31 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec32 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec33 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec34 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec35 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec36 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec37 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec38 | None | power off | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec39 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec40 | None | power on | deploy failed | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec41 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec42 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec43 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec44 | None | power on | active | False |
| a8cb6624-0d9f-c882-affc-046ebb96ec45 | None | power on | active | False |
+—————————————+—————+————-+———————+————-+

ironic node-show nodes 16, 17, 20:
+————————+—————————————————————————+
| Property | Value |
+————————+—————————————————————————+
| instance_uuid | None |
| target_power_state | None |
| properties | {u’memory_mb’: u’8160′, u’cpu_arch’: u’x86_64′, u’local_gb’: u’450′, |
| | u’cpus’: u’3′} |
| maintenance | False |
| driver_info | {u’ipmi_transit_address’: u’160′, u’ipmi_target_channel’: u’7′, |
| | u’ipmi_transit_channel’: u’0′, u’deploy_kernel’: |
| | u’http://10.99.1.1:8080/coreos_production_pxe.vmlinuz’, |
| | u’ipmi_username’: u’administrator’, u’ipmi_address’: u’10.0.1.5′, |
| | u’ipmi_target_address’: u’114′, u’ipmi_password’: u’******’, |
| | u’ipmi_bridging’: u’dual’, u’deploy_ramdisk’: u’http://10.99.1.1:8080 |
| | /coreos_production_pxe_image-oem.cpio.gz’} |
| extra | {} |
| last_error | Asynchronous exception for node a8cb6624-0d9f-c882-affc-046ebb96ec16: |
| | Node failed to move to active state. exception: IPMI call failed: |
| | chassis bootdev disk options=persistent. |
| created_at | 2015-03-03T23:17:35+00:00 |
| target_provision_state | active |
| driver | agent_ipmitool |
| updated_at | 2015-03-12T19:54:52+00:00 |
| maintenance_reason | None |
| instance_info | {u’root_gb’: 10, u’image_source’: |
| | u’http://10.99.1.1:8080/expemental.qcow2′, u’image_checksum’: |
| | u’7f76ce7cbff05f0254fd78d7bcaf3b45′, u’image_url’: |
| | u’http://10.99.1.1:8080/expemental.qcow2′, u’image_disk_format’: u’raw’, |
| | u’configdrive’: u’http://10.99.1.1:8080/configdrive-a8cb6624-0d9f-c882 |
| | -affc-046ebb96ec16.iso.gz’} |
| driver_internal_info | |
| chassis_uuid | |
| provision_state | deploy failed |
| reservation | None |
| power_state | power off |
| console_enabled | False |
| uuid | a8cb6624-0d9f-c882-affc-046ebb96ec16 |
+————————+—————————————————————————+
+————————+—————————————————————————+
| Property | Value |
+————————+—————————————————————————+
| instance_uuid | None |
| target_power_state | None |
| properties | {u’memory_mb’: u’8160′, u’cpu_arch’: u’x86_64′, u’local_gb’: u’450′, |
| | u’cpus’: u’3′} |
| maintenance | False |
| driver_info | {u’ipmi_transit_address’: u’162′, u’ipmi_target_channel’: u’7′, |
| | u’ipmi_transit_channel’: u’0′, u’deploy_kernel’: |
| | u’http://10.99.1.1:8080/coreos_production_pxe.vmlinuz’, |
| | u’ipmi_username’: u’administrator’, u’ipmi_address’: u’10.0.1.5′, |
| | u’ipmi_target_address’: u’114′, u’ipmi_password’: u’******’, |
| | u’ipmi_bridging’: u’dual’, u’deploy_ramdisk’: u’http://10.99.1.1:8080 |
| | /coreos_production_pxe_image-oem.cpio.gz’} |
| extra | {} |
| last_error | Failed to deploy. Error: IPMI call failed: raw 0x00 0x08 0x03 0x08. |
| created_at | 2015-03-03T23:17:41+00:00 |
| target_provision_state | active |
| driver | agent_ipmitool |
| updated_at | 2015-03-12T20:07:00+00:00 |
| maintenance_reason | None |
| instance_info | {u’root_gb’: 10, u’image_source’: |
| | u’http://10.99.1.1:8080/expemental.qcow2′, u’image_checksum’: |
| | u’7f76ce7cbff05f0254fd78d7bcaf3b45′, u’image_url’: |
| | u’http://10.99.1.1:8080/expemental.qcow2′, u’image_disk_format’: u’raw’, |
| | u’configdrive’: u’http://10.99.1.1:8080/configdrive-a8cb6624-0d9f-c882 |
| | -affc-046ebb96ec17.iso.gz’} |
| driver_internal_info | |
| chassis_uuid | |
| provision_state | deploy failed |
| reservation | None |
| power_state | power off |
| console_enabled | False |
| uuid | a8cb6624-0d9f-c882-affc-046ebb96ec17 |
+————————+—————————————————————————+
+————————+—————————————————————————+
| Property | Value |
+————————+—————————————————————————+
| instance_uuid | None |
| target_power_state | None |
| properties | {u’memory_mb’: u’8160′, u’cpu_arch’: u’x86_64′, u’local_gb’: u’450′, |
| | u’cpus’: u’3′} |
| maintenance | False |
| driver_info | {u’ipmi_transit_address’: u’168′, u’ipmi_target_channel’: u’7′, |
| | u’ipmi_transit_channel’: u’0′, u’deploy_kernel’: |
| | u’http://10.99.1.1:8080/coreos_production_pxe.vmlinuz’, |
| | u’ipmi_username’: u’administrator’, u’ipmi_address’: u’10.0.1.5′, |
| | u’ipmi_target_address’: u’114′, u’ipmi_password’: u’******’, |
| | u’ipmi_bridging’: u’dual’, u’deploy_ramdisk’: u’http://10.99.1.1:8080 |
| | /coreos_production_pxe_image-oem.cpio.gz’} |
| extra | {} |
| last_error | Asynchronous exception for node a8cb6624-0d9f-c882-affc-046ebb96ec20: |
| | Node failed to move to active state. exception: IPMI call failed: raw |
| | 0x00 0x08 0x03 0x08. |
| created_at | 2015-03-12T19:00:06+00:00 |
| target_provision_state | active |
| driver | agent_ipmitool |
| updated_at | 2015-03-12T20:07:15+00:00 |
| maintenance_reason | None |
| instance_info | {u’root_gb’: 10, u’image_source’: |
| | u’http://10.99.1.1:8080/expemental.qcow2′, u’image_checksum’: |
| | u’7f76ce7cbff05f0254fd78d7bcaf3b45′, u’image_url’: |
| | u’http://10.99.1.1:8080/expemental.qcow2′, u’image_disk_format’: u’raw’, |
| | u’configdrive’: u’http://10.99.1.1:8080/configdrive-a8cb6624-0d9f-c882 |
| | -affc-046ebb96ec20.iso.gz’} |
| driver_internal_info | |
| chassis_uuid | |
| provision_state | deploy failed |
| reservation | None |
| power_state | power off |
| console_enabled | False |
| uuid | a8cb6624-0d9f-c882-affc-046ebb96ec20 |
+————————+—————————————————————————+

I have a problem with our Power8 box. I can no longer use the IPMI interface — every command I issue even simple read only queries like «chassis status» result in the following output:

$ /opt/local/bin/ipmitool -I lanplus -H 192.168.70.147 -P admin chassis status
Error in open session response message : insufficient resources for session

Error: Unable to establish IPMI v2 / RMCP+ session
Error sending Chassis Status command

The issue with this is that I can no longer access the SOL interface. Any ideas how I get the box out of this state? I’ve tried power cycling — next idea is to reset to factory defaults.

Jeremy Kerr's user avatar

asked Aug 29, 2014 at 12:28

ppc64's user avatar

Access to IPMI is mutually exclusive between the PowerVM and PowerKVM enabled modes.

As seen, IPMI will work just fine in the OPAL hypervisor mode, PowerVM mode uses a different firmware load.

There’s an emerging page for IPMI considerations over on the DeveloperWorks realm.

There’s also some information on IPMI in the PowerKVM Redbook.

answered Aug 29, 2014 at 18:07

Bill Buros's user avatar

Skip to navigation
Skip to main content

Red Hat Customer Portal

Infrastructure and Management

  • Red Hat Enterprise Linux

  • Red Hat Virtualization

  • Red Hat Identity Management

  • Red Hat Directory Server

  • Red Hat Certificate System

  • Red Hat Satellite

  • Red Hat Subscription Management

  • Red Hat Update Infrastructure

  • Red Hat Insights

  • Red Hat Ansible Automation Platform

Cloud Computing

  • Red Hat OpenShift

  • Red Hat CloudForms

  • Red Hat OpenStack Platform

  • Red Hat OpenShift Container Platform

  • Red Hat OpenShift Data Science

  • Red Hat OpenShift Online

  • Red Hat OpenShift Dedicated

  • Red Hat Advanced Cluster Security for Kubernetes

  • Red Hat Advanced Cluster Management for Kubernetes

  • Red Hat Quay

  • OpenShift Dev Spaces

  • Red Hat OpenShift Service on AWS

Storage

  • Red Hat Gluster Storage

  • Red Hat Hyperconverged Infrastructure

  • Red Hat Ceph Storage

  • Red Hat OpenShift Data Foundation

Runtimes

  • Red Hat Runtimes

  • Red Hat JBoss Enterprise Application Platform

  • Red Hat Data Grid

  • Red Hat JBoss Web Server

  • Red Hat Single Sign On

  • Red Hat support for Spring Boot

  • Red Hat build of Node.js

  • Red Hat build of Thorntail

  • Red Hat build of Eclipse Vert.x

  • Red Hat build of OpenJDK

  • Red Hat build of Quarkus

Integration and Automation

  • Red Hat Process Automation

  • Red Hat Process Automation Manager

  • Red Hat Decision Manager

All Products

Issue

  • Ironic, using the ipmitool driver, queries IPMI so often that IPMI runs out of it’s (16) sessions
  • Ironic timeouts were tweaked, which have improved things, but the following error is still ocurring:
"Unable to establish IPMI v2 / RMCP+ session" 
  • The following error message will be displayed in /var/log/ironic/ironic-conductor.log:
Sep 25 17:08:01 hostname01 ironic-conductor: 2015-09-25 17:08:01.817 1451 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): ipmitool -I lanplus -H 10.10.10.10 -L ADMINISTRATOR -U ADMIN -R 6 -N 15 -f /tmp/tmpCeEmxP power status execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:223
Sep 25 17:08:01 hostname01 ironic-conductor: 2015-09-25 17:08:01.828 1451 DEBUG oslo_concurrency.processutils [-] CMD "ipmitool -I lanplus -H 10.10.10.10 -L ADMINISTRATOR -U ADMIN -R 6 -N 15 -f /tmp/tmpCeEmxP power status" returned: 1 in 0.011s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:254
Sep 25 17:08:01 hostname01 ironic-conductor: 2015-09-25 17:08:01.829 1451 DEBUG oslo_concurrency.processutils [-] u'ipmitool -I lanplus -H 10.10.10.10 -L ADMINISTRATOR -U ADMIN -R 6 -N 15 -f /tmp/tmpCeEmxP power status' failed. Not Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:291
Sep 25 17:08:01 hostname01 ironic-conductor: 2015-09-25 17:08:01.829 1451 WARNING ironic.drivers.modules.ipmitool [-] IPMI Error encountered, retrying "ipmitool -I lanplus -H 10.10.10.10 -L ADMINISTRATOR -U ADMIN -R 6 -N 15 -f /tmp/tmpCeEmxP power status" for node 74a06170-4375-48a4-b9ac-5c194ab35e57. Error: Unexpected error while running command.
Sep 25 17:08:01 hostname01 ironic-conductor: Command: ipmitool -I lanplus -H 10.10.10.10 -L ADMINISTRATOR -U ADMIN -R 6 -N 15 -f /tmp/tmpCeEmxP power status
Sep 25 17:08:01 hostname01 ironic-conductor: Exit code: 1
Sep 25 17:08:01 hostname01 ironic-conductor: Stdout: u''
Sep 25 17:08:01 hostname01 ironic-conductor: Stderr: u'Error in open session response message : insufficient resources for sessionnnError: Unable to establish IPMI v2 / RMCP+ sessionnUnable to get Chassis Power Statusn'
  • The following changes were applied:
    sudo openstack-config —set /etc/nova/nova.conf DEFAULT rpc_response_timeout 600
    sudo openstack-config —set /etc/ironic/ironic.conf DEFAULT rpc_response_timeout 600
    sudo openstack-config —set /etc/ironic/ironic.conf conductor sync_power_state_interval 90
    sudo openstack-config —set /etc/ironic/ironic.conf conductor power_state_sync_max_retries 3
    sudo openstack-config —set /etc/ironic/ironic.conf ilo power_wait 15
    sudo openstack-config —set /etc/ironic/ironic.conf ipmi retry_timeout 90
    sudo openstack-config —set /etc/ironic/ironic.conf ipmi min_command_interval 15
    sudo openstack-service restart nova
    sudo openstack-service restart ironic.

  • When deploying an overcloud, hosts are often in «wait / callback» state

Environment

  • Red Hat OpenStack 7.X (RHOS)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

Hi again,

sorry for late reply.

Your answer helped indeed, thank you.

I was able to Shut down One Server with

ipmitool -H ipipip -U Administrator -P pwpwpw -L Administrator -I lanplus power off

but often I get sth like:

 ERROR: error executing ipmitool: ipmitool: lanplus.c:2153: ipmi_lanplus_send_payload: Assertion `session->v2_data.session_state == LANPLUS_STATE_PRESESSION’ failed. Aborted

or

 ERROR: error executing ipmitool: ipmitool: lanplus.c:2167: ipmi_lanplus_send_payload: Assertion `session->v2_data.session_state == LANPLUS_STATE_OPEN_SESSION_RECEIEVED’ failed. Aborted

or

Unable to set Chassis Power Control to Down/Offionol: Error in open session response message : insufficient resources for session

Well it doesn’t work when somebody is logged in, but even if no one is logged in it only works sometimes, seems the connection is very unstable ?!

Surfing the web gui of ilo is very slow also, maybe this is a problem?

Why you do c 4 ? You have the same problems?

My stonith isn’t working anyway…

with:

stonith -t external/ipmi hostname=visn2 ipaddr=ipipip userid=Administrator passwd=pwpwpwpw interface=lanplus -c 4 -T off visn2

external/ipmi[9366]: ERROR: error executing ipmitool: ipmitool: lanplus.c:2153: ipmi_lanplus_send_payload: Assertion `session->v2_data.session_state == LANPLUS_STATE_PRESESSION’ failed. Aborted
CRIT: external_reset_req: ‘ipmi off’ for host visn2 failed with rc 1
 Unable to set Chassis Power Control to Down/Offtool: Error: Unable to establish IPMI v2 / RMCP+ session
CRIT: external_reset_req: ‘ipmi off’ for host visn2 failed with rc 1
external/ipmi[9691]: ERROR: error executing ipmitool: ipmitool: lanplus.c:2153: ipmi_lanplus_send_payload: Assertion `session->v2_data.session_state == LANPLUS_STATE_PRESESSION’ failed. Aborted
CRIT: external_reset_req: ‘ipmi off’ for host visn2 failed with rc 1
 Unable to set Chassis Power Control to Down/Offtool: Error: Unable to establish IPMI v2 / RMCP+ session
CRIT: external_reset_req: ‘ipmi off’ for host visn2 failed with rc 1

Even with -c 50 ;-)

Is there anyway to fix this?

**EDIT**

I created another User @Ilo webinterface and now stonith is working! but still need at least 4 tries with (-c) … thats makes me a little nervous :)

but THANK YOU!!!

У меня проблема с нашей коробкой Power8. Я больше не могу использовать интерфейс IPMI — каждая команда, которую я выдаю, даже простые запросы только для чтения, такие как «состояние шасси», приводит к следующему выводу:

$ /opt/local/bin/ipmitool -I lanplus -H 192.168.70.147 -P admin chassis status Error in open session response message : insufficient resources for session  Error: Unable to establish IPMI v2 / RMCP+ session Error sending Chassis Status command 

Проблема в том, что я больше не могу получить доступ к интерфейсу SOL. Есть идеи, как вытащить коробку из этого состояния? Я пробовал выключение и включение питания — следующая идея — сбросить до заводских настроек.


1 ответ на вопрос

Bill Buros

2014-08-29 в 18:07

Access to IPMI is mutually exclusive between the PowerVM and PowerKVM enabled modes.

As seen, IPMI will work just fine in the OPAL hypervisor mode, PowerVM mode uses a different firmware load.

There’s an emerging page for IPMI considerations over on the DeveloperWorks realm.

There’s also some information on IPMI in the PowerKVM Redbook.

Похожие вопросы

  • 3
    Какую ОС вы можете загрузить на iMac G4?


  • 2
    Загрузочный компакт-диск PowerPC, который может читать разделы NTFS


  • 1
    Debootstrapping Karmic PowerPC Arch приводит к ошибке?



  • 2
    Как заставить PowerMac G4 работать под Linux?


  • 1
    Какие музеи компьютерной истории нуждаются в аппаратных пожертвованиях?


  • 1
    Офисный пакет для eMac


  • 3
    Является ли архитектура PowerPC (G4) лучше, чем архитектура Intel x86 для определенных типов приложе…


  • 1
    Восстановление / Создание раздела NewWorld на Mac G4 (PPC) после неудачной установки Debian


  • 4
    Как вы используете Mac без мыши?


  • 2
    Как мне не дать моему Mac постоянно спать?


Понравилась статья? Поделить с друзьями:
  • Error in ole initialization что делать
  • Error in ole initialization перевод
  • Error in ole initialization компас 20 что это
  • Error in ole initialization компас 18 как исправить windows 10
  • Error in offset data acquisition