Problem
PAM authentication failed when SD failed
Symptom
You can do EGO related authentication successfully (For example: egosh user logon) but SOAM related authentication failed (For example: soamview) with following error message:
Cannot retrieve application information : Security error: Authentication failed.
Incorrect user name or password, or the security plugin setting
(specified in the EGO_SEC_PLUGIN parameter in the ego.conf file on both
the client and server) are incompatible.
This issue only happens when SD is running on a master candidate host.
Cause
PAM authentication failed
Diagnosing The Problem
Because this is a PAM authentication failure, you can start from checking PAM authentication plugin’s log. The log directory is defined by EGO_SEC_CONF in ego.conf. For example:
EGO_SEC_CONF=/opt/egoshare7.1.2/kernel/conf,0,WARN,/opt/ibm/platformsymphony/kernel/log
In the plugin log, you can see error message like following:
xxx xxx xx xx:xx:xx 2017 ERROR [7443] readParamsFromFile(): Error reading plugin configuration file /opt/ibm/platformsymphony/kernel/conf/seckey.conf
Above error message shows the PAM authentication plugin failed to read the key file which is defined in pamauth.conf
Then you can check pamauth.conf to see how the key file is configured. For example following configuration shows the key file is on local file system.
KEYFILE=/opt/ibm/platformsymphony/kernel/conf/seckey.conf
Based on the configuration, the problem could be when a daemon runs on a management host which doesn’t have the key file in the defined directory which leads to authentication failure.
Resolving The Problem
You can either copy the key file to the defined directory on each of the management hosts, or copy the key file to your shared file system and then modify pamauth.conf to use such directory.
[{«Product»:{«code»:»SSZUMP»,»label»:»IBM Spectrum Symphony»},»Business Unit»:{«code»:»BU059″,»label»:»IBM Software w/o TPS»},»Component»:»Not Applicable»,»Platform»:[{«code»:»PF025″,»label»:»Platform Independent»}],»Version»:»7.1.2;7.2.0″,»Edition»:»»,»Line of Business»:{«code»:»LOB10″,»label»:»Data and AI»}}]
HI,
When I connect to the jupyterhub interface via a localhost: 8000 URL.
I receive an error message on my vm cento7.
Here’s the message:
1-If user no admin I receive this message:
[I 2017-03-30 09:53:35.951 JupyterHub app:1453] Hub API listening on http://localhost:54321/hub/
[W 2017-03-30 09:53:35.960 JupyterHub app:1174] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
And In jupyterhub interface I receive this messge
[I 2017-03-30 09:53:35.960 JupyterHub app:1176] Starting proxy @ http:// localhost:8000/
09:53:36.458 - info: [ConfigProxy] Proxying http://localhost:8000 to http:// localhost:54321
09:53:36.471 - info: [ConfigProxy] Proxy API at http:// localhost:5432/api/routes
[I 2017-03-30 09:53:36.579 JupyterHub app:1485] JupyterHub is now running at http:// localhost:8000/
[I 2017-03-30 09:54:54.509 JupyterHub spawner:783] Spawning jupyterhub-singleuser '--user="team_k"' '--cookie-name="jupyter-hub-token-team_k"' '--base-url="/user/team_kleber"' '--hub-host=""' '--hub-prefix="/hub/"' '--hub-api-url="http:// localhost:54321/hub/api"' '--ip="127.0.0.1"' --port=37186
[E 2017-03-30 09:54:54.518 JupyterHub spawner:793] Permission denied trying to run '/root/anaconda3/bin/jupyterhub-singleuser'. Does team_k have access to this file?
[E 2017-03-30 09:54:54.527 JupyterHub user:251] Unhandled error starting team_k's server: [Errno 13] Permission denied
[E 2017-03-30 09:54:54.578 JupyterHub web:1548] Uncaught exception POST /hub/login?next= (10.16.79.166)
HTTPServerRequest(protocol='http', host='localhost:8000', method='POST', uri='/hub/login?next=', version='HTTP/1.1', remote_ip='10.16.79.166', headers={'Accept': 'application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*', 'Cache-Control': 'no-cache', 'Content-Length': '36', 'X-Forwarded-Proto': 'http', 'Accept-Language': 'fr-FR', 'X-Forwarded-Port': '8000', 'Content-Type': 'application/x-www-form-urlencoded', 'Accept-Encoding': 'gzip, deflate', 'Connection': 'close', 'X-Forwarded-For': '10.16.79.166', 'Dnt': '1', 'User-Agent': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E; InfoPath.3)', 'X-Forwarded-Host': ' localhost:8000', 'Host': localhost:8000', 'Referer': 'http:// localhost:8000/hub/login'})
Traceback (most recent call last):
File "/root/anaconda3/lib/python3.5/site-packages/tornado/web.py", line 1469, in _execute
result = yield result
File "/root/anaconda3/lib/python3.5/site-packages/jupyterhub/handlers/login.py", line 84, in post
yield self.spawn_single_user(user)
File "/root/anaconda3/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 328, in spawn_single_user
yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
File "/root/anaconda3/lib/python3.5/site-packages/jupyterhub/user.py", line 261, in spawn
raise e
File "/root/anaconda3/lib/python3.5/site-packages/jupyterhub/user.py", line 229, in spawn
ip_port = yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
File "/root/anaconda3/lib/python3.5/types.py", line 243, in wrapped
coro = func(*args, **kwargs)
File "/root/anaconda3/lib/python3.5/site-packages/jupyterhub/spawner.py", line 787, in start
start_new_session=True, # don't forward signals
File "/root/anaconda3/lib/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "/root/anaconda3/lib/python3.5/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
[E 2017-03-30 09:54:54.697 JupyterHub log:99] {
"Accept": "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*",
"Cache-Control": "no-cache",
"Content-Length": "36",
"X-Forwarded-Proto": "http",
"Accept-Language": "fr-FR",
"X-Forwarded-Port": "8000",
"Content-Type": "application/x-www-form-urlencoded",
"Accept-Encoding": "gzip, deflate",
"Connection": "close",
"X-Forwarded-For": "10.16.79.166",
"Dnt": "1",
"User-Agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E; InfoPath.3)",
"X-Forwarded-Host": " localhost:8000",
"Host": " localhost:8000",
"Referer": "http://localhost:8000/hub/login"
}
[E 2017-03-30 09:54:54.697 JupyterHub log:100] 500 POST /hub/login?next= (@10.16.79.166) 302.29ms
And
In jupyterhub interface I receive this messge
500 : Internal Server Error
Failed to start your server. Please contact admin.
2-If user admin I receive this message:
[E 2017-03-30 10:03:45.035 JupyterHub log:100] 500 POST /hub/login?next= (@10.16.79.166) 167.53ms
[I 2017-03-30 10:03:58.426 JupyterHub log:100] 302 GET / (@10.16.79.166) 2.05ms
[I 2017-03-30 10:03:58.428 JupyterHub log:100] 302 GET /hub (@10.16.79.166) 0.63ms
[I 2017-03-30 10:03:58.435 JupyterHub log:100] 302 GET /hub/ (@10.16.79.166) 1.37ms
[I 2017-03-30 10:03:58.441 JupyterHub log:100] 302 GET /login (@10.16.79.166) 1.29ms
[I 2017-03-30 10:03:58.460 JupyterHub log:100] 200 GET /hub/login (@10.16.79.166) 15.11ms
[W 2017-03-30 10:04:35.886 JupyterHub auth:471] PAM Authentication failed (u004753@10.16.79.166): [PAM Error 7] Authentication failure
[I 2017-03-30 10:04:35.899 JupyterHub log:100] 200 POST /hub/login?next= (@10.16.79.166) 1655.52ms
And
In jupyterhub interface I receive this messge
Sign in
Warning: JupyterHub seems to be served over an unsecured HTTP connection. We strongly recommend enabling HTTPS for JupyterHub.
Invalid username or password
Username: Password
you have to generate generate-db-auth-token
with your db_userx
from IAM policy
db-auth-token
will be your PGPASSWORD
export RDSHOST="MYRDSHOSTNAME.us-east-2.rds.amazonaws.com"
export PG_USER="db_userx"
export PGPASSWORD="$(aws rds generate-db-auth-token --hostname $RDSHOST --port 5432 --region us-west-2 --username $PG_USER )"
and than:
psql "host=$RDSHOST port=5432 sslmode=verify-full sslrootcert=./rds-combined-ca-bundle.pem dbname=db_roles_test user=$PG_USER"
this is correct for db_userx
CREATE USER db_userx WITH LOGIN;
GRANT rds_iam TO db_userx;
output of du
List of roles
Role name | Attributes | Member of
----------------------+------------------------------------------------+--------------------------------------------------------------
db_userx | | {rds_iam}
pg_monitor | Cannot login | {pg_read_all_settings,pg_read_all_stats,pg_stat_scan_tables}
pg_read_all_settings | Cannot login | {}
pg_read_all_stats | Cannot login | {}
pg_signal_backend | Cannot login | {}
pg_stat_scan_tables | Cannot login | {}
rds_iam | Cannot login | {}
rds_password | Cannot login | {}
rds_replication | Cannot login | {}
rds_superuser | Cannot login | {pg_monitor,pg_signal_backend,rds_replication,rds_password}
rdsadmin | Superuser, Create role, Create DB, Replication+| {}
| Password valid until infinity |
rdsrepladmin | No inheritance, Cannot login, Replication | {}
root | Create role, Create DB +| {rds_superuser}
so you can create as many users as necessary via
CREATE USER <you_user_name> WITH LOGIN;
be careful Authentication tokens have a lifespan of 15 minutes
so, after all of this, any AWS Resource
with your policy will have access to RDS Db.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:us-east-2:MYAWSROOTACCOUNTID:dbuser:*/db_userx"
]
}
]
}
I have an Ubuntu 16.04 GIT server with AD authentication configured. Authentication stopped working yesterday while nothing on GIT server or AD has changed. Here’s the error message:
pam_winbind(sshd:auth): request wbcLogonUser failed:
WBC_ERR_AUTH_ERROR, PAM error: PAM_AUTH_ERR (7), NTSTATUS:
NT_STATUS_LOGON_FAILURE, Error message was: Logon failure
I’ve noticed that there was an unattended security upgrade related to samba as you can see below.
Start-Date: 2019-04-09 06:59:58 Commandline:
/usr/bin/unattended-upgrade Upgrade: python-samba:amd64
(2:4.3.11+dfsg-0ubuntu0.16.04.18, 2:4.3.11+dfsg-0ubuntu0.16.04.19),
libwbclient0:amd64 (2:4.3.11+dfsg-0ubuntu0.16.04.18,
2:4.3.11+dfsg-0ubuntu0.16.04.19), libsystemd0:amd64 (229-4ubuntu21.16,
229-4ubuntu21.21), samba:amd64 (2:4.3.11+dfsg-0ubuntu0.16.04.18,
2:4.3.11+dfsg-0ubuntu0.16.04.19), samba-dsdb-modules:amd64
(2:4.3.11+dfsg-0ubuntu0.16.04.18, 2:4.3.11+dfsg-0ubuntu0.16.04.19),
udev:amd64 (229-4ubuntu21.16, 229-4ubuntu21.21), libudev1:amd64
(229-4ubuntu21.16, 229-4ubuntu21.21), samba-libs:amd64
(2:4.3.11+dfsg-0ubuntu0.16.04.18, 2:4.3.11+dfsg-0ubuntu0.16.04.19),
libpam-winbind:amd64 (2:4.3.11+dfsg-0ubuntu0.16.04.18,
2:4.3.11+dfsg-0ubuntu0.16.04.19), winbind:amd64
(2:4.3.11+dfsg-0ubuntu0.16.04.18, 2:4.3.11+dfsg-0ubuntu0.16.04.19),
samba-common:amd64 (2:4.3.11+dfsg-0ubuntu0.16.04.18,
2:4.3.11+dfsg-0ubuntu0.16.04.19), systemd-sysv:amd64
(229-4ubuntu21.16, 229-4ubuntu21.21), libnss-winbind:amd64
(2:4.3.11+dfsg-0ubuntu0.16.04.18, 2:4.3.11+dfsg-0ubuntu0.16.04.19),
libpam-systemd:amd64 (229-4ubuntu21.16, 229-4ubuntu21.21),
samba-vfs-modules:amd64 (2:4.3.11+dfsg-0ubuntu0.16.04.18,
2:4.3.11+dfsg-0ubuntu0.16.04.19), systemd:amd64 (229-4ubuntu21.16,
229-4ubuntu21.21), samba-common-bin:amd64
(2:4.3.11+dfsg-0ubuntu0.16.04.18, 2:4.3.11+dfsg-0ubuntu0.16.04.19)
End-Date: 2019-04-09 07:00:51 (END)
I’m not very familiar with samba/pam authentication so I’m open to any suggestions on how to resolve it (other than rolling back).
https://usn.ubuntu.com/3939-1/ -> This seems to be the vulnerability that triggered the security update.
I am building a mail server using Postfix, and set up the authentication to check against a database set-up using Postfixadmin.
I can authenticate via Courier IMAP okay, as it can authenticate against the hashed password properly, but I am suspecting that my SASL + PAM-MySQL SMTP authentication mechanism cannot.
I am getting these errors in /var/log/mail.log
:
pam_unix(smtp:auth): check pass; user unknown
Aug 22 03:23:08 omitted saslauthd[26402]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=
Aug 22 03:23:10 omitted saslauthd[26402]: DEBUG: auth_pam: pam_authenticate failed: Authentication failure
Aug 22 03:23:10 omitted saslauthd[26402]: do_auth : auth failure: [user=user@domain.com] [service=smtp] [realm=domain.com] [mech=pam] [reason=PAM auth error]
Here are the contents of /etc/pam.d/smtp
:
auth required pam_mysql.so user=postfixadmin passwd=omitted host=127.0.0.1 db=postfixadmin table=mailbox usercolumn=username passwdcolumn=password crypt=2
account sufficient pam_mysql.so user=postfixadmin passwd=omitted host=127.0.0.1 db=postfixadmin table=mailbox usercolumn=username passwdcolumn=password crypt=2
Here is the relevant snippet for password encryption from /etc/postfixadmin/config.inc.php
:
// Encrypt
// In what way do you want the passwords to be crypted?
// md5crypt = internal postfix admin md5
// md5 = md5 sum of the password
// system = whatever you have set as your PHP system default
// cleartext = clear text passwords (ouch!)
// mysql_encrypt = useful for PAM integration
// authlib = support for courier-authlib style passwords
// dovecot:CRYPT-METHOD = use dovecotpw -s 'CRYPT-METHOD'. Example: dovecot:CRAM-MD5
$CONF['encrypt'] = 'mysql_encrypt';
And here is the content of my /etc/postfix/sasl/smtp.conf
:
pwcheck_method: saslauthd
mech_list: plain login
log_level: 7
allow_plaintext: true
auxprop_plugin: sql
sql_engine: mysql
sql_hostnames: 127.0.0.1
sql_user: postfixadmin
sql_passwd: omitted
sql_database: postfixadmin
sql_select: select password from mailbox where username='%u@%r'
I tried using MD5 hash but Courier would fail. So thats out of the window…