2016-09-26 11:12:38,592 INFO [main] [ConsoleStatus] [1/32] [0ms] [0ms] Loading bootstrap
2016-09-26 11:12:38,706 INFO [main] [ConsoleStatus] [2/32] [114ms] [114ms] Loading config-defaults
11:17:40.292 [main] WARN o.eclipse.jetty.webapp.WebAppContext - Failed startup of context o.e.j.w.WebAppContext@3c4d3949{/,file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/,STARTING}{file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/}
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1512) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:296) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:293) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:184) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:157) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:152) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:97) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.start(Server.java:387) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.doStart(Server.java:354) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [58488b81592773dcae114770406e3e1b/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_111]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_111]
at io.cattle.platform.launcher.Main.run(Main.java:188) [58488b81592773dcae114770406e3e1b/:na]
at io.cattle.platform.launcher.Main.main(Main.java:251) [58488b81592773dcae114770406e3e1b/:na]
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:189) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:181) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299) ~[liquibase-core-3.1.0.jar:na]
at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:62) ~[cattle-framework-db-loader-0.5.0-SNAPSHOT.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1571) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
... 44 common frames omitted
11:17:40.294 [main] WARN o.e.j.u.component.AbstractLifeCycle - FAILED o.e.j.w.WebAppContext@3c4d3949{/,file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/,UNAVAILABLE}{file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/}: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1512) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:296) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:293) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:184) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:157) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:152) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:97) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.start(Server.java:387) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.doStart(Server.java:354) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [58488b81592773dcae114770406e3e1b/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_111]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_111]
at io.cattle.platform.launcher.Main.run(Main.java:188) [58488b81592773dcae114770406e3e1b/:na]
at io.cattle.platform.launcher.Main.main(Main.java:251) [58488b81592773dcae114770406e3e1b/:na]
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:189) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:181) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299) ~[liquibase-core-3.1.0.jar:na]
at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:62) ~[cattle-framework-db-loader-0.5.0-SNAPSHOT.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1571) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
... 44 common frames omitted
11:17:40.320 [main] WARN o.e.j.u.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@66b6983c: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1512) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:296) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:293) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:184) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:157) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:152) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:97) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.start(Server.java:387) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.doStart(Server.java:354) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) [58488b81592773dcae114770406e3e1b/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_111]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_111]
at io.cattle.platform.launcher.Main.run(Main.java:188) [58488b81592773dcae114770406e3e1b/:na]
at io.cattle.platform.launcher.Main.main(Main.java:251) [58488b81592773dcae114770406e3e1b/:na]
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:189) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:181) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299) ~[liquibase-core-3.1.0.jar:na]
at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:62) ~[cattle-framework-db-loader-0.5.0-SNAPSHOT.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1571) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
... 44 common frames omitted
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1512)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:296)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:184)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:157)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:152)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:97)
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56)
at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11)
at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15)
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.server.Server.start(Server.java:387)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:354)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at io.cattle.platform.launcher.jetty.Main.main(Main.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at io.cattle.platform.launcher.Main.run(Main.java:188)
at io.cattle.platform.launcher.Main.main(Main.java:251)
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168)
at liquibase.Liquibase.update(Liquibase.java:189)
at liquibase.Liquibase.update(Liquibase.java:181)
at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342)
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299)
at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:62)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1571)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509)
... 44 more
STARTUP FAILED [306200] ms
11:17:40.330 [main] ERROR ConsoleStatus - Startup Failed [306195ms]
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase-core' defined in URL [jar:file:/usr/share/cattle/58488b81592773dcae114770406e3e1b/WEB-INF/lib/cattle-app-config-0.5.0-SNAPSHOT.jar!/META-INF/cattle/system/spring-system-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1512) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:296) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:293) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479) ~[spring-context-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:184) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:157) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:348) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:358) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:333) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:152) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:97) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:75) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:56) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.SpringContext.<init>(SpringContext.java:11) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.context.WebContextLoaderListener.constructCloudStackSpringContext(WebContextLoaderListener.java:15) ~[cattle-iaas-system-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51) ~[cattle-framework-spring-module-0.5.0-SNAPSHOT.jar:na]
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505) ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.start(Server.java:387) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.doStart(Server.java:354) ~[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at io.cattle.platform.launcher.jetty.Main.main(Main.java:145) ~[58488b81592773dcae114770406e3e1b/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_111]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_111]
at io.cattle.platform.launcher.Main.run(Main.java:188) [58488b81592773dcae114770406e3e1b/:na]
at io.cattle.platform.launcher.Main.main(Main.java:251) [58488b81592773dcae114770406e3e1b/:na]
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by b4f9bbee3bde (fe80:0:0:0:42:acff:fe11:2%14) since 9/26/16 10:36 AM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:189) ~[liquibase-core-3.1.0.jar:na]
at liquibase.Liquibase.update(Liquibase.java:181) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342) ~[liquibase-core-3.1.0.jar:na]
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299) ~[liquibase-core-3.1.0.jar:na]
at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:62) ~[cattle-framework-db-loader-0.5.0-SNAPSHOT.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1571) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509) ~[spring-beans-3.2.5.RELEASE.jar:3.2.5.RELEASE]
... 44 common frames omitted
Problem
Crowd Server does not start and the following error can be seen in the atlassian-crowd.log
:
2020-08-27 22:15:50,436 localhost-startStop-1 INFO [liquibase] Waiting for changelog lock....
...
2020-08-27 22:20:50,501 localhost-startStop-1 ERROR [crowd.console.listener.StartupListener] Failed to initialise Crowd container
java.lang.RuntimeException: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by my.test-crowd-server (192.168.48.3) since 8/27/20 4:03 AM
at com.atlassian.crowd.util.persistence.hibernate.LiquibaseSchemaHelper.runLiquibaseUpdate(LiquibaseSchemaHelper.java:142)
at com.atlassian.crowd.util.persistence.hibernate.LiquibaseSchemaHelper.lambda$updateSchemaIfNeeded$1(LiquibaseSchemaHelper.java:80)
at com.atlassian.crowd.util.persistence.hibernate.LiquibaseSchemaHelper.withServiceRegistry(LiquibaseSchemaHelper.java:166)
at com.atlassian.crowd.util.persistence.hibernate.LiquibaseSchemaHelper.withServiceRegistry(LiquibaseSchemaHelper.java:152)
at com.atlassian.crowd.util.persistence.hibernate.LiquibaseSchemaHelper.updateSchemaIfNeeded(LiquibaseSchemaHelper.java:79)
at com.atlassian.crowd.console.listener.StartupListener.migrateAndUpgradeCrowd(StartupListener.java:193)
at com.atlassian.crowd.console.listener.StartupListener.contextInitialized(StartupListener.java:60)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1420)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1410)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by my.test-crowd-server (192.168.48.3) since 8/27/20 4:03 AM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:190)
at liquibase.Liquibase.update(Liquibase.java:196)
at liquibase.Liquibase.update(Liquibase.java:192)
at com.atlassian.crowd.util.persistence.hibernate.LiquibaseSchemaHelper.runLiquibaseUpdate(LiquibaseSchemaHelper.java:137)
... 15 more
The following is also presented on the Crowd User Interface:
Diagnosis
Check for the above mentioned log errors in atlassian-crowd.log
as well as checking the above screen presented on Crowd.
Alternatively, check the database to see if the is locked whilst Crowd has been shut down:
Lock table is held by something (Bad)
-- POSTGRES
> select * from cwd_databasechangeloglock;
-- MySQL, MSSQL and Oracle
> select * from CWD_DATABASECHANGELOGLOCK;
id locked lockgranted lockedby
-- ------ ----------------------- ---------------------------------------
1 true 2020-08-27 04:03:41.451 my.test-crowd-server.com (192.168.48.3)
Lock table is not held by anything (Good)
-- POSTGRES
> select * from cwd_databasechangeloglock;
-- MySQL, MSSQL and Oracle
> select * from CWD_DATABASECHANGELOGLOCK;
id locked lockgranted lockedby
-- ------ ----------- --------
1 false (null) (null)
Cause
The cwd_databasechangeloglock
table has not been updated with the release lock information.
The likely cause of this is that the Crowd Server instance was forced to quit while it was trying to startup, with the consequence that the lock was not released. You should always wait for Crowd Server to start up sufficiently for it to provide error messages – never assume that it has hung and kill the process.
Resolution
Ensure that Crowd Server application is shutdown prior to running the below SQL queries to manually force unlock the cwd_databasechangeloglock
table.
- Shutdown Crowd Server
-
Run the following SQL query in respect to your database engine:
- Start Crowd Server and the «Could not acquire change lock» error should no longer appear
If you’re using Liquibase for database versioning with Kubernetes to deploy your app, you might have faced an issue when a migration gets stuck because Liquibase can’t acquire lock. It might look somewhat like this:
Code language: Properties (properties)
liquibase.exception.LockException: Could not acquire change log lock. Currently locked by LockOwner ... at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:236) at liquibase.Liquibase.update(Liquibase.java:184) at liquibase.Liquibase.update(Liquibase.java:179)
Probably if you face this issue, you run DB migration when you start the app. Also, chances are high that you’re using Spring Framework. The reasons to face this issue might vary, but one of them might be auto scaling enabled in Kubernetes.
Contents
- 1 The issue
- 2 The solution
- 2.1 Using command line to alter the context
- 2.2 Using Spring profiles to disable migrations on app startup
- 2.3 Using init containers to run migration
- 3 Summary
The issue
When running DB migrations Liquibase uses a table for locking, to make sure no other instances will run the same migration. The table is called DATABASECHANGELOGLOCK
. Liquibase sets value in column LOCKED to 1 before running a migration and to 0 after it finishes. But if the instance running the migration gets killed before it can finish, or any other issue happens, you can end up with the exception above.
In that case you have two ways of solving this issue:
- Run Liquibase’s
releaseLocks
command from command line:liquibase --changeLogFile=mainchangelog.xml releaseLocks
You can read more about it here. - Run a SQL statement:
UPDATE DATABASECHANGELOGLOCK SET LOCKED=0, LOCKGRANTED=null, LOCKEDBY=null where ID=1;
The solution
The official Liquibase blog has a working solution for the issue, but it requires you to:
- have another copy of your migration scripts in the container;
- provide authentication details to connect to the DB in a way different from the one you use in your app (be it a Hashicorp’s vault integration or something else);
- probably even create a different new Docker image.
But there’s a way to take advantage of your current authentication/connection infrastructure packed into the jar file while still using init containers.
In the following examples I’ll be using Kotlin and Spring Framework, but you can apply a similar solution with any other language/framework.
Using command line to alter the context
Let’s change our main app class to take an argument and change the context configuration based on that:
Code language: Kotlin (kotlin)
import org.springframework.boot.ApplicationContextFactory.ofContextClass import org.springframework.boot.autoconfigure.SpringBootApplication import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration import org.springframework.boot.autoconfigure.liquibase.LiquibaseAutoConfiguration import org.springframework.boot.builder.SpringApplicationBuilder import org.springframework.boot.runApplication import org.springframework.context.annotation.AnnotationConfigApplicationContext import org.springframework.context.annotation.Import @SpringBootApplication class Application @Import(DataSourceAutoConfiguration::class, LiquibaseAutoConfiguration::class) class LiquibaseInit fun main(args: Array<String>) { if (args.contains("dbinit")) { SpringApplicationBuilder(LiquibaseInit::class.java) .contextFactory(ofContextClass(AnnotationConfigApplicationContext::class.java)) .profiles("dbinit") .run(*args) return } runApplication<Application>(*args) }
Let’s go through the code in this example.
We have 2 classes declared here Application
(the main app class having @SpringBootApplication
annotation) and LiquiBaseInit
(having @DataSourceAutoConfiguration
and @LiquibaseAutoConfiguration
), the first one will spin up the whole context, while the latter will only spin up the beans necessary for Liquibase to run migration.
❗NOTE: with Spring Boot 2.4.0 and earlier use SpringApplicationBuilder
#contextClass instead of contextFactory
method.
Inside the main function we check if the arguments array has a string dbinit
and if it is there we start an application context out of LiquiBaseInit
class. We also activate the same-named Spring profile, more on that below.
Using Spring profiles to disable migrations on app startup
If we’re to run DB migrations using Kubernetes’ init containers, we should make sure to not run them on app startup. Otherwise we might still face the same issue. This is where we will use the profile mentioned above: we will only run Liquibase migrations when dbinit
profile is active.
Here’s an example of how to do that in application.yml
:
Code language: YAML (yaml)
spring: liquibase: change-log: classpath:/db/changelog/db.changelog-master.xml user: db_user password: password default-schema: db_schema enabled: false --- spring: profiles: dbinit liquibase: enabled: true
Here we have Liquibase disabled by default, while dbinit
profile enables it. Keep in mind that I’ve omitted some of the configuration properties unrelated to Liquibase, like datasource configuration and other stuff.
Using init containers to run migration
Now that we have everything we need, there’s only one step left – to configure Kubernetes to run the container with a custom endpoint where we’ll specify the new run argument.
Here’s a part of myapp.yaml
where we configure it:
Code language: YAML (yaml)
apiVersion: v1 kind: Pod metadata: name: myapp-pod labels: app: myapp spec: containers: - name: myapp-container image: myapp-image envFrom: - secretRef: name: db-secrets initContainers: - name: myapp-liquibase image: myapp-image command: - "java" - "-jar" - "/app/service.jar" - "dbinit" envFrom: - secretRef: name: db-secrets
You can read more about init containers in the official Kubernetes documentation. But what’s happening here is that we reuse the same DB credentials for both myapp-container
and maypp-liquibase
. We also reuse the same image for both of them.
Summary
With this approach you can have a single Docker image to run your DB migrations and to run the app itself, while making sure there won’t be a deadlock.
Hope this article helps you!
Happy coding!
I’m a software engineer with 10+ years of experience in IT. I’m passionate about backend engineering, high load, testing, open source and hacking. 15 posts Andrei Nevedomskii