Operational Defect Database
BugZero updated this defect 31 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
VMware | 85326
Enabling Lockdown Mode on an ESXi host results in - Cannot login user root@127.0.0.1: no permission events
Last update date:
4/19/2024
Affected products:
vSphere Replication
vSphere ESXi
vSphere
Affected releases:
7.08.0
Fixed releases:
No fixed releases provided.
Description:
Symptoms
When the 'root' user is not added to the exception users list, the host starts triggering the event - Date Time: 03/25/2024, 2:05:19 PMType: ErrorUser: rootTarget: ESXi.host.localDescription: Cannot login user root@127.0.0.1: no permissionEvent Type Description: A user could not log in due to insufficient access permissionPossible Causes: The user account has insufficient access permission Action: Log in with a user account that has the necessary access permissions or grant additional access permissions to the current userBefore adding 'root' user to Exception List:2024-03-18T18:26:03.317Z info hostd[2100450] [Originator@6876 sub=Default opID=b1306eaf] Accepted password for user root from 127.0.0.12024-03-18T18:26:03.317Z info hostd[2100450] [Originator@6876 sub=Vimsvc opID=b1306eaf] [Auth]: User root2024-03-18T18:26:03.317Z warning hostd[2100450] [Originator@6876 sub=Vimsvc opID=b1306eaf] Refresh function is not configured.User data can't be added to scheduler.User name: root2024-03-18T18:26:03.317Z info hostd[2100450] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=b1306eaf] Event 16503 : Cannot login user root@127.0.0.1: no permission2024-03-18T18:26:03.937Z info hostd[2100451] [Originator@6876 sub=Vimsvc.TaskManager opID=sps-Main-584081-203-786363-b-2c-6eb2 user=vpxuser:VSPHERE.LOCAL\vpxd-extension-ff65a043-ba8f-4dcf-bb1e-d63f09da9491] Task Created : haTask--vim.vslm.host.CatalogSyncManager.queryCatalogChange-27045966142024-03-18T18:26:03.942Z info hostd[2100454] [Originator@6876 sub=Libs opID=sps-Main-584081-203-786363-b-2c-6eb2 user=vpxuser:VSPHERE.LOCAL\vpxd-extension-ff65a043-ba8f-4dcf-bb1e-d63f09da9491] notFound(403)2024-03-18T18:26:03.942Z info hostd[2100454] [Originator@6876 sub=Libs opID=sps-Main-584081-203-786363-b-2c-6eb2 user=vpxuser:VSPHERE.LOCAL\vpxd-extension-ff65a043-ba8f-4dcf-bb1e-d63f09da9491] New error before the previous is handled2024-03-18T18:26:03.942Z info hostd[2100454] [Originator@6876 sub=Vimsvc.TaskManager opID=sps-Main-584081-203-786363-b-2c-6eb2 user=vpxuser:VSPHERE.LOCAL\vpxd-extension-ff65a043-ba8f-4dcf-bb1e-d63f09da9491] Task Completed : haTask--vim.vslm.host.CatalogSyncManager.queryCatalogChange-2704596614 Status success2024-03-18T18:26:06.320Z info hostd[2099895] [Originator@6876 sub=Solo.Vmomi opID=b1306eaf] Activation finished; <<52639ae0-ae87-a6f4-1075-6fb1a2eaf73b, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : 21957'>>, ha-sessionmgr, vim.SessionManager.login>2024-03-18T18:26:06.320Z verbose hostd[2099895] [Originator@6876 sub=Solo.Vmomi opID=b1306eaf] Arg userName:--> "local-root"2024-03-18T18:26:06.320Z verbose hostd[2099895] [Originator@6876 sub=Solo.Vmomi opID=b1306eaf] Arg password:--> (not shown)-->2024-03-18T18:26:06.320Z verbose hostd[2099895] [Originator@6876 sub=Solo.Vmomi opID=b1306eaf] Arg locale:--> (null)2024-03-18T18:26:06.320Z info hostd[2099895] [Originator@6876 sub=Solo.Vmomi opID=b1306eaf] Throw vim.fault.NoPermission2024-03-18T18:26:06.320Z info hostd[2099895] [Originator@6876 sub=Solo.Vmomi opID=b1306eaf] Result:--> (vim.fault.NoPermission) {--> object = 'vim.Folder:ha-folder-root',--> privilegeId = "System.View",--> msg = "",--> }After adding 'root' user to Exception List:2024-03-18T18:27:03.318Z info hostd[2102861] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=b1306efd] Event 16507 : User root@127.0.0.1 logged in as hbr-agent/7.0.3-20217181This issue has been reproduced with vCenter 8.0.2 and VRMS 8.8.0.2. When ESXi host is in lockdown mode, ESXi shows the event "Cannot login user root@127.0.0.1: no permission" every 1 minute.less hbr-agent.log | grep -i 'Create login request for user local-root' (In the ESXi host)2024-04-15T09:10:18.554Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:10:18.554052 hbr-agent-bin [1060145] [0x000000d5254da700] trace: [HostdVmomiHttp] Create login request for user local-root2024-04-15T09:11:18.553Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:11:18.553592 hbr-agent-bin [1060145] [0x000000d525459700] trace: [HostdVmomiHttp] Create login request for user local-root2024-04-15T09:12:18.553Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:12:18.553125 hbr-agent-bin [1060145] [0x000000d525459700] trace: [HostdVmomiHttp] Create login request for user local-root2024-04-15T09:13:18.553Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:13:18.553866 hbr-agent-bin [1060145] [0x000000d525459700] trace: [HostdVmomiHttp] Create login request for user local-root2024-04-15T09:14:18.553Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:14:18.553382 hbr-agent-bin [1060145] [0x000000d52555b700] trace: [HostdVmomiHttp] Create login request for user local-root2024-04-15T09:15:18.554Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:15:18.554915 hbr-agent-bin [1060145] [0x000000d525459700] trace: [HostdVmomiHttp] Create login request for user local-roothbr agent connects to the host repeatedly:2024-04-15T09:19:18.549Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.549832 hbr-agent-bin [1060145] [0x000000d525459700] trace: [AsyncVmomiClient] Connected2024-04-15T09:19:18.549Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.549900 hbr-agent-bin [1060145] [0x000000d525459700] trace: [HostdVmomiHttp] Create acquire local ticket request2024-04-15T09:19:18.549Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.549915 hbr-agent-bin [1060145] [0x000000d525459700] trace: [AsyncVmomiClient] Write request2024-04-15T09:19:18.549Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.549950 hbr-agent-bin [1060145] [0x000000d525459700] trace: [AsyncVmomiClient] Read response2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552711 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [Http] HTTP 1/1 200 response2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552774 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [Http] Got 6 HTTP headers2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552787 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [Http] Content length: 5582024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552796 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [AsyncVmomiClient] Transfer content: 0 bytes (558 already in buffer)2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552900 hbr-agent-bin [1060145] [0x000000d5253d8700] debug: [AsyncVmomiClient] Acquired local ticket, logging in...2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552917 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [HostdVmomiHttp] Create login request for user local-root2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552939 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [AsyncVmomiClient] Write request2024-04-15T09:19:18.552Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:18.552969 hbr-agent-bin [1060145] [0x000000d5253d8700] trace: [AsyncVmomiClient] Read response2024-04-15T09:19:21.557Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:21.557827 hbr-agent-bin [1060145] [0x000000d52555b700] error: [Http] Unexpected HTTP status code: 5002024-04-15T09:19:21.557Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:21.557873 hbr-agent-bin [1060145] [0x000000d52555b700] trace: [AsyncVmomiClient] Close connection2024-04-15T09:19:21.557Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:21.557883 hbr-agent-bin [1060145] [0x000000d52555b700] trace: [HostdVmomiHttp] Clear session cookies2024-04-15T09:19:21.557Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:19:21.557931 hbr-agent-bin [1060145] [0x000000d52555b700] trace: [AsyncVmomiClient] Connection closed2024-04-15T09:20:17.583Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:20:17.583308 hbr-agent-bin [1060145] [0x000000d525459700] trace: [Server] Removed 0 dead connections2024-04-15T09:20:18.550Z In(166) hbr-agent-bin[1060145]: 2024-04-15T09:20:18.550689 hbr-agent-bin [1060145] [0x000000d52555b700] trace: [AsyncVmomiClient] Connected
Cause
When an ESXi host is in Lockdown Mode , you can use the Exception User list to Specify Lockdown Mode Exception Users & add the accounts of third-party solutions and external applications that need to access the host directly when the host is in lockdown mode. vSphere Replication software requires hbr-agent to query information from ESXi, such as VM or network configuration every 60 seconds. So for each host, there will be 1440 login events and 1440 logout events every day. This process only uses the 'root' user to perform this activity. Now, due to the nature of ESXi lockdown mode and vSphere Replication products requiring to work in their own way to achieve the results of their own, it creates a catch-22 situation. Therefore, the ESXi host ends up generating the errors when 'root' user isn't added to the Exception Users list.
Impact / Risks
The host will perpetually fill up with Cannot login user root@127.0.0.1: no permission events, thereby obstructing other important events from populating in the events tab.
Resolution
Currently, there is no resolution for this problem.
Workaround
NOTE: You won't be able to configure vSphere replication encryption on VMs, if you disable hbr-agent or uninstall it from the host. Disable hbr-agent service from the host and set it to start & stop manually. This must be done on all the ESXi hosts that you choose to enable lockdown mode on.
Related Information
User root@127.0.0.1 logged in as hbr-agent messages are filling up host event logs (87700) - https://kb.vmware.com/s/article/87700?lang=en_US
