Operational Defect Database
BugZero updated this defect 48 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
VMware | 95862
In NAPP environments, the 'rawflowcorrelator' and 'overflowcorrelator' pods can sometimes malfunction, potentially leading to indefinite growth in disk usage and causing failures in log bundle collection processes.
Last update date:
4/1/2024
Affected products:
NSX-T
Affected releases:
4.x
Fixed releases:
No fixed releases provided.
Description:
Symptoms
1. Alarms for platform services disk usage growing.2. Failure to generate a support bundle.Relevant Log LocationsThe issue can be confirmed by collecting the "diskusage.txt" file after executing the following commands: napp-k exec -it `napp-k get pods | grep cluster | cut -d ' ' -f 1` -c cluster-api -- find /var/log/napp/supportbundle -type f -exec ls -lh {} \; > ./diskusage.txt napp-k exec -it `napp-k get pods | grep cluster | cut -d ' ' -f 1` -c cluster-api -- df -h >> ./diskusage.txt
Purpose
From the 'df -h' command output, it's evident that the '/var/log/napp/supportbundle' directory on the 'cluster-api' pod is almost full or full. To address this issue, a patch named "cluster_api_patch.yaml" is suggested. This patch is to be applied to the 'cluster-api'. The steps to apply this patch are provided in 'Workaround Section'.
Cause
In certain scenarios, the 'rawflowcorrelator' and 'overflowcorrelator' pods can sometimes malfunction, leading to excessive and uncontrolled log generation. This abnormal behavior results in rapid disk space consumption, potentially causing indefinite growth in disk usage. This issue is particularly severe if these pods crash or restart frequently, exacerbating log accumulation and posing a risk to system performance and storage capacity.
Impact / Risks
Alarms will be seen about platform services disk usage, and it will not be possible to generate the NAPP support bundle.It is a known issue in 4.1.x and lower versions.
Resolution
This is now fixed in NAPP 4.1.2 and higher versionsNAPP 4.1.2 do not need to patch the setup.The patch "cluster_api_patch" attached to this KB has been tested on NAPP 4.1.1 release version and after applying the patch , flowcorelator files older than 2 days were deleted.
Workaround
Download the attached file "cluster_api.yaml" and move it to the NSX manager node.Execute the following command on the NSX shell from the same directory where the file is present: napp-k patch deployment cluster-api --type merge --patch-file=cluster_api_patch.yaml
