Collecting crash dump from Frends Agent

Obtaining more information in case of an Agent crashing.

A crash dump is a snapshot of the Agent process memory at the time of a failure, which is essential for Frends support to diagnose unexpected terminations, process hangs, or severe memory-related issues like Out of Memory errors.

For effective memory analysis, it is often critical to capture a Full dump, as partial dumps frequently lack the necessary information.

Windows Agents

For Frends Agents running on Windows, you can choose between configuring the .NET runtime for automatic basic dumps or using the external utility ProcDump for comprehensive full dumps.

Environment Variable Method (Basic Dump)

This method configures the underlying .NET runtime to automatically generate a minidump when the Frends Agent process crashes. To enable this, you must set the environment variable DOTNET_DbgEnableMiniDump to a value of 1 on the Agent machine.

It is crucial that this environment variable is set before the Agent service starts and is accessible to the user account running the service. If the Agent crashes, the resulting dump file will typically be created in the %LOCALAPPDATA%\CrashDumps folder or within the Agent's working directory.

ProcDump Method (Full Dump)

For a more detailed Full dump, which is highly recommended for memory analysis, you should use the Microsoft Sysinternals utility ProcDump. This tool allows you to capture a dump when the process encounters an unhandled exception.

You can download ProcDump from Microsoft from here: Download ProcDump.

After downloading ProcDump, you can execute the following command to monitor the Frends Agent process and capture a full dump:

bashprocdump -ma -e -n 1 -w "Frends.Agent.exe" C:\temp\

In this command, the -ma switch ensures that a 'Full' dump file is written, capturing all memory data. The -e switch triggers the dump when an unhandled exception occurs. The -n 1 limits the capture to a single dump, and -w "Frends.Agent.exe" instructs ProcDump to wait for the Frends Agent executable to start if it is not already running. The dump file will be saved to the specified location, C:\temp\ in this example.

Linux Agents

On Linux, the method for collecting a dump depends on whether the Agent is running directly on the operating system or within a container.

Environment Variable Method (Basic Dump)

This method utilizes the .NET runtime's built-in capability to generate a minidump automatically upon a crash.

For a Containerized Linux Agent, you should enable automatic crash dump generation by setting the environment variable directly within the container configuration, typically by including ENV DOTNET_DbgEnableMiniDump=1 in the Dockerfile. If the containerized Agent crashes, the dump file will be created in /tmp/coredump.<pid> or in the working directory of the Agent.

For a Linux Agent running directly on the operating system, you would set the DOTNET_DbgEnableMiniDump=1 environment variable for the user running the Frends Agent service before the service starts. The dump location will be /tmp/coredump.<pid> or the Agent's working directory.

ProcDump Method (Full Dump)

The Linux port of ProcDump can be used on Linux Agents to capture a dump based on triggers, such as an unhandled exception. You must ensure ProcDump for Linux is installed on the Agent machine.

You can download ProcDump from Microsoft from here: Download ProcDump.

The command to capture a dump when the process encounters an unhandled exception is:

bashsudo procdump -e -n 1 -w frendsagent -o /tmp/frendsagent_dump

In this command, the -e switch triggers the dump on an unhandled exception, and -n 1 ensures only one dump is taken. The -w frendsagent waits for the Frends Agent process to start, and the -o /tmp/frendsagent_dump specifies the output directory for the resulting dump file.