Motivation

While Unreal's AIPerception system comes with a varied list of senses right out of the box, there might come a time when you feel the need to create your own sense.
Let's say you want to add a monster that cannot see the player, but react to their vibrations? While you could fake such a system using an already existing sense (like hearing), it might be better to create a custom "vibration sense", so you have full control over how the monster perceives vibrations.

Anatomy of a Sense

All senses consist of two classes: a config class and an implementation class.

The config class is used by the AIPerception component to adjust how the sense is perceived.
Here is an example for the hearing sense:

The implementation class contains the actual sense logic. In the picture above, the hearing sense implementation is referenced in the "Implementation" property of the config.

This splitting of the config and the implementation in theory allows us to add multiple implementations for a sense or override the implementation for a built-in one. I haven't seen this in action, however.

Creating a Basic Custom Sense

In this tutorial, we will use C++ to define the config and implementation of our sense. This gives us the most flexibility, as certain aspects relevant to the system (like affiliation detection) aren't currently exposed to Blueprints.

Preparation

To use AIPerception classes in our projects, we need to add the AIModule module to our project's Build.cs file like so:

PublicDependencyModuleNames.AddRange(new string[] { "AIModule", "Core", "CoreUObject", "Engine", "InputCore" });

Creating the Config

Let's start by creating a barebones config class inheriting from the AISenseConfig class. For the sake of this tutorial, we will name our custom sense Vibration. The engine calls their config classes AISenseConfig_[ConfigName], so we will do the same.

Below is the code for our basic config class header:

#pragma once

#include "CoreMinimal.h"
#include "Perception/AISenseConfig.h"
#include "AISense_Vibration.h" // We will add this later.
#include "AISenseConfig_Vibration.generated.h"

UCLASS(meta = (DisplayName = "AI Vibration config"))
class YOURGAME_API UAISenseConfig_Vibration : public UAISenseConfig
{
	GENERATED_BODY()
	
public:
	/** Sets common config values. */
	UAISenseConfig_Vibration(const FObjectInitializer& ObjectInitializer);

	/** The implementation class that defines the sense's functionality. */
	UPROPERTY(EditDefaultsOnly, BlueprintReadOnly, Category = "Sense", NoClear, config)
	TSubclassOf<UAISense_Vibration> Implementation;

	/** Radius at which vibrations should be detected. */
	UPROPERTY(EditDefaultsOnly, BlueprintReadOnly, Category = "Sense")
	float DetectionRadius;

	/** Returns Implementation class. */
	virtual TSubclassOf<UAISense> GetSenseImplementation() const override;
};

By default, a sense config only includes three properties: DebugColor, MaxAge and StartsEnabled. You can read about what these three properties do in the official documentation.

However, a bunch of the pre-defined senses choose to add at least one more property: Implementation. As noted above, this allows us to change the implementation for a given sense. Depending on whether or not we choose to add this option, the implementation of GetSenseImplementation() changes:

No implementation property:

TSubclassOf<UAISense> UAISenseConfig_Vibration::GetSenseImplementation() const 
{ 
	return UAISense_Vibration::StaticClass(); 
}

With implementation property:

TSubclassOf<UAISense> UAISenseConfig_Vibration::GetSenseImplementation() const 
{ 
	return *Implementation; 
}

Whether you choose to add an implementation property is up to you. I have included it in this tutorial to showcase the somewhat more complex case of providing one over hardcoding it.

Last but not least, you can use the constructor to change some of the values that come with the AISenseConfig class like so:

UAISenseConfig_Vibration::UAISenseConfig_Vibration(const FObjectInitializer& ObjectInitializer) : Super(ObjectInitializer)
{
	DebugColor = FColor::Cyan;
}

Both the constructor and GetSenseImplementation() can be put in the source file of your sense config, in this case AISenseConfig_Vibration.cpp:

#include "AISenseConfig_Vibration.h"

UAISenseConfig_Vibration::UAISenseConfig_Vibration(const FObjectInitializer& ObjectInitializer) : Super(ObjectInitializer)
{
	DebugColor = FColor::Cyan;
}

TSubclassOf<UAISense> UAISenseConfig_Vibration::GetSenseImplementation() const
{
	return *Implementation;
}

You are free to add as many or as few properties as you please to control how (or if) listeners perceive stimuli from your custom sense.
Examples of properties to add are a max detection range for stimuli and detection by affiliation (can the listener sense stimuli from allied Enemies, Friendlies or Neutrals).

Creating the Implementation

With a config file set up, it is now time to create the implementation file. There is a bunch to set up, so let's go step by step.

What a lot of built-in senses do is to define a struct which specifies an event that listeners of this sense are interested in. In our case, it would be a FAIVibrationEvent like this one:

/*
* Class that contains information about each registered vibration event.
*/
USTRUCT(BlueprintType)
struct YOURGAME_API FAIVibrationEvent
{
	GENERATED_BODY()

	/** Location of the new vibration stimulus. */
	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Sense")
	FVector VibrationLocation;

	/** The actor that caused the vibration. */
	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Sense")
	TObjectPtr<AActor> Instigator;

	FAIVibrationEvent(AActor* InInstigator, const FVector& InVibrationLocation);
};

This struct contains all the relevant information for each event registered with the sense. We need a way to keep track of all registered events of course, so add this line to the class definition of AISense_Vibration:

UPROPERTY()
TArray<FAIVibrationEvent> VibrationEvents;

Define the constructor like so to set the event's member variables:

FAIVibrationEvent::FAIVibrationEvent(AActor* InInstigator, const FVector& InVibrationLocation)
	: VibrationLocation(InVibrationLocation), Instigator(InInstigator)
{
}

Now that we have an array of events that we can iterate through, we need a way to actually register that events have occurred and add them to said array:

// in AISense_Vibration.h, public section

void RegisterEvent(const FAIVibrationEvent& Event);

// -----------------------
// in AISense_Vibration.cpp

void UAISense_Vibration::RegisterEvent(const FAIVibrationEvent& Event)
{
	VibrationEvents.Add(Event);

	RequestImmediateUpdate();
}

RequestImmediateUpdate() is called so that the Update() function of the sense is called immediately. Let's also add that one:

// in AISense_Vibration.h, protected section

virtual float Update() override;

// -----------------------
// in AISense_Vibration.cpp

float UAISense_Vibration::Update()
{
	AIPerception::FListenerMap& ListenersMap = *GetListeners();
    
    // Iterate through all the listeners that have the Vibration sense. Listeners are actors with an AIPerception component.
	for (AIPerception::FListenerMap::TIterator ListenerIt(ListenersMap); ListenerIt; ++ListenerIt)
	{
		FPerceptionListener& Listener = ListenerIt->Value;

		if (Listener.HasSense(GetSenseID()) == false)
		{
			// skip listeners not interested in this sense
			continue;
		}

		for (const FAIVibrationEvent& Event : VibrationEvents)
		{
		    /*
		    * Let the listener know that it has received a stimulus from this event.
		    * The relevant info (like the location and instigator will be provided).
		    */
			Listener.RegisterStimulus(Event.Instigator, FAIStimulus(*this, 1.f, Event.VibrationLocation, Listener.CachedLocation, FAIStimulus::SensingSucceeded, Name_NONE));
		}
	}
    
    // Remove all currently registered vibration events, so they only get handled once.
	VibrationEvents.Reset();

	// Next tick of this function won't happen on it's own, only when RequestImmediateUpdate() is called again.
	return SuspendNextUpdate;
}

There's a lot to unpack here, therefore I have provided lots of comments to explain what the Update() function does to the best of my abilities. Basically, we iterate through all interested listeners and let them each know about all currently registered vibration events. We can then do whatever we want with the received stimuli by using the built-in events, which are also documented.

Here is the final code to see how everything fits together.

AISense_Vibration.h:

#pragma once

#include "CoreMinimal.h"
#include "Perception/AISense.h"
#include "AISense_Vibration.generated.h"

class UAISenseConfig_Vibration;

/*
* Class that contains information about each registered vibration event.
*/
USTRUCT(BlueprintType)
struct YOURGAME_API FAIVibrationEvent
{
	GENERATED_BODY()

	/** Location of the new vibration stimulus. */
	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Sense")
	FVector VibrationLocation;

	/** The actor that caused the vibration. */
	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Sense")
	TObjectPtr<AActor> Instigator;
	
	FAIVibrationEvent(AActor* InInstigator, const FVector& InVibrationLocation);
};

/*
* Implementation class of the vibration sense to be used by the AIPerception system.
*/
UCLASS()
class YOURGAME_API UAISense_Vibration : public UAISense
{
	GENERATED_BODY()

protected:
	/** List of all currently registered vibration events by the AIPerception system. */
	UPROPERTY()
	TArray<FAIVibrationEvent> VibrationEvents;
	
public:
	UAISense_Vibration(const FObjectInitializer& ObjectInitializer);

	/** Called by the PerceptionSystem after calling OnEvent() in ReportVibrationEvent(). Used to run Update() again. */
	void RegisterEvent(const FAIVibrationEvent& Event);

protected:
	/** Registers vibration stimuli with all interested listeners that are in range of said stimuli. */
	virtual float Update() override;
};

AISense_Vibration.cpp:

#include "AISense_Vibration.h"
#include "AISenseConfig_Vibration.h"

#include "Perception/AIPerceptionComponent.h"
#include "Perception/AIPerceptionSystem.h"

UAISense_Vibration::UAISense_Vibration(const FObjectInitializer& ObjectInitializer) :
	Super(ObjectInitializer)
{
}

void UAISense_Vibration::RegisterEvent(const FAIVibrationEvent& Event)
{
	VibrationEvents.Add(Event);

	RequestImmediateUpdate();
}

float UAISense_Vibration::Update()
{
	AIPerception::FListenerMap& ListenersMap = *GetListeners();
    
    // Iterate through all the listeners that have the Vibration sense. Listeners are actors with an AIPerception component.
	for (AIPerception::FListenerMap::TIterator ListenerIt(ListenersMap); ListenerIt; ++ListenerIt)
	{
		FPerceptionListener& Listener = ListenerIt->Value;

		if (Listener.HasSense(GetSenseID()) == false)
		{
			// skip listeners not interested in this sense
			continue;
		}

		for (const FAIVibrationEvent& Event : VibrationEvents)
		{
		    /*
		    * Let the listener know that it has received a stimulus from this event.
		    * The relevant info (like the location and instigator will be provided).
		    */
			Listener.RegisterStimulus(Event.Instigator, FAIStimulus(*this, 1.f, Event.VibrationLocation, Listener.CachedLocation, FAIStimulus::SensingSucceeded, Name_NONE));
		}
	}
    
    // Remove all currently registered vibration events, so they only get handled once.
	VibrationEvents.Reset();

	// Next tick of this function won't happen on it's own, only when RequestImmediateUpdate() is called again.
	return SuspendNextUpdate;
}

//----------------------------------------------------------------------//
// FAIVibrationEvent
//----------------------------------------------------------------------//

FAIVibrationEvent::FAIVibrationEvent(AActor* InInstigator, const FVector& InVibrationLocation)
	: VibrationLocation(InVibrationLocation), Instigator(InInstigator)
{
}

Going further

There are many things that you can improve about our vibration sense. You could add a way to specify the strength of each event, allow Blueprints to report vibration events (like Report Noise Event does for hearing events) or limit the detection range of listeners. This would make our vibration detecting monster fairer to play against, since we could program it to only detect the player when they're close by and only pick up vibrations of a certain strength, like when the player is not crouching.

Adding a way to specify the strength can be achieved rather easily by adding a Strength parameter to FAIVibrationEvent and changing the second parameter of the FAIStimulus constructor to Event.Stength. We can then simply get the stimulus strength like this and do whatever we please with it:

To respect the listener's maximum detection range, we need to do a little more work.

Getting each listener's sense properties

The basic idea here is to map each listener's sense properties to their ID. We can then get the listener's properties in Update() and use them to add a maximum range check. We also need to make sure that we only keep track of listener which are actually interested in this sense.

To store the listener's properties, a lot of engine senses use so-called FDigested[Sense]Property structs (with [Sense] being the name of the sense of course). We will do the same for our custom sense to stay consistent.

// in AISense_Vibration.h, public section

/** Used to consume the properties of the config class. */
struct FDigestedVibrationProperties {
	float DetectionRadius;

	FDigestedVibrationProperties();
	FDigestedVibrationProperties(const UAISenseConfig_Vibration& SenseConfig);
};

/** Maps listeners to their digested property configs. */
TMap<FPerceptionListenerID, FDigestedVibrationProperties> DigestedProperties;

// -----------------------
// in AISense_Vibration.cpp

UAISense_Vibration::FDigestedVibrationProperties::FDigestedVibrationProperties()
{
	DetectionRadius = 10.f;
}

UAISense_Vibration::FDigestedVibrationProperties::FDigestedVibrationProperties(const UAISenseConfig_Vibration& SenseConfig)
{
	DetectionRadius = SenseConfig.DetectionRadius;
}

Next we need to keep the DigestedProperties map up to date, so we need to add events for when a listener registers/unregisters this sense and bind our new functions to the relevant delegates.

// in AISense_Vibration.h, protected section

/** Called when a new listener gained the Vibration sense. */
void OnNewListenerImpl(const FPerceptionListener& NewListener);

/** Called when a new listener potentially lost the Vibration sense. */
void OnListenerUpdateImpl(const FPerceptionListener& UpdatedListener);

/** Called when a new listener lost the Vibration sense. */
void OnListenerRemovedImpl(const FPerceptionListener& UpdatedListener);

// -----------------------
// in AISense_Vibration.cpp

UAISense_Vibration::UAISense_Vibration(const FObjectInitializer& ObjectInitializer) :
	Super(ObjectInitializer)
{
    OnNewListenerDelegate.BindUObject(this, &UAISense_Vibration::OnNewListenerImpl);
	OnListenerUpdateDelegate.BindUObject(this, &UAISense_Vibration::OnListenerUpdateImpl);
	OnListenerRemovedDelegate.BindUObject(this, &UAISense_Vibration::OnListenerRemovedImpl);
}

void UAISense_Vibration::OnNewListenerImpl(const FPerceptionListener& NewListener)
{
	UE_LOG(LogTemp, Display, TEXT("Vibration listener added!"))

	UAIPerceptionComponent* ListenerPtr = NewListener.Listener.Get();
	check(ListenerPtr);

	const UAISenseConfig_Vibration* SenseConfig = Cast<const UAISenseConfig_Vibration>(ListenerPtr->GetSenseConfig(GetSenseID()));
	check(SenseConfig);
	
	const FDigestedVibrationProperties PropertyDigest(*SenseConfig);
	DigestedProperties.Add(NewListener.GetListenerID(), PropertyDigest);
}

void UAISense_Vibration::OnListenerUpdateImpl(const FPerceptionListener& UpdatedListener)
{
	const FPerceptionListenerID ListenerID = UpdatedListener.GetListenerID();

	if (UpdatedListener.HasSense(GetSenseID()))
	{
		const UAISenseConfig_Vibration* SenseConfig = Cast<const UAISenseConfig_Vibration>(UpdatedListener.Listener->GetSenseConfig(GetSenseID()));
		check(SenseConfig);
		FDigestedVibrationProperties& PropertiesDigest = DigestedProperties.FindOrAdd(ListenerID);
		PropertiesDigest = FDigestedVibrationProperties(*SenseConfig);
	}
	else
	{
		DigestedProperties.Remove(ListenerID);
	}
}

void UAISense_Vibration::OnListenerRemovedImpl(const FPerceptionListener& UpdatedListener)
{
	DigestedProperties.FindAndRemoveChecked(UpdatedListener.GetListenerID());
}

Last but not least, to get a listener's sense properties, add this line in Update(), right before iterating through vibration events:
const FDigestedVibrationProperties& PropDigest = DigestedProperties[Listener.GetListenerID()];

You can now access PropDigest.DetectionRadius and use it however you like. If you wanna add more properties to our sense config, don't forget to also add them to FDigestedVibrationProperties.

Thank you!

And that's it. Thank you for reading this far! I hope this tutorial was helpful in showing you the ropes when it comes to adding custom AI senses. If you found any issues with this article, feel free to correct them. I always appreciate contributions to my work!

Anyways, that's it from me. Creator out!

Further Reading

This wiki page is inspried by this great tutorial about adding custom AI senses.

Other tutorials about the AIPerception system:

• https://sologamedevblog.com/tutorials/unreal-perception-c-friend-or-enemy/
• https://unrealistic.dev/posts/journey-into-ai-perception