C++ API Reference

The native interface and the SDK’s canonical, most complete surface. All public types live in the affdex::vision namespace (pipeline types in affdex::vision::event; shared basics in affdex).

Header	Provides
<Frame.h>	Frame, ColorFormat, Rotation
<Detector.h>	Detector base class, BaselineMode
<FrameDetector.h> / <SyncFrameDetector.h>	the two concrete detectors
<Face.h>	Face and the metric enums
<Feature.h>	Feature enum + string conversions
<FaceListener.h> / <ProcessStatusListener.h> / <ImageListener.h>	callback interfaces
<Sensors.h> / <FaceConnector.h> / <MetricsConnector.h> / <Events.h>	the sensor/connector pipeline
<Core.h> / <Exception.h> / <BoundingBox.h>	core types, Exception, Point / BoundingBox

Core types

Type	Definition	Notes
affdex::Timestamp	int64_t	a point in time, in milliseconds
affdex::Duration	int64_t	elapsed time in milliseconds
affdex::vision::Id	int	generic identifier type
affdex::vision::FaceId	Id	per-face tracking identifier, stable while a face remains tracked
affdex::Byte	unsigned char	pixel data element
affdex::Str / affdex::Path	std::wstring on Windows, else std::string	use the STR("...") macro for portable literals

Point

A 2-D coordinate in image pixels.

BoundingBox

An axis-aligned rectangle defined by top-left and bottom-right points. The constructor throws affdex::Exception if br is above or left of tl. Note that Face::getBoundingBox() returns a std::vector<Point> (the two corners), not a BoundingBox.

Method	Description
BoundingBox(Point tl, Point br)	constructor
Point getTopLeft() const / Point getBottomRight() const	corner accessors
float area() const	rectangle area in px²
bool isEmpty() const	true if width or height is zero
bool contains(const BoundingBox&) const	full containment test
bool overlaps(const BoundingBox&) const	intersection test
float overlappingArea(const BoundingBox&) const	area of the intersection in px²

affdex::Exception

Derived from std::runtime_error; every SDK error throws this type. Errors raised on background processing threads are delivered through a ProcessStatusListener instead of being thrown.

Frame

Wraps one image and its timestamp. Frames are cheap value objects — construct one per captured image and pass it to process(). The pixel buffer is consumed and converted during construction, so it only needs to remain valid for the constructor call.

Method	Description
Timestamp getTimestamp() const	timestamp in ms
int getWidth() const / int getHeight() const	dimensions in px
ColorFormat getColorFormat() const	stored format — may differ from the constructor argument after internal conversion
std::shared_ptr<affdex::Byte> getBGRByteArray() const	pixel data as BGR bytes
int getBGRByteArrayLength() const	length of that byte array
static std::string colorFormatToString(ColorFormat) static std::string rotationToString(Rotation)	debug string helpers

Frame::ColorFormat

Value	Layout
RGB	24-bit, R-G-B channel order
BGR	24-bit, B-G-R channel order (OpenCV default)
RGBA	32-bit, R-G-B-A channel order
BGRA	32-bit, B-G-R-A channel order
YUV_NV21	12-bit YUV, NV21 encoding (Android camera)
YUV_I420	12-bit YUV, I420 encoding
YUV_YUY2	16-bit YUV, YUY2 encoding
GRAY	8-bit grayscale (e.g. near-IR sensors)

Frame::Rotation

Declares how the image is rotated relative to an upright face, so you never rotate pixel buffers yourself: UPRIGHT, CW_90, CW_180, CW_270 (clockwise).

Detectors

Detector (base class)

Common behavior shared by both concrete detectors; never instantiated directly.

Lifecycle

Method	Description
void start(const std::string& license = "")	validates the license and initializes processing. An empty string falls back to the AFFECTIVA_LICENSE_KEY environment variable; throws affdex::Exception if no valid license is found.
void stop()	stops processing and deallocates resources
void reset()	re-initializes for a new session (new video/camera) without stop/start; frame timestamps may restart at 0 afterwards
bool isRunning()	true after start() and before stop()

Feature control

Features select which metric families are computed. Face detection/tracking metrics (bounding box, face points, measurements, qualities, confidence) are always delivered for tracked faces; expressions and emotions are computed only when enabled. Enabling fewer features reduces CPU load.

Method	Description
void enable(Feature) / void enable(const std::set<Feature>&)	adds to the enabled set
void disable(Feature) / void disable(const std::set<Feature>&)	removes from the enabled set
bool isEnabled(Feature) const	true if the feature is enabled
bool isEnabled(const std::set<Feature>&) const	true only if all given features are enabled
std::set<Feature> getEnabledFeatures() const	the currently enabled set

Listener registration

Method	Description
void setFaceListener(FaceListener*)	register the results callback; the SDK does not take ownership — the listener must outlive its registration
void setProcessStatusListener(ProcessStatusListener*)	register the background-error callback
void setImageListener(ImageListener*) Deprecated	use setFaceListener instead

Connector pipeline

Method	Description
void connect(event::FaceConnector&) void connect(std::shared_ptr<event::FaceConnector>)	attaches a connector; automatically enables the features it declares. With the reference overload, the caller must keep the connector alive while connected.
template<typename T, typename... Args> void connectChain(T& first, Args&&... args)	connects a series of source/sink stages in one call (overloads for references, shared_ptr, and unique_ptr)
void disconnect(...)	detaches a connector (same overloads as connect)
void disconnectAllMetricsConnectors()	detaches everything

Detector::BaselineMode

• OFF — metrics are calculated statically (default).
• ON — metrics are calculated against a rolling-average “baseline”, adapting to the individual’s neutral expression over time.

Selected via the detector constructors that take a BaselineMode first argument.

FrameDetector — asynchronous

Queues pushed frames and processes them on a background thread with real-time semantics: frames that arrive closer together than 1 / processFrameRate are discarded so processing never falls behind. Callbacks are invoked on the background thread — synchronize shared state.

SyncFrameDetector — synchronous

Processes every pushed frame (never drops) and invokes callbacks on the thread that called process(), which blocks until the frame’s callbacks complete. Best for offline video analysis and deterministic tests.

Face

One tracked face in a processed frame, delivered through your listener as std::map<FaceId, Face>. All getters return copies; the object is safe to store.

Method	Returns
FaceId getId() const	tracking identifier, stable while the face stays in view
float getConfidence() const	confidence in the presence of the face, 0–100
std::map<Expression, float> getExpressions() const	one value per enabled expression
std::map<Emotion, float> getEmotions() const	one value per enabled emotion
std::map<Measurement, float> getMeasurements() const	head pose and measurements
std::map<Quality, float> getQualities() const	image quality metrics
std::vector<Point> getBoundingBox() const	two points: top-left, then bottom-right
std::map<FacePoint, Point> getFacePoints() const	landmark positions in px

Listeners

FaceListener

The primary callback interface. Subclass it, override all three methods, and register with setFaceListener() before start().

Callback	Called when
void onFaceFound(Timestamp timestamp, FaceId faceId)	the tracker starts tracking a new face
void onFaceLost(Timestamp timestamp, FaceId faceId)	a tracked face is no longer found
void onFaceResults(std::map<FaceId, Face> faces, const Frame& frame)	a frame has been processed; delivers all tracked faces with their metrics

ProcessStatusListener

ImageListener Deprecated

Superseded by FaceListener. Callbacks: onImageResults(std::map<FaceId, Face>, Frame) and onImageCapture(Frame) (fires for every input frame, including frames the asynchronous detector chose not to process).

Sensors & connectors

A composable way to consume metrics: instead of reading whole Face objects in a listener, connect small “sensor” objects to a detector, each of which re-emits one metric (or one metric family) to downstream sinks — including plain lambdas. Connecting a sensor automatically enables the features it needs.

example — stream just the smile value

#include <Sensors.h>

using namespace affdex::vision;

event::ExpressionSensor smileSensor(Expression::SMILE);
detector.connect(smileSensor);                    // auto-enables Feature::EXPRESSIONS

smileSensor.connect(
    [](const std::map<Id, float>& valuesByFace, affdex::Timestamp ts)
    {
      for (const auto& [faceId, smile] : valuesByFace)
        std::cout << ts << " ms  face " << faceId << "  smile=" << smile << "n";
    });

Available sensors

Sensor	Constructor argument	Emits (per face)	Auto-enables
event::AllExpressionsSensor	—	std::map<Expression, float>	EXPRESSIONS
event::ExpressionSensor	Expression	float	EXPRESSIONS
event::AllEmotionsSensor	—	std::map<Emotion, float>	EMOTIONS
event::EmotionSensor	Emotion	float	EMOTIONS
event::AllMeasurementsSensor	—	std::map<Measurement, float>	—
event::MeasurementSensor	Measurement	float	—
event::AllFacePointsSensor	—	std::map<FacePoint, Point>	—
event::FacePointSensor	FacePoint	Point	—
event::QualitySensor	Quality	float	—
event::ConfidenceSensor	—	float	—

Underlying interfaces

• event::MetricsConnector<T> — base interface for anything a detector can feed: override void onResults(const std::map<Id, T>& metrics, Timestamp timestamp_ms); declare required features via the constructor so the detector enables them on connect().
• event::FaceConnector — MetricsConnector<Face>; the type Detector::connect() accepts.
• affdex::event::Source<O> / Sink<I> — the generic dataflow primitives behind sensors. Source::connect() accepts a Sink, a shared_ptr<Sink>, or a lambda of type std::function<void(const std::map<Id, O>&, Timestamp)>.

Enumerations

The tables below give concise definitions. For a full explanation of each metric — the facial actions behind it, value semantics, and interpretation guidance — see the Metrics guide.

Feature

Metric families that can be enabled per detector. Helpers in <Feature.h>: featureToString(), stringToFeature(), and the set ALL_FEATURES.

Value	Enables
Feature::FACES	face detection/tracking metrics
Feature::EXPRESSIONS	the Expression family
Feature::EMOTIONS	the Emotion family

Expression

Facial-action metrics. Continuous metrics range 0 (absent) to 100 (fully present); event metrics report 0 or 100. The vector ALL_EXPRESSIONS contains every value.

Value	Description	Range
SMILE	upward pull of the lip corners	0–100
INNER_BROW_RAISE	raising of the inner brow corners	0–100
BROW_RAISE	raising of both brows	0–100
BROW_FURROW	lowering/drawing together of the brows	0–100
NOSE_WRINKLE	wrinkling of the nose	0–100
UPPER_LIP_RAISE	raising of the upper lip	0–100
LIP_CORNER_DEPRESSOR	downward pull of the lip corners	0–100
CHIN_RAISE	upward push of the chin boss	0–100
LIP_PUCKER	puckering of the lips	0–100
LIP_PRESS	pressing the lips together	0–100
LIP_SUCK	pulling the lips inward	0–100
MOUTH_OPEN	parting of the lips	0–100
SMIRK	asymmetric lip corner pull	0–100
EYE_CLOSURE	both eyelids closed	0–100
SPEAKING	the person is speaking	0–100
ATTENTION	visual attention toward the camera (head orientation)	0–100
EYE_WIDEN	widening of the eyes	0–100
CHEEK_RAISE	raising of the cheeks	0–100
LID_TIGHTEN	tightening of the eyelids	0–100
DIMPLER	tightening of the lip corners (dimpling)	0–100
LIP_STRETCH	horizontal stretching of the lips	0–100
JAW_DROP	lowering of the jaw	0–100
BLINK	a blink occurred	0 / 100
BLINK_RATE	blinks per minute; NaN during the first 10 s after a face is detected	blinks/min
YAWN	a yawn occurred	0 / 100
PAIN	continuous estimate that the person is expressing pain; 50 is the model’s decision threshold, so values above 50 mean pain is more likely present than not	0–100

Emotion

Emotion estimates, 0–100 except the valence metrics (−100 to +100). The vector ALL_EMOTIONS contains every value.

Value	Description	Range
JOY	degree of expressed joy	0–100
FEAR	degree of expressed fear	0–100
DISGUST	degree of expressed disgust	0–100
SADNESS	degree of expressed sadness	0–100
ANGER	degree of expressed anger	0–100
SURPRISE	degree of expressed surprise	0–100
CONTEMPT	degree of expressed contempt	0–100
NEUTRAL	degree to which the face shows no emotion	0–100
SENTIMENTALITY	degree of expressed sentimentality	0–100
CONFUSION	degree of expressed confusion	0–100
VALENCE	overall positivity (+) or negativity (−) of the expression	−100–100
ENGAGEMENT	overall facial expressiveness	0–100
CONVERSATIONAL_VALENCE	valence variant tuned for conversational settings	−100–100
CONVERSATIONAL_ENGAGEMENT	engagement variant tuned for conversational settings	0–100
ADAPTIVE_VALENCE	valence variant that adapts to the individual’s baseline	−100–100
ADAPTIVE_ENGAGEMENT	engagement variant that adapts to the individual’s baseline	0–100

Measurement

Head-pose angles are in degrees, relative to a reference position of the face looking straight at the camera (0/0/0). The vector ALL_MEASUREMENTS contains every value.

Value	Description	Unit
PITCH	positive = head tilted up, negative = tilted down	degrees
YAW	positive = head turned to the person’s right, negative = to their left	degrees
ROLL	positive = head tilted to the person’s left, negative = to their right	degrees
INTEROCULAR_DISTANCE	distance between the two outer eye corner landmarks	pixels

FacePoint

Landmark identifiers for getFacePoints(); each maps to a Point in image pixels: OUTER_RIGHT_EYE, OUTER_LEFT_EYE, NOSE_TIP, CHIN_TIP. The vector ALL_FACE_POINTS contains every value.

Quality

Image-quality metrics for getQualities(): BRIGHTNESS (0–100) — useful for detecting under- or over-exposed capture conditions. The vector ALL_QUALITIES contains every value.

Constants & defaults

Constant	Value	Meaning
Detector::DEFAULT_PROCESSING_FRAMERATE	30	default target processing rate (fps)
Detector::DEFAULT_MAX_NUM_FACES	1	default maximum tracked faces
Detector::DEFAULT_MAX_THREAD_POOL_SIZE	0	0 = “use number of CPU cores − 1” analysis threads
ALL_EXPRESSIONS, ALL_EMOTIONS, ALL_MEASUREMENTS, ALL_FACE_POINTS, ALL_QUALITIES, ALL_FEATURES	—	collections of every value of each enum

Deprecated APIs

These remain functional in 5.4 but will be removed in a future release; new code should not use them.

Deprecated	Use instead
ImageListener and Detector::setImageListener()	FaceListener / setFaceListener()
Frame(width, height, pixels, color_format[, rotation], timestamp_ms) — timestamp-last constructors	the timestamp-first constructor: Frame(timestamp_ms, width, height, pixels, color_format[, rotation])