Android Quick Start

The Android SDK is a single AAR containing the Java API (com.affectiva.vision) and the native arm64 libraries. You feed it camera frames as byte arrays; it returns Face objects with the same expressions, emotions, and measurements as the other platforms.

1 · Requirements

• Device: 64-bit ARM (arm64-v8a) running Android 8.0 (API level 26) or newer. 32-bit devices and x86 emulator images are not supported; use a physical device or an arm64 emulator image.
• Native libraries are 16 KB page-size aligned, so the AAR is compatible with Android 15+ devices that enforce 16 KB pages.
• License key — passed to Detector.start(license) (see Licensing).

2 · Project setup

The delivered AAR names the platform, NDK, and version, so the filename changes with every release — for example affectiva-emotion-sdk-sdk-android-arm64-ndk24-clang-5.3.1.aar. Drop it into app/libs/ and match it by extension rather than by name, so it stays correct across upgrades:

app/build.gradle

android {
    defaultConfig {
        minSdkVersion 26
        ndk { abiFilters 'arm64-v8a' }   // ship only the supported ABI
    }
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.aar'])
    implementation 'androidx.annotation:annotation:1.7.0'
}

To pin one exact file instead, name it in full:

    implementation files('libs/affectiva-emotion-sdk-sdk-android-arm64-ndk24-clang-5.3.1.aar')

The AAR declares package="com.affectiva.vision" and minSdkVersion 26, and carries native libraries for arm64-v8a only (libaffectiva-vision.so, its JNI wrapper, and the ONNX Runtime libraries — all bundled, nothing extra to ship).

…or, if your organization mirrors the Affectiva Maven repository, depend on the published coordinates:

dependencies {
    implementation 'com.affectiva.android:emotion-sdk:5.4.0'
}

Request the camera permission in your manifest (and at runtime, as usual for dangerous permissions):

AndroidManifest.xml

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required="true" />

3 · Using the detector

The Java API mirrors the C++ flow: construct → enable → listen → start → process. On Android, per-frame results are delivered through ImageListener.onImageResults; FaceListener carries the face found/lost events.

FaceAnalyzer.java

import android.content.Context;
import com.affectiva.vision.Face;
import com.affectiva.vision.FaceListener;
import com.affectiva.vision.Feature;
import com.affectiva.vision.Frame;
import com.affectiva.vision.FrameDetector;
import com.affectiva.vision.ImageListener;
import com.affectiva.vision.ProcessStatusListener;
import com.affectiva.vision.AffectivaException;

import java.util.Arrays;
import java.util.Map;

public class FaceAnalyzer {

    private FrameDetector detector;

    public void create(Context context, String licenseKey) {
        // 1. Construct: 30 fps target, track 1 face, auto thread pool.
        detector = new FrameDetector(context);

        // 2. Enable the metric families you need.
        detector.enable(Arrays.asList(Feature.EXPRESSIONS, Feature.EMOTIONS));

        // 3. Register listeners.
        detector.setImageListener(new ImageListener() {
            @Override
            public void onImageResults(Map<Integer, Face> faces, Frame frame) {
                for (Face face : faces.values()) {
                    Float smile = face.getExpressions().get(Face.Expression.SMILE);
                    Float joy = face.getEmotions().get(Face.Emotion.JOY);
                    android.util.Log.d("Affectiva", "t=" + frame.getTimeStamp()
                            + " face=" + face.getId()
                            + " smile=" + smile + " joy=" + joy);
                }
            }

            @Override
            public void onImageCapture(Frame frame) {
                // called for every input frame, including dropped ones
            }
        });

        detector.setFaceListener(new FaceListener() {
            @Override
            public void onFaceFound(long timestamp, int faceId) { }

            @Override
            public void onFaceLost(long timestamp, int faceId) { }
        });

        detector.setProcessStatusListener(new ProcessStatusListener() {
            @Override
            public void onProcessingException(AffectivaException exception) {
                android.util.Log.e("Affectiva", "processing error", exception);
            }
        });

        // 4. Start with your license key.
        detector.start(licenseKey);
    }

    // 5. Call for every camera frame (NV21 bytes + timestamp in ms).
    public void onCameraFrame(byte[] nv21, int width, int height, long timestampMs) {
        Frame frame = new Frame(width, height, nv21,
                Frame.ColorFormat.YUV_NV21, Frame.Rotation.CW_90, timestampMs);
        detector.process(frame);
    }

    public void destroy() {
        if (detector != null) {
            detector.stop();
            detector.dispose();   // releases the native detector — required
            detector = null;
        }
    }
}

The example, explained

Constructing the detector

detector = new FrameDetector(context);

Input: any Android Context — the SDK uses it to initialize its runtime storage on first run. We use the asynchronous FrameDetector because camera callbacks must never block: it queues each frame for a background thread and returns immediately. The single-argument constructor applies the defaults processingFrameRate = 30 (frames arriving faster than this are dropped, which is what keeps the pipeline real-time), maxNumFaces = 1, and maxThreadPoolSize = 0 (auto: CPU cores − 1).

Enabling features

detector.enable(Arrays.asList(Feature.EXPRESSIONS, Feature.EMOTIONS));

Input: a list of the metric families to compute. Only enabled families cost CPU and appear in the Face maps; tracking metrics (bounding box, measurements, confidence) are always delivered. On Android, Feature has exactly these two values — face tracking itself is always on. Call before start().

ImageListener — where results arrive

detector.setImageListener(new ImageListener() {
    @Override
    public void onImageResults(Map<Integer, Face> faces, Frame frame) {

On Android, ImageListener.onImageResults is the per-frame results callback. Input: a map of tracking id → Face for every currently tracked face (empty if none) and the Frame the results belong to — its getTimeStamp() tells you which moment they describe. Output you read: face.getExpressions() / getEmotions() return Map<Enum, Float> with one entry per enabled metric, values 0–100 (see the Metrics guide). Because this is the asynchronous detector, the callback runs on a background thread — post to the main thread before touching views.

onImageCapture — every input frame

@Override
public void onImageCapture(Frame frame) {
    // called for every input frame, including dropped ones
}

Fires for each frame you submit, including frames the detector drops to honor its processing rate — unlike onImageResults, which fires only for processed frames. Useful for keeping a preview or recording in sync with what you fed in; leave it empty otherwise.

Face events and background errors

detector.setFaceListener(new FaceListener() { ... });

detector.setProcessStatusListener(new ProcessStatusListener() {
    @Override
    public void onProcessingException(AffectivaException exception) {

FaceListener delivers the found/lost moments (input: frame timestamp in ms + tracking id) — on Android it carries only these two events, since results flow through ImageListener. ProcessStatusListener is how errors from the background processing thread reach you; without it they would be invisible. Register all listeners before start().

Starting

detector.start(licenseKey);

Input: your license key string. Validates the license and initializes the embedded models; throws AffectivaException if the key is missing or invalid. There is no environment-variable fallback on Android — always pass the key (load it from your app’s secure configuration, not a hardcoded string, in production).

Feeding camera frames

public void onCameraFrame(byte[] nv21, int width, int height, long timestampMs) {
    Frame frame = new Frame(width, height, nv21,
            Frame.ColorFormat.YUV_NV21, Frame.Rotation.CW_90, timestampMs);
    detector.process(frame);
}

Input: the camera’s pixel buffer plus geometry and timing. For YUV_NV21, the byte array holds width × height × 3/2 bytes (the standard Android camera preview layout). Rotation.CW_90 declares that the sensor image is rotated 90° clockwise relative to an upright face — typical for portrait phone orientation; derive the real value from your camera’s rotation info rather than hardcoding it. The timestamp (ms) must strictly increase; SystemClock.elapsedRealtime() is a good source. process() returns immediately — output arrives later on the background thread via onImageResults, and frames may be silently skipped if they exceed the processing rate.

Tearing down

detector.stop();
detector.dispose();   // releases the native detector — required
detector = null;

stop() halts processing and frees processing resources; dispose() releases the underlying native detector object. Java garbage collection does not reclaim native memory, so skipping dispose() leaks it. After dispose(), further calls on the object are ignored (and logged) — hence dropping the reference.

4 · Camera integration

The SDK does not open the camera; it consumes pixel buffers your app captures. Typical setup with CameraX:

• Use an ImageAnalysis use case. CameraX delivers YUV_420_888 images; convert each to an NV21 byte array (Y plane followed by interleaved VU) before wrapping it in a Frame with ColorFormat.YUV_NV21.
• Pass the frame’s rotation instead of rotating pixels: map imageProxy.getImageInfo().getRotationDegrees() (90/180/270) to Frame.Rotation.CW_90 / CW_180 / CW_270.
• Use SystemClock.elapsedRealtime() (or the image timestamp converted to milliseconds) for the frame timestamp — timestamps must strictly increase.

Other capture stacks (Camera2, legacy Camera with NV21 preview callbacks, video decoders) work the same way: any source that yields one of the supported color formats plus a timestamp can feed the detector.

5 · Lifecycle & threading

• Always call dispose() when you are done with a detector (for example in onDestroy()). It releases the underlying native detector; relying on garbage collection alone leaks native memory.
• FrameDetector is asynchronous: process() returns immediately and callbacks arrive on a background thread. Post results to the main thread before touching views. SyncFrameDetector processes every frame on the calling thread — do not use it on the UI thread.
• Between camera sessions, call reset() instead of recreating the detector; timestamps may then restart at 0.
• Constructor knobs: new FrameDetector(context, processingFrameRate, maxNumFaces, maxThreadPoolSize) — see the detector reference for all overloads and defaults.

Next steps

• The Android API reference documents every Java class and enum — including the Android-specific behaviors (Feature exposes EXPRESSIONS/EMOTIONS; results arrive via ImageListener; errors use AffectivaException).