In order to run the RE.DOCTOR Face Scan SDK by yourself, you’ll need three things:
In Full UX Flow mode, the SDK provides a complete user experience flow, guiding the user through the measurement process and displaying the results. This mode is the default mode and is recommended for most use cases. It requires the least amount of effort to integrate but does not allow for customization of the user interface.
In Camera Feedback mode, the SDK provides a simple camera feedback component, displaying a camera preview along with real-time visualization. The feedback elements may optionally be turned off, down to a minimum of just displaying the camera feed. This mode is recommended for use cases where a customized user interface is required.
The SDK may also be used without any visual elements at all. This mode is recommended for use cases where the SDK is used to perform measurements without any user interaction such as in the background or if camera feedback is provided by other means.
To customize the visual experience of the SDK, you will need to set the appropriate options when initializing the SDK or later at runtime (see Configuration).
The SDK can only be used with an active license. The license is fetched automatically whenever the SDK is initialized – internet connection is required for the first SDK usage on any device.
On mobile devices the license is locally persisted and valid for 3 days without requiring connectivity with the licensing server.
In order to initialize the SDK on any platform, you’ll need an API key. The API key consists of 32 hex characters (such as aaaa00baba11abba1081081234567890
) and is connected with a particular license.
The SDK requires a camera device that can be accessed via native platform APIs and provides stable 30 FPS video with the resolution of at least 640×480 pixels.
Note that a higher quality camera sensor can enable more accurate measurements under worse light and stability conditions.
Internet connectivity is required for license validation, and optional for telemetry and crash reporting.
Beyond that, there are minimum requirements depending on the target platform.
Android Version | Android 10.0 (API level 29) or higher |
---|---|
CPU architecture | arm64-v8a |
RAM | at least 4 GB |
OpenGL ES version | at least 3.0 |
iOS Version | 14 or higher |
---|---|
minimum model | iPhone X |
The Web version of the SDK utilizes some of the most recent web standards in order to make high-performance computations in the browser possible. Browser support for those features is limited .
The following features must be available on the client browser in order to use the SDK:
See feature summary (intersection) at the bottom here(opens in a new tab).
To add the SDK to an existing app, see documentation for the relevant platform:
See Web for installation instructions.
To utilize the SDK, your app will need to have the Network and Camera permissions.
It’s not necessary to set any permissions upfront for the Web SDK.
By default, the SDK will open the camera after being initialized, which will prompt the user for camera access. If you’d like to control the moment of asking for permissions:
Before starting the video measurement, you’ll need to initialize the SDK.
Under the hood that step verifies the license and connects to a native platform camera device. The app must have camera permissions before this step.
You’ll need to provide an API key – see Authorization for more information.
import LoadShenaiSDK from "shenai-sdk"; const shenai_sdk = await LoadShenaiSDK(); const API_KEY = "";const USER_ID = ""; shenai.initialize(API_KEY, USER_ID, (initResult) => {});
You should check if the initialization was successful before proceeding further.
Possible errors at this stage include:
INVALID_API_KEY
– the provided API key is malformed or inactiveCONNECTION_ERROR
– there is a problem with network connection to the licensing serverINTERNAL_ERROR
– other errors such as missing camera permissionsenum InitializationResult { OK, INVALID_API_KEY, CONNECTION_ERROR, INTERNAL_ERROR,} if (initResult != shenai_sdk.InitializationResult.OK) { // handle error}
If the SDK has been successfully initialized, repeated calls to initialize
will immediately return success – to start a new session first deinitialize the SDK.
The SDK can be configured at the initialization stage to determine it’s behavior and visual appearance. The options can also be changed at runtime.
See Configuration for detailed information on each option.
Unless you’re implementing a custom solution that doesn’t require a user interface, you’ll need to embed the SDK’s UI in your app. The SDK uses OpenGL/WebGL for rendering, embedded in a platform-specific native view component.
To display the SDK output, you’ll need to provide a canvas
element.
If you create a canvas
with id="mxcanvas"
, the SDK will automatically connect to it on initialization and use it for rendering.
<canvas id="mxcanvas" />
Alternatively, you can connect the canvas manually using the attachToCanvas
method.
<canvas id="not-mxcanvas" />...shenai_sdk.attachToCanvas("#not-mxcanvas");
You should use CSS to provide size boundaries for the canvas. The SDK will automatically expand the canvas to fit the available space. Don’t set directly the width
and height
attributes of the canvas as those will be set by the SDK itself to properly size the rendering buffer.
After finishing using the SDK you should deinitialize it – that will free up any allocated resources and disconnect from the camera.
shenai_sdk.deinitialize();
You can use the SDK with the default configuration or customize it to your needs.
You can visit the Web Playground at playground to see all configuration options in context (you’ll need an API key to access the playground).
Initializing the SDK without providing a custom configuration will use the default one – a one-minute strict-precision HR/HRV/BR measurement with full user interface. You can customize the behaviour and appearance of the SDK by providing a custom configuration object at initialization or by changing the configuration properties at runtime.
OperatingMode
The SDK can be in one of two operating modes:
Positioning
: the SDK will track the user’s face position and give feedback to the user to keep the face in a proper position for starting the measurement. This is the default mode after the SDK is initialized.Measure
: the SDK will start the measurement as soon as the face is in a proper position.Clicking on the START/STOP button in the embedded user interface will switch between those two modes. You can also switch between those modes programmatically or start the SDK in the Measure
mode by providing a custom configuration object at initialization:
shenaiSDK.initialize(..., { operatingMode: shenaiSDK.OperatingMode.MEASURE, ...}); const mode = shenaiSDK.getOperatingMode(); shenaiSDK.setOperatingMode(shenaiSDK.OperatingMode.POSITIONING);shenaiSDK.setOperatingMode(shenaiSDK.OperatingMode.MEASURE);
PrecisionMode
By default, the SDK will use the strict precision mode, which will only return results if the signal quality is high enough (the threshold is determined by tests on internal datasets with high-quality ECG-based Ground Truth). You may choose to use the relaxed precision mode, which will return results even if the signal quality is low.
Strict
: the SDK will only return results if the signal quality is high enough. This is the default mode after the SDK is initialized.Relaxed
: the SDK will return results on a best-effort basis. The results obtained in this mode should not be relied upon and represent the best guess given the current conditions.shenaiSDK.initialize(..., { precisionMode: shenaiSDK.PrecisionMode.STRICT, ...}); const mode = shenaiSDK.getPrecisionMode(); shenaiSDK.setPrecisionMode(shenaiSDK.PrecisionMode.STRICT);shenaiSDK.setPrecisionMode(shenaiSDK.PrecisionMode.RELAXED);
MeasurementPreset
At this time, there are three measurement presets available:
OneMinuteHrHrvBr
: a one-minute measurement that returns Heart Rate, Heart Rate Variability and Breathing Rate. This is the default preset after the SDK is initialized.OneMinuteBetaMetrics
: a one-minute measurement that includes metrics that are currently in beta – Blood Pressure and Stress IndexInfiniteHr
: an infinite measurement that provides continous Heart Rate without any end condition.shenaiSDK.initialize(..., {measurementPreset: shenaiSDK.MeasurementPreset.ONE_MINUTE_HR_HRV_BR, ...}); const preset = shenaiSDK.getMeasurementPreset(); shenaiSDK.setMeasurementPreset(shenaiSDK.MeasurementPreset.ONE_MINUTE_HR_HRV_BR);shenaiSDK.setMeasurementPreset(shenaiSDK.MeasurementPreset.ONE_MINUTE_BETA_METRICS);shenaiSDK.setMeasurementPreset(shenaiSDK.MeasurementPreset.INFINITE_HR);
CameraMode
The SDK can be in one of three camera modes:
Off
: the camera is turned off.FacingUser
: the user-facing camera of the device is used (such as the front camera on a phone). This is the default mode after the SDK is initialized.FacingEnvironment
: the outside-facing camera of the device is used (such as the back camera on a phone). shenaiSDK.initialize(..., {cameraMode: shenaiSDK.CameraMode.FACING_USER,...}); const mode = shenaiSDK.getCameraMode(); shenaiSDK.setCameraMode(shenaiSDK.CameraMode.OFF);shenaiSDK.setCameraMode(shenaiSDK.CameraMode.FACING_USER);shenaiSDK.setCameraMode(shenaiSDK.CameraMode.FACING_ENVIRONMENT);
You can customize the user feedback interface by changing the following settings:
ShowUserInterface
controls whether the user feedback interface is shown. If set to false
, the SDK will not display any interactive elements such as the START/STOP button.ShowFacePositioningOverlay
controls whether the face positioning overlay is shown. If set to false
, the SDK will not display the overlay that guides the user to position their face correctly.ShowVisualWarnings
controls whether visual warnings are shown. If set to false
, the SDK will not display any visual warnings such as the red bulb that indicates that the signal quality is low.EnableCameraSwap
controls whether the user can switch between the front and back camera. If set to false
, the user will not be able to switch between the front and back camera.ShowFaceMask
controls whether the face mask visualization is shown. If set to false
, the SDK will not display the 3d face mask visualization.ShowBloodFlow
controls whether the blood flow visualization is shown. If set to false
, the SDK will not display the blood flow visualization as an overlay on the user’s face.EnableStartAfterSuccess
controls whether the START button is available after a successful measurement. If set to false
, the START button will not be available after a successful measurement until the SDK is reinitialized.shenaiSDK.setShowUserInterface(true);const showUserInterface = shenaiSDK.getShowUserInterface(); shenaiSDK.setShowFacePositioningOverlay(true);const showFacePositioningOverlay = shenaiSDK.getShowFacePositioningOverlay(); shenaiSDK.setShowVisualWarnings(true);const showVisualWarnings = shenaiSDK.getShowVisualWarnings(); shenaiSDK.setEnableCameraSwap(true);const enableCameraSwap = shenaiSDK.getEnableCameraSwap(); shenaiSDK.setShowFaceMask(true);const showFaceMask = shenaiSDK.getShowFaceMask(); shenaiSDK.setShowBloodFlow(true);const showBloodFlow = shenaiSDK.getShowBloodFlow();