Getting Started - RE.DOCTOR

Getting Started
getting started

Getting Started

By: / / Comments Off on Getting Started

In order to run the RE.DOCTOR Face Scan SDK by yourself, you’ll need three things:

  • API Key – you can get one by contacting us at [email protected]
  • SDK binary – you will receive after signing the contract
  • App code – the best way to arrive at a working app quickly is to start with one of our demo apps.

Choosing Face Scan SDK mode

Full UX Flow

In Full UX Flow mode, the SDK provides a complete user experience flow, guiding the user through the measurement process and displaying the results. This mode is the default mode and is recommended for most use cases. It requires the least amount of effort to integrate but does not allow for customization of the user interface.

Camera feedback

In Camera Feedback mode, the SDK provides a simple camera feedback component, displaying a camera preview along with real-time visualization. The feedback elements may optionally be turned off, down to a minimum of just displaying the camera feed. This mode is recommended for use cases where a customized user interface is required.

No visual elements

The SDK may also be used without any visual elements at all. This mode is recommended for use cases where the SDK is used to perform measurements without any user interaction such as in the background or if camera feedback is provided by other means.


To customize the visual experience of the SDK, you will need to set the appropriate options when initializing the SDK or later at runtime (see Configuration).

Authorization

License

The SDK can only be used with an active license. The license is fetched automatically whenever the SDK is initialized – internet connection is required for the first SDK usage on any device.

On mobile devices the license is locally persisted and valid for 3 days without requiring connectivity with the licensing server.

API key

In order to initialize the SDK on any platform, you’ll need an API key. The API key consists of 32 hex characters (such as aaaa00baba11abba1081081234567890) and is connected with a particular license.

System requirements

Camera

The SDK requires a camera device that can be accessed via native platform APIs and provides stable 30 FPS video with the resolution of at least 640×480 pixels.

Note that a higher quality camera sensor can enable more accurate measurements under worse light and stability conditions.

Network

Internet connectivity is required for license validation, and optional for telemetry and crash reporting.

Beyond that, there are minimum requirements depending on the target platform.


Android

Android Version Android 10.0 (API level 29) or higher
CPU architecture arm64-v8a
RAM at least 4 GB
OpenGL ES version at least 3.0

iOS

iOS Version 14 or higher
minimum model iPhone X

Web

The Web version of the SDK utilizes some of the most recent web standards in order to make high-performance computations in the browser possible. Browser support for those features is limited .

Client browser requirements

💡
TLDR: Google Chrome Desktop 94+ (or equivalent) is required.

The following features must be available on the client browser in order to use the SDK:

See feature summary (intersection) at the bottom here(opens in a new tab).

Server requirements

Transfer requirements

  • One-time ~30MB WebAssembly download (after compression)

Installation

To add the SDK to an existing app, see documentation for the relevant platform:

See Web for installation instructions.

Permissions

To utilize the SDK, your app will need to have the Network and Camera permissions.

It’s not necessary to set any permissions upfront for the Web SDK.

By default, the SDK will open the camera after being initialized, which will prompt the user for camera access. If you’d like to control the moment of asking for permissions:

  • you can briefly open and close the camera before initializing the SDK
  • or you can initialize the SDK with the camera off, and then turn it on when you’re ready to ask for permissions (see Configuration).

Initialization

Before starting the video measurement, you’ll need to initialize the SDK.

Under the hood that step verifies the license and connects to a native platform camera device. The app must have camera permissions before this step.

You’ll need to provide an API key – see Authorization for more information.

index.ts
import LoadShenaiSDK from "shenai-sdk"; const shenai_sdk = await LoadShenaiSDK(); const API_KEY = "";const USER_ID = ""; shenai.initialize(API_KEY, USER_ID, (initResult) => {});

Error handling

You should check if the initialization was successful before proceeding further.

Possible errors at this stage include:

  • INVALID_API_KEY – the provided API key is malformed or inactive
  • CONNECTION_ERROR – there is a problem with network connection to the licensing server
  • INTERNAL_ERROR – other errors such as missing camera permissions
enum InitializationResult {  OK,  INVALID_API_KEY,  CONNECTION_ERROR,  INTERNAL_ERROR,} if (initResult != shenai_sdk.InitializationResult.OK) {  // handle error}

If the SDK has been successfully initialized, repeated calls to initialize will immediately return success – to start a new session first deinitialize the SDK.

Custom initialization options

The SDK can be configured at the initialization stage to determine it’s behavior and visual appearance. The options can also be changed at runtime.

See Configuration for detailed information on each option.

Providing the user interface

Unless you’re implementing a custom solution that doesn’t require a user interface, you’ll need to embed the SDK’s UI in your app. The SDK uses OpenGL/WebGL for rendering, embedded in a platform-specific native view component.

To display the SDK output, you’ll need to provide a canvas element.

If you create a canvas with id="mxcanvas", the SDK will automatically connect to it on initialization and use it for rendering.

<canvas id="mxcanvas" />

Alternatively, you can connect the canvas manually using the attachToCanvas method.

<canvas id="not-mxcanvas" />...shenai_sdk.attachToCanvas("#not-mxcanvas");

You should use CSS to provide size boundaries for the canvas. The SDK will automatically expand the canvas to fit the available space. Don’t set directly the width and height attributes of the canvas as those will be set by the SDK itself to properly size the rendering buffer.

Deinitialization

After finishing using the SDK you should deinitialize it – that will free up any allocated resources and disconnect from the camera.

shenai_sdk.deinitialize();

Configuration

You can use the SDK with the default configuration or customize it to your needs.

You can visit the Web Playground at playground to see all configuration options in context (you’ll need an API key to access the playground).

Initializing the SDK without providing a custom configuration will use the default one – a one-minute strict-precision HR/HRV/BR measurement with full user interface. You can customize the behaviour and appearance of the SDK by providing a custom configuration object at initialization or by changing the configuration properties at runtime.

Measurement settings

OperatingMode

The SDK can be in one of two operating modes:

  • Positioning: the SDK will track the user’s face position and give feedback to the user to keep the face in a proper position for starting the measurement. This is the default mode after the SDK is initialized.
  • Measure: the SDK will start the measurement as soon as the face is in a proper position.

Clicking on the START/STOP button in the embedded user interface will switch between those two modes. You can also switch between those modes programmatically or start the SDK in the Measure mode by providing a custom configuration object at initialization:

shenaiSDK.initialize(..., {  operatingMode: shenaiSDK.OperatingMode.MEASURE,  ...}); const mode = shenaiSDK.getOperatingMode(); shenaiSDK.setOperatingMode(shenaiSDK.OperatingMode.POSITIONING);shenaiSDK.setOperatingMode(shenaiSDK.OperatingMode.MEASURE); 

PrecisionMode

By default, the SDK will use the strict precision mode, which will only return results if the signal quality is high enough (the threshold is determined by tests on internal datasets with high-quality ECG-based Ground Truth). You may choose to use the relaxed precision mode, which will return results even if the signal quality is low.

  • Strict: the SDK will only return results if the signal quality is high enough. This is the default mode after the SDK is initialized.
  • Relaxed: the SDK will return results on a best-effort basis. The results obtained in this mode should not be relied upon and represent the best guess given the current conditions.
shenaiSDK.initialize(..., {  precisionMode: shenaiSDK.PrecisionMode.STRICT,  ...}); const mode = shenaiSDK.getPrecisionMode(); shenaiSDK.setPrecisionMode(shenaiSDK.PrecisionMode.STRICT);shenaiSDK.setPrecisionMode(shenaiSDK.PrecisionMode.RELAXED); 

MeasurementPreset

At this time, there are three measurement presets available:

  • OneMinuteHrHrvBr: a one-minute measurement that returns Heart Rate, Heart Rate Variability and Breathing Rate. This is the default preset after the SDK is initialized.
  • OneMinuteBetaMetrics: a one-minute measurement that includes metrics that are currently in beta – Blood Pressure and Stress Index
  • InfiniteHr: an infinite measurement that provides continous Heart Rate without any end condition.
shenaiSDK.initialize(..., {measurementPreset: shenaiSDK.MeasurementPreset.ONE_MINUTE_HR_HRV_BR, ...}); const preset = shenaiSDK.getMeasurementPreset(); shenaiSDK.setMeasurementPreset(shenaiSDK.MeasurementPreset.ONE_MINUTE_HR_HRV_BR);shenaiSDK.setMeasurementPreset(shenaiSDK.MeasurementPreset.ONE_MINUTE_BETA_METRICS);shenaiSDK.setMeasurementPreset(shenaiSDK.MeasurementPreset.INFINITE_HR); 

CameraMode

The SDK can be in one of three camera modes:

  • Off: the camera is turned off.
  • FacingUser: the user-facing camera of the device is used (such as the front camera on a phone). This is the default mode after the SDK is initialized.
  • FacingEnvironment: the outside-facing camera of the device is used (such as the back camera on a phone).
 shenaiSDK.initialize(..., {cameraMode: shenaiSDK.CameraMode.FACING_USER,...}); const mode = shenaiSDK.getCameraMode(); shenaiSDK.setCameraMode(shenaiSDK.CameraMode.OFF);shenaiSDK.setCameraMode(shenaiSDK.CameraMode.FACING_USER);shenaiSDK.setCameraMode(shenaiSDK.CameraMode.FACING_ENVIRONMENT); 

Preview settings

You can customize the user feedback interface by changing the following settings:

  • ShowUserInterface controls whether the user feedback interface is shown. If set to false, the SDK will not display any interactive elements such as the START/STOP button.
  • ShowFacePositioningOverlay controls whether the face positioning overlay is shown. If set to false, the SDK will not display the overlay that guides the user to position their face correctly.
  • ShowVisualWarnings controls whether visual warnings are shown. If set to false, the SDK will not display any visual warnings such as the red bulb that indicates that the signal quality is low.
  • EnableCameraSwap controls whether the user can switch between the front and back camera. If set to false, the user will not be able to switch between the front and back camera.
  • ShowFaceMask controls whether the face mask visualization is shown. If set to false, the SDK will not display the 3d face mask visualization.
  • ShowBloodFlow controls whether the blood flow visualization is shown. If set to false, the SDK will not display the blood flow visualization as an overlay on the user’s face.
  • EnableStartAfterSuccess controls whether the START button is available after a successful measurement. If set to false, the START button will not be available after a successful measurement until the SDK is reinitialized.
shenaiSDK.setShowUserInterface(true);const showUserInterface = shenaiSDK.getShowUserInterface(); shenaiSDK.setShowFacePositioningOverlay(true);const showFacePositioningOverlay = shenaiSDK.getShowFacePositioningOverlay(); shenaiSDK.setShowVisualWarnings(true);const showVisualWarnings = shenaiSDK.getShowVisualWarnings(); shenaiSDK.setEnableCameraSwap(true);const enableCameraSwap = shenaiSDK.getEnableCameraSwap(); shenaiSDK.setShowFaceMask(true);const showFaceMask = shenaiSDK.getShowFaceMask(); shenaiSDK.setShowBloodFlow(true);const showBloodFlow = shenaiSDK.getShowBloodFlow();



en_USEnglish