K4A.Net (K4AdotNet)

K4A.Net — Three-in-one managed .NET library to work with Azure Kinect depth sensors (also known as Kinect for Azure, K4A, Kinect v4). Also it can be used to work with Orbbec Femto Bolt and Mega depth sensors via Orbbec SDK K4A Wrapper. It consists of the following "components":

1. Sensor API — access to depth camera, RGB camera, accelerometer and gyroscope, plus device-calibration data and synchronization control
   • Corresponding namespace: K4AdotNet.Sensor
   • Corresponding native API: k4a.h
2. Record API — data recording from device to MKV-files, and data reading from such files
   • Corresponding namespace: K4AdotNet.Record
   • Corresponding native API: record.h and playback.h
3. Body Tracking API — body tracking of multiple skeletons including eyes, ears and nose
   • Corresponding namespace: K4AdotNet.BodyTracking
   • Corresponding native API: k4abt.h

Key features

• Written fully on C#
• CLS-compliant (can be used from any .Net-compatible language, including C#, F#, VB.Net)
• Library K4AdotNet is compiled against NET 6.0, .NET Standard 2.0 and .NET Framework 4.6.1 target frameworks
  • This makes it compatible with .NET 5-8, .NET Core 2.0-3.1, .NET Framework 4.6.1 and later, Unity 2018.1 and later, etc.
  • See https://docs.microsoft.com/en-us/dotnet/standard/net-standard for details
• Clean API, which is close to C/C++ native API from Azure Kinect Sensor SDK (or Orbbec SDK K4A Wrapper) and Azure Kinect Body Tracking SDK.
• Plus useful helper methods, additional checks and meaningful exceptions.
• Full feature set (all API provided by native SDKs are available in this C# wrapper)
• Up-to-date with the latest versions of native SDKs
• No additional dependencies
  • Except dependencies on native libraries (DLLs) from Azure Kinect Sensor SDK and Azure Kinect Body Tracking SDK
  • Native libraries from Azure Kinect Sensor SDK are included to repository (see externals directory) and NuGet package
  • But native libraries from Azure Kinect Body Tracking SDK are not included to repository. It is recommended to install Azure Kinect Body Tracking SDK separately. For details see below
• Plenty of powerful samples:
  • .NET 7 Console Application
  • .NET 7 WPF Application (Windows only)
  • Unity Project
• Well documented
• Unit-tested
• Multi-platform (Windows, Linux)
  • But most of samples are written using WPF
• Available as NuGet packages:
  • Azure Kinect version: https://www.nuget.org/packages/K4AdotNet
  • Orbbec Femto Bolt/Mega version: https://www.nuget.org/packages/K4AdotNet-Femto

Dependencies

K4AdotNet depends on the following native libraries (DLLs) from Azure Kinect Sensor SDK and Azure Kinect Body Tracking SDK:

Library "component"	Depends on	Version in use	Location in repository	Included in NuGet package
Sensor API	k4a.dll, depthengine_2_0.dll(1)	1.4.1	externals/k4a/windows-desktop/amd64	YES
Record API	k4arecord.dll	1.4.1	externals/k4a/windows-desktop/amd64	YES
Body Tracking API	k4abt.dll(2), dnn_model_2_0_op11.onnx and/or dnn_model_2_0_lite_op11.onnx	1.1.2		no(3)

Notes:

• ⁽¹⁾ depthengine_2_0.dll is required only if you are using Transformation or Device classes. All other Sensor API (types from K4AdotNet.Sensor namespace) depends only on k4a.dll.
• ⁽²⁾ k4abt.dll uses ONNX Runtime — onnxruntime.dll, which in turn depends on the following NVIDIA cuDNN and NVIDIA CUDA 11.2.1 libraries. Also, Visual C++ Redistributable for Visual Studio 2015 is required: vcomp140.dll.
• ⁽³⁾ The full list of libraries and data files required for Body Tracking:
  k4abt.dll (4.4 MB),
  dnn_model_2_0_op11.onnx (159 MB) and/or dnn_model_2_0_lite_op11.onnx (43 MB),
  cublas64_11.dll (136 MB),
  cublasLt64_11.dll (269 MB),
  cudart64_110.dll (0.5 MB),
  cudnn_cnn_infer64_8.dll (600 MB),
  cudnn_ops_infer64_8.dll (346 MB),
  cudnn64_8.dll (0.3 MB),
  cufft64_10.dll (345 MB),
  vcomp140.dll (0.2 MB).
  (Plus additional libraries for TensorRT and DirectML tracking modes.)
  It is mostly unpractical to have such bulky files in repositories. For this reason they are not included to the repository. Also, they are not included to NuGet package.

How to use Body Tracking runtime:

• The easiest way to use Body Tracking is to ask user to install Body Tracking SDK by him/herself: https://docs.microsoft.com/en-us/azure/Kinect-dk/body-sdk-setup
• But you can also put all required libraries and data files to the output directory of your project (on post-build step, for example). All required libraries and data files can be found in directory tools of Body Tracking SDK.
• K4AdotNet is trying to find Body Tracking runtime in the following locations:
  • directory with executable file
  • directory with K4AdotNet assembly
  • installation directory of Body Tracking SDK under Program Files
• Use bool Sdk.IsBodyTrackingRuntimeAvailable(out string message) method to check if Body Tracking runtime and all required components are available/installed
• Also, you can optionally call bool Sdk.TryInitializeBodyTrackingRuntime(TrackerProcessingMode mode, out string message) method on start of your application to initialize Body Tracking runtime (it can take a few seconds)

Versions

See https://github.com/bibigone/k4a.net/releases

How to build

• Open K4AdotNet.sln in Visual Studio 2022
• Build solution (Ctrl+Shift+B)
• After that you can run and explore samples:
  • K4AdotNet.Samples.Console.BodyTrackingSpeed — sample .NET 7 console application to measure speed of Body Tracking.
  • K4AdotNet.Samples.Console.Recorder — sample .NET 7 console application to record data from Azure Kinect device to MKV file.
  • K4AdotNet.Samples.Wpf.Viewer — sample WPF application to demonstrate usage of Sensor API and Record API.
  • K4AdotNet.Samples.Wpf.BodyTracker — sample WPF application to demonstrate usage of Body Tracking API.
  • K4AdotNet.Samples.Wpf.BackgroundRemover — sample WPF application which implements the background removal effect for color picture with the help of depth data.
• Instruction on building Unity sample can be found here.

Orbbec Femto Bolt/Mega support

This library is compatible with Orbbec Femto Bolt and Mega depth sensors. It is possible due to Orbbec SDK K4A Wrapper that implements the API of original native K4A library via Orbbec SDK.

ORBBECSDK_K4A_WRAPPER constant must be defined to compile the Orbbec-compatible version of the library. See Product.props file for details.

Note. There is no full compatibility between Orbbec SDK K4A Wrapper and original K4A library. Major differences between them are described here. But actually there are much more inconsistencies. Check value of the Sdk.IsOrbbecSdkK4AWrapper property in client code to determine what version of library is in use.