Release 1.38 - November 14th - Cross-Platform Path Tracer

• The new 16_Raytracing unit test shows a simple cross-platform path tracer. On iOS this path tracer requires A11 or higher. It is meant to be used in tools in the future and doesn't run in real-time.
  To support the new path tracer, the Metal raytracing backend has been overhauled to use a sort-and-dispatch based approach, enabling efficient support for multiple hit groups and miss shaders. The most significant limitation for raytracing on Metal is that only tail recursion is supported, which can be worked around using larger per-ray payloads and splitting up shaders into sub-shaders after each TraceRay call; see the Metal shaders used for 16_Raytracing for an example on how this can be done.

macOS 1920x1080 AMD Pro Vega 64

iOS iPhone X 812x375

Windows 10 1080p NVIDIA RTX 2080 with DXR Driver version 441.12

Windows 10 1080p NVIDIA RTX 2080 with RTX Driver version 441.12

Linux 1080p NVIDIA RTX 2060 with RTX Driver version 435

• File System: Fixed an issue wherein compiled shader binaries weren’t being saved to the RD_SHADER_BINARIES resource directory
• GitHub issues fixed:
  • #150 - [Vulkan] Failed to extend descriptor pool
  • #151 - [Vulkan] rootcbv of detection is case sensitive
  • #152 - [Vulkan] updateDescriptorSet is different from the DirectX12

Release 1.37 - October 30th - New Features Ephemeris 2 | Update Metal

The Forge Interactive Inc., the company behind The Forge became a Khronos Associate member.

• Ephemeris 2
  • New features
    • Add Earth radius: controls the radius of clouds' radius with scale factor. The clouds field will be flatter and the user can see further along the horizon if the radius increase
    • Add noise flow: controls the direction and intensity of clouds' noise flow
    • Add rotation: rotates clouds based on a certain pivot position.
    • Add the second layer: it is possible to generate the second cloud layer which can act, independently
    • Add FXAA
  • Improvement
    • Ray-marching: now, hard-edge artifact is significantly reduced
    • Silver-lining: improved its quality
    • God ray: improved its quality
  • Performance: up to 25% performance increased

Click on the following image to see a video:

Head over to Custom Middleware to check out the source code.

• macOS / iPad / iOS: ICB support for Metal renderer (draw; draw indexed; pipeline state switch with ICB; ICB optimization with BlitEncoder). The Visibility Buffer example now uses ICB features on MacOS
  • reduced memory consumption for argument buffers in Metal
  • fixes for Metal implementation of descriptor set
  • minor fixes and optimizations in Metal renderer
  • Due to bugs in the run-time for argument buffers we still can't run unit test 04, 06, and 10

The Visibility Buffer example runs now faster on macOS

Release 1.36 - October 18th - New File System

• New cross-platform FileSystem C API, supporting disk-based files, memory streams, and files in zip archives. The API can be viewed in IFileSystem.h, and all of the example code has been updated to use the new API.
  • The API is based around Paths, where each Path represents an absolute, canonical path string on a particular file system. You can query information about the files at Paths, open files as FileStreams, and copy files between different Paths.
  • The concept of FileSystemRoots has been replaced by ResourceDirectorys. ResourceDirectorys are predefined directories where resources are expected to exist, and there are convenience functions to open files in resource directories. If your resources don’t exist within the default directory for a particular resource type, you can call fsSetPathForResourceDirectory to relocate the resource directory; see the unit tests for sample code on how to do this.
  • There's a new 12_FileSystem unit test that demonstrates how to read files from zip archives:

• Vulkan: Adaptive Order Independent Transparency with Raster Order Views is now supported when VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FRAGMENT_SHADER_INTERLOCK_FEATURES_EXT is supported.

Release 1.35 - October 3rd - New Ozz Animation Unit Test | Updated Shader Translator | Maintenance update macOS / iOS

After only a bit more than one week we wanted to ship a quick update ...

• Ozz Animation System: there is a new unit test added to The Forge.

Ozz Inverse Kinematic: This unit test shows how to use Aim and Two bone IK solvers

• Shader Translator:
  • Implicit cast fixes submitted for review
  • Added automated testing on mac via ssh from Windows - still requires integration in CI.
  • Argument buffer support
  • Fixed scalar swizzle in MSL
  • Fixed texture array output in MSL
  • Fixed indentation in MSL for compute shaders
  • Fixed redundant parenthesis in if statement warning in HLSL and MSL
  • Fixed texture object Load method output for MSL
• Maintenance Update macOS / iOS:
  • Unified depth-stencil texture handling across macOS and iOS. To use a combined
  • The TEXTURE_CREATION_FLAG_ON_TILE flag now specifies that a render target attachment should neither be loaded or stored, in addition to ensuring that the attachment is memoryless on iOS
  • Added TEXTURE_CREATION_FLAG_ON_TILE to the depth buffers within many of the unit tests
  • Fixed a resource barrier issue in the Tessellation unit test on macOS and iOS
  • We still encounter many bugs especially on INTEL based devices. We have RADARs with Apple and we are hoping those will go away over time. Let us know if you need any help with the macOS / iOS run-time.

If you want to join us in sunny Encinitas, CA, USA as a graphics programmer, we would like to hear from you. If you are interested in working in our offices world-wide please let us know as well. We are in Spain, Netherlands, Ukraine, India, New Zealand and other places.

Release 1.34 - September 23rd - TinyImageFormat | Microprofiler uses imGUI | 4th gen Descriptor System

• TinyImageFormat support: Deano Calver @DeanoC added his image format support library to The Forge. TinyImageFormat provides a query mechanism and encode/decode for many CPU and GPU image formats. Allowing you to use whatever pixel data is best whether its loading/saving or procedural generation.
• Microprofiler: @zeuxcg this is the third rewrite of the Microprofiler. We replaced the proprietary UI with imGUI and simplified the usage. Now it is much more tightly and consistently integrated in our code base.

Here are screenshots of the Microprofiler running a unit test on iOS:

Check out the Wikipage for an explanation on how to use it.

• Descriptor System: @gavkar this is at least the fourth rewrite of the descriptor system now with support for the brand new argument buffers on macOS / iOS. It requires latest OS and XCode versions. This is a major update to the macOS / iOS runtime and it comes with many implementation changes:

  • improved Metal resource usage with useHeaps and useResorces

  • Metal shader reflection system was refactored

  • fixes and optimizations for some unit tests on MacOS and iOS platforms

  • initial support for paramIndex

  • more informative debug labels for Metal resources

    This is probably the first engine integration of argument buffers, so there are issues with the following unit tests on iPadOS/iOS platforms:

  • 04_ExecuteIndirect_iOS: GPU hangs due to argument buffers corruption in latest iOS 13.1. The bug doesn't occur if validation layer is enabled!

  • 06_MaterialPlayground_iOS: Fails to compile shaders trying to write to Texture2DArray (iOS 13.1 beta 2 & 3)

  • 10_PixelProjectedReflections on iOS: iOS Metal shader compiler crashes

  Check out the Wikipage for a high-level view of the architecture.

• Light & Shadow Playground: we cleaned up the code base of the Light & Shadow Playground and integrated missing pieces into The Forge Eco system.

• Shader Translator: a lot of work went into the Shader Translator since the last release. Let us know how it works for you. There is a how-to on the Wiki page here:

  How to use the Shader Translator

• Issues: fixed #129 "Metal backend buffer namespace collision"

Release 1.33 - August 30th - glTF model viewer | Basis Universal Texture Support | Updated Shader Translator | New Input System Architecture

• glTF model viewer - we build a simple cross-platform glTF model viewer by integrating Arseny Kapoulkine @zeuxcg excellent meshoptimizer and the same PBR as used in the Material Playground unit test.
  • It optimizes the geometry data set with all the features meshoptimizer offers
  • It will be extended in the future with more functionality, following some of our internal tools
  • Uses the cgltf reader from the same repository

glTF model viewer running on iPad with 2048x1536 resolution

glTF model viewer running on Samsung Galaxy S10 with Vulkan with 1995x945 resolution

glTF model viewer running on Ubuntu AMD RX 480 with Vulkan with 1920x1080 resolution

• Basis Universal Texture Support: TF now supports Binomials @richgel999 @google Basis Universal Texture Support as an option to load textures. Support was added to the Image class as a "new image format". So you can pick basis like you can pick DDS or KTX.

• Shader Translator: since more than a year we are developing on and off a shader translator that allows us to define our own shader language. This shader language is an extension to HLSL and will in the future offer the opportunity to store more data for the various platforms. For example we will able to pre-compile pipelines with this setup. We are using it extensively to translate all the shaders you see in The Forge. You can find the source code in

  Common_3/ThirdParty/OpenSource/hlslparser

  The public version translates at the moment to HLSL, GLSL and Metal. Metal support is still work in progress. There is a test scenario where the shader translator translates most of the shaders of the unit tests and examples to HLSL and GLSL. It will be integrated into our Jenkins test system and it gets tested with every code change.

• Input system: we re-architected the app level interface for our cross-platform input system based on gainput. Our previous interface somehow leaned toward PC input systems and we wanted to better focus on our main target platforms.

• Ozz animation system: we cleaned up the code base. There was a lot of unused code because we wired up the system with all the "service providers" of The Forge. So Ozz like any other third party library is using the math, log, error, assert, file system, memory management and others sub-systems from The Forge. Previously, it had a lot of those implemented on its own.

• Issues resolved:

  • 128 "The DepthStateDesc.mDepthTest is ignored on the Metal backend"
  • 130 "Vulkan and D3D12 backend set sampler maxLod to zero for nearest mipmap mode"
  • 131 "Vulkan sampler set the anistropy flag to false always."

Release 1.32 - August 9th - Ephemeris Night Sky update | New Light and Shadow Playground | Android Visual Studio Support

• Ephemeris - Skydome System
  • Added Procedural Night Sky and Star-field - the night sky's nebula colors are now customizable
  • Added Glow effects on Sun and Moon
  • Fix the issue that Godrays can be drawn on unexpected area

Click on the following screenshot to see a movie:

• New Light and Shadow Playground: we rebuild the light and shadow playground by using the St. Miguel art assets and adding some new shadowing techniques:
  • Exponential Shadow Map - this is based on Marco Salvi's @marcosalvi papers. This technique filters out the edge of the shadow map by approximating the shadow test using exponential function that involves three subjects: the depth value rendered by the light source, the actual depth value that is being tested against, and the constant value defined by the user to control the softness of the shadow
  • Adaptive Shadow Map with Parallax Correction Cache - this is based on the article "Parallax-Corrected Cached Shadow Maps" by Pavlo Turchyn in GPU Zen 2. It adaptively chooses which light source view to be used when rendering a shadow map based on a hiearchical grid structure. The grid structure is constantly updated depending on the user's point of view and it uses caching system that only renders uncovered part of the scene. The algorithm greatly reduce shadow aliasing that is normally found in traditional shadow map due to insufficient resolution. Pavlo Turchyn's paper from GPU Pro 2 added an additional improvement by implementing multi resolution filtering, a technique that approximates larger size PCF kernel using multiple mipmaps to achieve cheap soft shadow. He also describes how he integrated a Parallax Correction Cache to Adaptive Shadow Map, an algorithm that approximates moving sun's shadow on static scene without rendering tiles of shadow map every frame. The algorithm is generally used in an open world game to approximate the simulation of day & night’s shadow cycle more realistically without too much CPU/GPU cost.
  • Signed Distance Field Soft Shadow - this is based on Daniel Wright's Siggraph 2015 @EpicShaders presentation. To achieve real time SDF shadow, we store the distance to the nearest surface for every unique Meshes to a 3D volume texture atlas. The Mesh SDF is generated offline using triangle ray tracing, and half precision float 3D volume texture atlas is accurate enough to represent 3D meshes with SDF. The current implementation only supports rigid meshes and uniform transformations (non-uniform scale is not supported). An approximate cone intersection can be achieved by measuring the closest distance of a passed ray to an occluder which gives us a cheap soft shadow when using SDF.

To achieve high-performance, the playground runs on our signature rendering architecture called Triangle Visibility Buffer. The step that generates the SDF data also uses this architecture.

Click on the following screenshot to see a movie:

The following PC screenshots are taken on Windows 10 with a AMD RX550 GPU (driver 19.7.1) with a resolution of 1920x1080.

Exponential Shadow Maps:

Adaptive Shadow Map with Parallax Correction Cache

Signed Distance Field Soft Shadow:

Signed Distance Field Soft Shadows - Debug Visualization

The following shots show Signed Distance Field Soft Shadows running on iMac with a AMD RADEON Pro 580

The following shots show Signed Distance Field Soft Shadows running on XBOX One:

Readme for Signed Distance Field Soft Shadow Maps:

To generate the SDF Mesh data you should select “Signed Distance Field” as the selected shadow type in the Light and Shadow Playground. There is a button called “Generate Missing SDF” and once its clicked, it shows a progress bar that represents the remaining SDF mesh objects utilized for SDF data generation. This process is multithreaded, so the user can still move around the scene while waiting for the SDF process to be finished. This is a long process and it could consume up to 8+ hours depending on your CPU specs. To check how many SDF objects there are presently in the scene, you can mark the checkbox "Visualize SDF Geometry On The Scene".

• Android: we switched from AndroidStudio to Visual Studio, because the AndroidStudio environment didn't allow us to work effectively and integrating it into our Jenkins test system made the build time explode.
• Vulkan:
  • support of GPU assisted validation
  • new Vulkan SDK 1.1.114 is supported
• Art package: we updated the St. Miguel scene with colored flags for the future Aura Global Illumination example. Because it is used by several unit tests, you want to download the art package again. See the Install instructions below for how to do this.

P.S: we have a new logo :-)

Release 1.31 - July 12th - Metal 2.2 | More Android Support | Discord Channel | User Group Meetings | Support for Education

• macOS / iOS - we are now supporting Metal 2.2 on those platforms. The macOS version of the Visibility Buffer now uses primitive_id argument that allows to use indexed geometry similar to the Vulkan and DirectX 12 versions. There is a significant increase in performance and reduction in memory consumption
  • Debug labels for buffers and textures now present in frame captures;
  • cmdSynchronizeResources for MacOS and iOS;
  • Minor fixes in GPU synchronization with memory barriers
  • Minor fixes in ArgumentBuffers implementation

Please note that we use the early beta system and XCode versions for development. So there might some instabilities.

Here is a screenshot: Macbook Pro 2017 with Radeon Pro 560 3360x2100 resolution

• Android - we increased the number of unit tests support. With this release we additionally support on the devices mentioned below:

  • 06_MaterialPlayground
  • 18_Playback
  • 19_Blending
  • 20_JoinAttachment
  • 21_PartialBlending
  • 22_AdditiveBlending
  • 23_BakedPhysics
  • 24_MultiThread
  • 25_Skinning
  • 26_Audio

• Vulkan:

  • Updated volk Metaloader for Vulkan to latest
  • The Forge supports now as the min spec for the Vulkan SDK 1.1.82.0 and as the max spec is 1.1.101.0

• Discord: we offer now also support through a discord channel. Sign up here:
  Join the Discord channel at https://discord.gg/hJS54bz

• User Group Meetings - there will be a user group meeting during GDC. In case you want to organize a user group meeting in your country / town at any other point in time, we would like to support this. We could send an engineer for a talk.

• Support for Education - in case your School / College / University uses The Forge for education, we would like to support this as well. We could send an engineer or help create material. So far the following schools use The Forge for teaching:

Breda University of Applied Sciences

        Contact:
        Jeremiah van Oosten 
        Monseigneur Hopmansstraat 1
        4817 JT Breda

Ontario Tech University

        Contact:
        Andrew Hogue
        Ontario Tech University
        SIRC 4th floor
        2000 Simcoe St N
        Oshawa, ON, L1H 7K4

• Writing Guidelines - For contributions to The Forge we apply the following writing guidelines:
  • We limit now all code to C++ 11 by setting the Clang and other compiler flags
  • We follow the [Orthodox C++ guidelines] (https://gist.github.com/bkaradzic/2e39896bc7d8c34e042b) minus C++ 14 support (see above)

Release 1.30 - June 28th, 2019 - Ephemeris 2 - New Skydome System | Android Unit Tests | New Entity Component System | SoLoud Audio

• Ephemeris 2: this is a new volumetric skydome system developed for PS4 / XBOX One class of hardware. Click on the image to watch a video:

For Ephemeris and the rest of our commercial custom middleware there is now a new GitHub repository here Custom-Middleware

We also have a skydome system for mobile hardware called Ephemeris 1, that will be released on GitHub later.

• Android: we are supporting now more and more unit tests in Android by improving the run-time support. Here are screenshots:

01_Transformations

02_Compute

05_FontRendering

09_LightAndShadow

13_imGUI

17_EntityComponentSystem

We added the Samsung S10 Galaxy phone (Qualcomm Adreno 640 Graphics Card (Vulkan 1.1.87)) to the test devices for Android.

• ENTT: we decided to remove ENTT and replace it with our own ECS system that we use internally for tools. ENTT in debug is too slow for practical usage because it decreases execution speed and increases compile times substantially. It appears that "modern C++ 17" and probably also "modern C++ 14" is not ready for usage in a team environment because it decreases productivity too much. We tried to remove C++ 17 and 14 features to make it run faster but it ended up too much work. We went from more than 200 ms with ENTT to 60 ms with our own ECS running a Debug build on a Intel Core i7-6700T 2.8GHz. In release our own system is in the moment not as fast as ENTT but we will fix that.

• Audio: we did a first pass on integrating SoLoud for all our platforms. There is a new unit test:

• Linux: following STEAM, we are switching to the Mesa RADV driver in our test environment for AMD GPUs. For NVIDIA GPUs we are still using the NVIDIA driver.

• Texture Asset pipeline: we did a first pass on a unified texture asset pipeline. On the app level only the name of the texture needs to be provided and then depending on the underlying platform it will attempt to load the "optimal compressed" texture, which in the moment is either KTX or dds. In the future there will be Google Basis support as well.

  • Removed support for various non-optimal texture file formats - png, jpg, tga, hdr, exr
  • Add ASTC support for iOS through KTX container
  • Add compressed textures for all unit test resources
  • Add BC6H signed and unsigned float variants

Please make sure you download the art asset zip file again with the help of the batch file.

• Issue list:
  • issue #109 "Texture updates broken" is fixed now
  • NVIDIA GTX 1660 bug: this card with the Vulkan run-time and driver 419.35 became unresponsive, while the DirectX 12 run-time works as expected. Any other NVIDIA GPU works fine ... this looks like a driver bug ...

Release 1.29 - June 6th, 2019 - EASTL Integration | Micro-profiler improvements | Neon intrinsic Support

• We replaced for all platforms TinySTL with EASTL for support of additional data structures and functionality. This was a major change and we still expect a few bugs to appear.
• ARM based platforms (iOS/Android) can pick a new NEON intrinsics code path in our math library
• Microprofiler
  • Multithreaded GPU Profiling is now supported
  • Microprofiler is now enabled on all the Unit-tests and togglable using UI checkbox.
  • Added new common interface, IProfiler.h
• Issue fixed:
  • #105 - 04_ExecuteIndirect crash on macOS