Multifragment rendering (MFR) is a genre of image synthesis techniques and associated data structures tightly coupled with the rasterisation pipeline, which has helped deliver important improvements to the visual quality of primitive-order rendering and has enabled the real-time display of complex phenomena and structures. An MFR method encompasses the algorithms and image-space data structures that are necessary to produce, maintain, process and exploit a set of geometry fragments that are associated with a single image, in the sense that multiple samples correspond to the same location in image space.
The main advantage of these approaches is that they encompass additional rasterised geometry, by retaining more information from the fragment sampling domain, thus augmenting the visibility determination stage. For more details, please refer to the recent state-of-the-art report which was presented at the Eurographics 2020 conference.
Figure 1. Multifragment rendering has been deployed in a wide spectrum of rendering applications in order to generate compelling graphics effects at interactive frame rates.
The goals of this github repository are to:
- Present formally the MFR pipeline.
- Offer a centralised location that contains source code and examples of the MFR-related research that has been done by the CG AUEB Group.
- Provide a generalised MFR framework for prototyping constructions methodologies and their applications, ranging from order-independent transparency to global illumination and data visualisation.
At a conceptual level, the complex primary visibility determination is part of a more general multifragment pipeline responsible for generating, storing, processing and evaluating information from incoming fragments (Fig. 2, top). This pipeline is comprised of three main steps: construction, operation and image composition, which are defined by the context of the particular application in mind. The construction step is responsible for generating and storing per-pixel fragments through a common rasterisation procedure. Even though storing and sorting are two fundamental building blocks of the construction step, the latter stage is implicitly or explicitly performed. After construction has taken place, the next step performs one or more operations on the ordered fragment data structure. Finally, the application accesses and exploits the fragment information stored in the MFR structure to compose the final output (Fig. 2, bottom).
Figure 2. Diagram of building an application using the MFR pipeline (top). For example, Order-independent transparency requires the sorting of an arbitrary sequence of out-of-order fragments before alpha compositing them in a linear traversal fashion (bottom).
A comprehensive shader source code bundle for efficiently solving the visibility determination problem in screen space is provided. This extensive collection includes the most widely-used multi-fragment rendering solutions such as the depth peeling variants as well as k-buffer and A-buffer alternatives (summarized in our recent survey [VVP20]). The source code is mainly written using the OpenGL 4.4 API, except from the parts that do not require GPU-accelerated atomic memory operations (OpenGL 3.3).
Table 1. List of multifragment data structures
| Name/Location | Methodology | Citation |
|---|---|---|
| Depth Peeling | ||
| F2B | Front-to-back (Forward) | [Eve01] |
| F2B_D | Front-to-back (Deferred) | [VF13] |
| F2B_ZF | Front-to-back (Z-fighting) | [VF13] |
| DUAL | Dual | [BM08b] |
| BUN | Uniform Buckets | [LHLW09] |
| A-buffer | ||
| AB_LL | Linked-Lists | [YHG*10] |
| AB_LL_D | Linked-Lists (Double) | [VVP16a] |
| AB_LL_BUN | Linked-Lists (Uniform Bins) | [VF13] |
| AB_AF | Fixed Arrays | [Cra10a] |
| AB_AV | Variable Arrays | [VF12] |
| k-buffer | ||
| KB | Fixed Arrays | [BCL*07] |
| KB_SR | Fixed Arrays (Stencil-routed) | [BM08a] |
| KB_MULTI | Fixed Arrays (Multipass) | [LWXW*09] |
| KB_AB_LL | Fixed Arrays (A-buffer, Linked Lists) | [SML11] |
| KB_LL | Linked Lists | [YYH∗12] |
| KB_PS | Fixed Arrays (Pixel Synchronized) | [Sal13] |
| KB_MDT_32 | Fixed Arrays (Multidepth Testing 32bit) | [MCTB13] |
| KB_MDT_64 | Fixed Arrays (Multidepth Testing 64bit) | [Kub14] |
| KB_MAX_ARRAY | Fixed Arrays (Max Array) | [VF14] |
| KB_MAX_HEAP | Fixed Arrays (Max Heap) | [VF14] |
The tables below link to shader implementations of various A-buffer data structures tailored to image-space ray tracing. The additional characteristics of these implementations:
- Exploiting larger shading blocks for storing shading data (albedo, normals, etc.)
- Using linear-Z for storing and sorting the fragments.
- Visibility and shading data are decoupled, i.e. stored in separate buffers, to improve sorting and traversal.
- Support for multiple views. Here, it is provided only for the optimal data structures, with respect to performance.
Table 2. List of single-view multifragment data structures for ray tracing
| Name/Location | Type | Characteristics | Original Method | Implemented in |
|---|---|---|---|---|
| AB_LL | Linked-Lists | Yes | [YHG*10] | [VVP16a] |
| AB_LL_BUN | Linked-Lists (Uniform Bins) | Yes | [VF13] | [VVP16a] |
| AB_LLD | Linked-Lists (Double) | Yes | [VVP16a] | [VVP16a] |
| AB_LLD_BUN | Linked-Lists (Double - Uniform Bins) | Yes | [VVP16a] | [VVP16a] |
| AB_SB | Arrays (Variable) | Yes | [VF12] | [VVP16a] |
Table 3. List of multi-view multifragment data structures for ray tracing
| Intersections on | Data Structure | Application | Citation |
|---|---|---|---|
| Fragments | Linked-Lists (Double - Uniform Bins - Decoupled) | Path Tracing - Ambient Occlusion | [VVP16a] |
| Primitives | Linked-Lists (Uniform Bins - Decoupled) | Path Tracing | [VVP16b] |
- A simple forward rendering engine that can be used as a research prototyping platform for testing various MFR algorithms for shading and illumination effects using modern OpenGL. (TBD)
- Screen-space Ray Tracing demos from VVP16a and VVP16b papers can be found here.
- Evangelou I., Papaioannou G., Vardis K., Vasilakis A. A., 'Rasterization-based Progressive Photon Mapping', The Visual Computer (Computer Graphics International 2020).
- Vasilakis A. A., Vardis K., Papaioannou G., 'A Survey of Multifragment Rendering', Computer Graphics Forum (Eurographics 2020 - STAR Papers).
- Vasilakis A. A., Vardis K., Papaioannou G. and Moustakas K.,'Variable k-buffer using Importance Maps', Eurographics 2017 - Short Papers.
- Vardis K., Vasilakis A. A., Papaioannou G., 'DIRT: Deferred Image-based Ray Tracing', High-Performance Graphics 2016.
- Vardis K., Vasilakis A. A., Papaioannou G., 'A Multiview and Multilayer Approach for Interactive Ray Tracing', Interactive 3D Graphics and Games 2016.
- Vasilakis A. A., Papaioannou G., Fudos I. 'k+-buffer: An efficient, memory-friendly and dynamic k-buffer framework', TVCG, 2015.
- Vasilakis A. A., Papaioannou G., 'Improving k-buffer methods via Occupancy Maps', Eurographics 2015 - Short Papers.
The license is MIT. If you use the contents of this repository for your work, please cite it as described below:
In our work, we have used the shader source code~\cite{VVP_EG_2020_STAR}, available at 'https://github.com/cgaueb/MFR' repository, that implements the algorithm described in the research paper~\cite{XXX}.@article{VVP_EG_2020_STAR, title = {A Survey of Multifragment Rendering}, author = {Vasilakis, Andreas Alexandros and Vardis, Konstantinos and Papaioannou, Georgios}, journal = {Computer Graphics Forum}, volume = {39}, number = {2}, pages = {623-642}, year = {2020}, issn = {1467-8659}, doi = {10.1111/cgf.14019}, publisher = {The Eurographics Association and John Wiley & Sons Ltd.} }
- [VVP20] Vasilakis et al., "A Survey of Multifragment Rendering", CGF (EG STAR), 2020.
- [Eve01] Everitt, "Interactive Order-Independent Transparency", Tech. rep., Nvidia Corporation, 2001.
- [BM08b] Bavoil and Myers, "Order Independent Transparency with Dual Depth Peeling", Tech. rep., Nvidia Corporation, 2008.
- [LHLW09] Liu et al., "Efficient Depth Peeling via Bucket Sort", HPG, 2009.
- [VF13] Vasilakis and Fudos, "Depth-Fighting Aware Methods for Multifragment Rendering", TVCG, 2013.
- [BCL*07] Bavoil et al., "Multi-fragment Effects on the GPU Using the k-buffer", I3D, 2007.
- [BM08a] Bavoil and Myers, "Deferred Rendering using a Stencil Routed k-Buffer", ShaderX6: Advanced Rendering Techniques, 2008.
- [LWXW*09] Liu et al., "Multi-layer depth peeling via fragment sort", CAD&CG, 2009.
- [SML11] Salvi et al., "Adaptive Transparency", HPG, 2011.
- [YYH∗12] Yu et al., "A Framework for Rendering Complex Scattering Effects on Hair", I3D, 2012.
- [MCTB13] Maule et al., "Hybrid Transparency", I3D, 2013.
- [Sal13] Salvi, "Advances in Real-Time Rendering in Games: Pixel Synchronization: Solving old graphics problems with new data structures", SIGGRAPH Courses, 2013.
- [Kub14] Kubish, "Order Independent Transparency In OpenGL 4.x.", GTC, 2014.
- [VF14] Vasilakis and Fudos, "k+-buffer: Fragment Synchronized k-buffer", I3D, 2014.
- [VP15] Vasilakis and Papaioannou, "Improving k-buffer Methods via Occupancy Maps", EG (Short Papers), 2015.
- [VPF15] Vasilakis et al., "k+-buffer: An efficient, memory-friendly and dynamic k-buffer framework", TVCG, 2015.
- [VVPM17] Vasilakis et al., "Variable k-buffer using Importance Maps", EG (Short Papers), 2017.
- [YHG*10] Yang et al., "Real-time concurrent linked list construction on the GPU", CGF (EGSR'10), 2010.
- [Cra10a] Crassin, "Fast and accurate single-pass A-buffer", Blog post, 2010.
- [Cra10b] Crassin, "Linked lists of fragment pages", Blog post, 2010.
- [VF12] Vasilakis and Fudos, "S-buffer: Sparsity-aware multifragment rendering", EG (Short Papers), 2012.
- [VVP16a] Vardis et al., "A Multiview and Multilayer Approach for Interactive Ray Tracing", I3D, 2016.
- [VVP16b] Vardis et al., "DIRT: Deferred Image-based Ray Tracing", HPG, 2016.