RAII - V2

docs/raii.md

@@ -0,0 +1,177 @@
+# Deterministic Automated Resource Management
+
+Resource management in programming languages generally falls into one of the following categories:
+
+1. **Manual allocation and deallocation**
+2. **Automatic garbage collection**
+3. **Automatic scope-bound resource management** (commonly referred to as [RAII](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization), or *Resource Acquisition Is Initialization*).
+
+Traditionally, Terra has only supported manual, C-style resource management. While functional, this approach limits the full potential of Terra’s powerful metaprogramming capabilities. To address this limitation, the current implementation introduces **automated resource management**.
+
+---
+
+## Scope-Bound Resource Management (RAII)
+
+The new implementation provides **scope-bound resource management (RAII)**, a method typically associated with systems programming languages like C++ and Rust. With RAII, a resource's lifecycle is tied to the stack object that manages it. When the object goes out of scope and is not explicitly returned, the associated resource is automatically destructed.
+
+### Examples of Resources Managed via RAII:
+- Allocated heap memory
+- Threads of execution
+- Open sockets
+- Open files
+- Locked mutexes
+- Disk space
+- Database connections
+
+---
+
+## Experimental Implementation Overview
+
+The current Terra implementation supports the **Big Three** (as described by the [Rule of Three](https://en.wikipedia.org/wiki/Rule_of_three_(C%2B%2B_programming)) in C++):
+
+1. **Object destruction**
+2. **Copy assignment**
+3. **Copy construction**
+
+However, **rvalue references** (introduced in C++11) are not supported in Terra. As a result, the current RAII implementation is comparable to that of **C++03** or **Rust**.
+
+### Key Features:
+Compiler support is provided for the following methods:
+```terra
+A.methods.__init(self : &A)
+A.methods.__dtor(self : &A)
+(A or B).methods.__copy(from : &A, to : &B)
+```
+These methods facilitate the implementation of smart containers and pointers, such as `std::string`, `std::vector` and `std::unique_ptr`, `std::shared_ptr`, `boost:offset_ptr` in C++.
+
+### Design Overview
+* No Breaking Changes: This implementation does not introduce breaking changes to existing Terra code. No new keywords are required, ensuring that existing syntax remains compatible.
+* Type Checking Integration: These methods are introduced during the type-checking phase (handled in terralib.lua). They can be implemented as either macros or Terra functions.
+* Composability: The implementation follows simple, consistent rules that ensure smooth compatibility with existing Terra syntax for construction, casting, and function returns.
+* Heap resources: Heap resources are allocated and deallocated using standard C functions like malloc and free, leaving memory allocation in the hands of the programmer. The idea here is that remaining functionality, such as allocators, are implemented as libraries.
+
+---
+
+## Safety and Future Work
+
+While safety is a growing concern in programming, the current implementation has several safety challenges, similar to those in C++ or Rust's unsafe mode.
+
+### Future Work Includes:
+1. **Library support for composable allocators**:
+   - Tracing or debugging allocators to detect memory leaks or other faulty behavior.
+2. **Compile-time borrow checking**:
+   - Similar to Rust or Mojo, ensuring safer resource usage.
+3. **Improved lifetime analysis**:
+   - Making the compiler aware of when resources (e.g., heap allocations) are introduced.
+   - Making the compiler aware of when resources are last used.
+
+---
+
+## Compiler supported methods for RAII
+A managed type is one that implements at least `__dtor` and optionally `__init` and `__copy` or, by induction, has fields or subfields that are of a managed type. In the following we assume `struct A` is a managed type.
+
+To enable RAII, import the library */lib/terralibext.t* using
+```terra
+require "terralibext"
+```
+The compiler only checks for `__init`, `__dtor` and `__copy` in case this library is loaded.
+
+### Object initialization
+`__init` is used to initialize managed variables:
+```
+A.methods.__init(self : &A)
+```
+The compiler checks for an `__init` method in any variable definition statement, without explicit initializer, and emits the call right after the variable definition, e.g.
+```
+var a : A
+a:__init()    --generated by compiler
+```
+### Copy assignment
+`__copy` enables specialized copy-assignment and, combined with `__init`, copy construction. `__copy` takes two arguments, which can be different, as long as one of them is a managed type, e.g.
+```
+A.metamethods.__copy(from : &A, to : &B)
+```
+and / or
+```
+A.metamethods.__copy(from : &B, to : &A)
+```
+If `a : A` is a managed type, then the compiler will replace a regular assignment by a call to the implemented `__copy` method
+```
+b = a   ---->   A.methods.__copy(a, b) 
+```
+or 
+```
+a = b   ---->   A.methods.__copy(b, a)
+```
+`__copy` can be a (overloaded) terra function or a macro.
+
+The programmer is responsible for managing any heap resources associated with the arguments of the `__copy` method.
+
+### Copy construction
+In object construction, `__copy` is combined with `__init` to perform copy construction. For example,
+```
+var b : B = a
+```
+is replaced by the following statements
+```
+var b : B
+b:__init()              --generated by compiler if `__init` is implemented
+A.methods.__copy(a, b)  --generated by compiler
+```
+If the right `__copy` method is not implemented but a user defined `__cast` metamethod exists that can cast one of the arguments to the correct type, then the cast is performed and then the relevant copy method is applied.
+
+### Object destruction
+`__dtor` can be used to free heap memory
+```
+A.methods.__dtor(self : &A)
+```
+The implementation adds a deferred call to `__dtor ` near the end of a scope, right before a potential return statement, for all variables local to the current scope that are not returned. Hence, `__dtor` is tied to the lifetime of the object. For example, for a block of code the compiler would generate
+```
+do
+    var x : A, y : A
+    ...
+    ...
+    defer x:__dtor()    --generated by compiler
+    defer y:__dtor()    --generated by compiler
+end
+```
+or in case of a terra function
+```
+terra foo(x : A)
+    var y : A, z : A
+    ...
+    ...
+    defer z:__dtor()    --generated by compiler
+    return y
+end
+```
+`__dtor` is also called before any regular assignment (if a __copy method is not implemented)  to free 'old' resources. So
+```
+a = b
+```
+is replaced by
+```
+a:__dtor()  --generated by compiler
+a = b
+```
+## Compositional API's
+If a struct has fields or subfields that are managed types, but do not implement `__init`, `__copy` or `__dtor`, then the compiler will generate default methods that inductively call existing `__init`, `__copy` or `__dtor` methods for its fields and subfields. This enables compositional API's like `vector(vector(int))` or  `vector(string)`. This is implemented as an extension to *terralib.lua* in *lib/terralibext.t*.
+
+## Examples
+The following files have been added to the terra testsuite:
+* *raii.t* tests whether `__dtor`, `__init`, and `__copy` are evaluated correctly for simple datatypes.
+* *raii-copyctr.t* tests `__copy`.
+* *raii-copyctr-cast.t* tests the combination of `metamethods.__cast` and `__copy`.
+* *raii-unique_ptr.t* tests some functionality of a unique pointer type.
+* *raii-shared_ptr.t* tests some functionality of a shared pointer type.
+* *raii-offset_ptr.t* tests some functionality of an offset pointer implementation, found e.g. in *boost.cpp*.
+* *raii-compose.t* tests the compositional aspect.
+
+You can have a look there for some common code patterns.
+
+## Current limitations
+* The implementation is not aware of when an actual heap allocation is made and therefore assumes that a managed variable always carries a heap resource. It is up to the programmer to properly initialize pointer variables to nil to avoid calling 'free' on uninitialized pointers.
+* Tuple (copy) assignment (regular or using `__copy`) are prohibited by the compiler in case of managed variables. This is done to prevent memory leaks or unwanted deletions in assignments such as
+    ```
+    a, b = b, a
+    ```