Pull in utf8 support

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## [0.4.0] - 2024-03-03
+
+### Added
+
+- Add UTF8 support (Unicode v14.0)
+
+### Fixed
+
+- Fix bug in multiline cursor offset position with escape codes
+
 ## [0.3.0] - 2024-02-10
 
 ### Added

Makefile

@@ -7,11 +7,14 @@ default: extension lint test
 example: extension
 	crystal run example/example.cr
 
-extension: src/lib/linenoise.o
+extension: src/lib/linenoise.o src/lib/utf8.o
 
 src/lib/linenoise.o: ext/linenoise.c ext/linenoise.h
 	$(CC) -o $@ $< $(CFLAGS)
 
+src/lib/utf8.o: ext/utf8.c ext/utf8.h
+	$(CC) -o $@ $< $(CFLAGS)
+
 check: extension
 	crystal build --no-codegen -o example_program example/example.cr
 
@@ -35,3 +38,4 @@ test: specs expect
 
 clean:
 	rm -f src/lib/linenoise.o
+	rm -f src/lib/utf8.o

README.md

@@ -2,7 +2,9 @@
 
 Crystal bindings for the lightweight [Linenoise](https://github.com/antirez/linenoise) line editor library written in C. It is a minimal alternative to readline and libedit.
 
-Linenoise is written in C code that supports most distributions of the Linux, macOS and BSD operating systems. We compile the library on install so linking should not be a problem and the library is lightwieght (less than 900 lines of code) so the resulting binary should be small.
+Linenoise is written in C code that supports most distributions of the Linux, macOS and BSD operating systems. We compile the library on install so linking should not be a problem and the library is lightwieght (less than 1500 lines of code) so the resulting binary should be small.
+
+As of `v0.4.0`, UTF-8 support has been added. This means that the cursor won't have problems when writing in languages with non-Roman alphabets or using emoji.
 
 ## Installation
 
@@ -50,9 +52,6 @@ end
 For more information look at the files in the `example/` directory and the [documentation website](https://apainintheneck.github.io/crystal-linenoise/).
 
 ## Missing Features & Known Bugs
-UTF-8 support is not included but there have been multiple attempts at adding that in the past. It might be worth investigating more if any of those is mature enough to be worth including here as a patch. See https://github.com/apainintheneck/crystal-linenoise/issues/21 for more info.
-
-The multiline mode doesn't currently support escape codes when calculating cursor position so the presence of them will mean that the cursor will end up further to the right than expected. Single line mode support for escape codes was added in https://github.com/apainintheneck/crystal-linenoise/pull/22 in v0.3.0.
 
 There is no high-level wrapper around the multiplexing API but the Crystal bindings have been added for it. See `src/lib/lib_linenoise.cr` for more details.
 
@@ -88,6 +87,12 @@ The `crystal docs` command can be used to build the docs locally or you can visi
 
 - [apainintheneck](https://github.com/apainintheneck) - creator and maintainer
 
+## Acknowledgments
+
+I have pulled in a few PRs from upstream that haven't been merged into the main Linenoise project yet.
+
+Check out the [extension changes](ext/CHANGES.md) file for more information.
+
 ## License
 
 This library is released under the [MIT license](LICENSE).

ext/CHANGES.md

@@ -73,3 +73,26 @@ All credit goes to antirez as mentioned in the license includes in this director
   - `https://github.com/gdawg`
 - Note
   - This just removes an extra include.
+
+---
+
+## Branch: Another implementation for utf8 support.
+
+- Last Copied
+  - 2024-02-26
+- Last Commit
+  - Date
+    - 2024-04-19
+  - SHA
+    - 34fbffeef4489418527e1267e8e5c093b58aa5d4
+- Branch URL
+  - `https://github.com/antirez/linenoise/pull/187`
+- User URL
+  - `https://github.com/yhirose`
+- Note
+  - This is a massive PR which provides UTF8 support and at
+    the same time fixes the cursor position errors in multiline
+    mode. It currently supports Unicode 14.0.
+  - The list of UTF-8 character mappings was created by parsing
+    the UTF-8 character files downloaded from the Unicode
+    Consortium website.