Code split improvement

[Plemeur]

For anyone that cares and want to make a pull request

I was trying to come up with some code splitter of my own using but resulted in using basically the same implementation that you guys have with a little tweak

    def _chunk_node(self, node: Any, text: str, last_end: int = 0, current_chunk: str = "") -> List[str]:
        new_chunks = []
        for child in node.children:
            if child.end_byte - child.start_byte > self.max_chars:
                # Child is too big, recursively chunk the child
                if len(current_chunk) > self.min_chars:
                    new_chunks.append(current_chunk)
                    current_chunk = ""
                new_chunks.extend(self._chunk_node(child, text, last_end, current_chunk))
                current_chunk = new_chunks[-1]
                new_chunks = new_chunks[:-1]
            elif (
                len(current_chunk) + child.end_byte - child.start_byte > self.max_chars
            ):
                # Child would make the current chunk too big, so start a new chunk
                new_chunks.append(current_chunk)
                current_chunk = text[last_end : child.end_byte]
            else:
                current_chunk += text[last_end : child.end_byte]
            last_end = child.end_byte
        if len(current_chunk) > 0:
            new_chunks.append(current_chunk)
        return new_chunks

Basically the first children get information from the parent and it avoid having orphan characters, still some quirks tho

This is the reference file I used if anyone is wondering
https://github.com/bigskysoftware/htmx/blob/master/src/htmx.js

[logan-markewich]

If this is a better/improved implementation, I definitely encourage a PR