Does Rubix ML support Natural Language Processing (NLP)?

[programarivm]

Hi there,

At ChesslaBlab we're experimenting with different ways of rephrasing the robotic-like output of the PHP Chess Tutor into a user-friendly, human-like explanation. So far we've tried out a number of APIs such as ChatGPT, Google Bard as well as NLP Cloud.

Robotic-like text example:

White has a slightly better control of the center. The black pieces are significantly better connected. The white player is really pressuring more squares than its opponent. The white pieces are timidly approaching the other side's king. The pawn on b5 is unprotected. The pawn on f6 is doubled. The pawn on h7 is isolated. The pawns on a6, d6 and f6 are backward. d5, f5 and h3 are outpost squares. The knight on d5 is nicely placed on an outpost. Black has the bishop pair.

Human-like paraphrase:

White exhibits a minor edge when it comes to central control, while Black's pieces display a greater degree of connection. Additionally, White casts an intense pressure over a larger number of squares compared to the opponent. Yet, White's pieces are making somewhat tentative advances towards the enemy's King. The b5 pawn is left undefended. The f6 pawn is in double trouble, whereas the h7 pawn stands isolated. Pawns on a6, d6, and f6 are lagging behind. The squares at d5, f5, and h3 serve as outposts while the d5 knight stands poised at a strategic outpost. Finally, Black possesses the advantage of a bishop pair.

While the result obtained is okay I was wondering if Rubix ML could be used to train our own paraphrase generation model.

Any help will be very much appreciated!

Thank you,

[andrewdalpino]

Hey @programarivm unfortunately Rubix ML does not make a good base library for a language generation project in my opinion. The reason is because language generation models, like the autoregressive type that predict the next word) tend to require a ton of computation especially at training but inference as well. Single-threaded CPU with additional PHP overhead would take forever to train. Additionally, we do not provide support for Transformers which is the current state-of-the-art neural network architecture at tackling these language generation problems.

I would take a look at fine-tuning a pretrained GPT with your chess-oriented corpus using PyTorch for your problem.

Where Rubix ML may be able to help you is in predicting intent from text. With that, you could respond with pregenerated text appropriately. This may or may not make your answers sound less robotic. It may even make it worse.

Hope all is well with you! Keep up the great work.