Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PS/2 Mouse / Trackpoint / Trackpad / Trackball support #1751

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

infused-kim
Copy link
Contributor

@infused-kim infused-kim commented Apr 12, 2023

Hey guys, this PR adds support for PS2 Mouse Devices.

Kim's TP

Note:

This PR has recently been updated with a new driver that uses the UART chip to process the PS/2 signals, which leads to MUCH better performance.

If you tried the driver before and had issue, please try the new version.

The instructions below have been updated for the new version.

1. How to test

I have recently converted this PR into a standalone zmk module that you can use to add PS/2 mouse & trackpoint support to your keyboard.

You can find it here:
https://github.com/infused-kim/kb_zmk_ps2_mouse_trackpoint_driver

The code here won't be updated anymore until we get closer to the stage where it may get merged into zmk main.

2. Features

In addition to supporting all PS/2 pointing devices, the driver has a few noteworthy features...

2.1. Great performance through the use of the UART chip

PS/2 is a very old protocol and unfortunately not used in many devices anymore (apart from TrackPoints). Therefore most microcontrollers don't include hardware chips that handle the communication with PS/2 devices (unlike, for example, SPI).

TrackPoints send data at a rate of almost 15,000 times per second and unfortunately, the most popular controllers in the zmk community are not able to keep up with the communication, at this speed, when you process it using GPIO interrupts.

However, I found a way to take advantage of the UART hardware chip to process the data. This ensures great performance even without proper PS/2 hardware support on the MCU.

If you are interested in how this works, you can read more about it in the development journey section.

2.2. Automatic Layer Toggling on mouse movement

You can also automatically toggle a layer whenever there is mouse movement. This way you can turn your thumb cluster keys into mouse buttons whenever you touch the TrackPoint.

2.3. TrackPoint related features

2.3.1. TrackPoint Configuration with key behaviors

TrackPoints have various settings that can change the algorithm for the mouse movement, but recompiling and reflashing your keyboard to test them is very tedious and makes comparing settings very hard.

So, I added the ability to adjust these settings at run-time through key behaviors, such as &mms MS_TP_SENSITIVITY_INCR. This way you can easily find the right settings for your keyboard and preference.

You can adjust the following settings:

  • Sensitivity
  • Negative Inertia
  • Transfer Function Upper Plateau Speed (value6)
  • The pressure sensitivity for press-to-select

Read the TrackPoint System Version 4.0 Engineering Specification to better understand how these settings affect the mouse behavior.

And check out the instructions in the example zmk-config to see how to use the settings key layer.

2.3.2. Power-On-Reset Support

Many TrackPoints require an additional reset pin. Most people in the ergo keyboard community create a reset circuit that uses resistors and capacitors to implement the functionality.

This allows you to save a pin and use the TrackPoint with a firmware that doesn’t support power-on-reset.

But if you have a pin to spare, you can connect the TrackPoint directly and configure this driver to perform the power-on-reset.

2.3.3. Press-To-Select (Clicking by pressing the TrackPoint)

TrackPoints have an interesting feature that allows you to click by pressing a little harder on the TrackPoint and this driver lets you enable it.

But, unfortunately, not all TrackPoints support this feature. My older Lenovo T430 TP supports it, but my newer, red T460S TP does not.

3. Acknowledgements

I also want to take a quick moment to thank some people.

Most of all to krikun98. Without your work this would not have been possible.

And to all the guys who have tirelessly helped me and answered my questions on discord. Most of all xudongz, joelspadin and of course petejohanson.

I look forward to your testing and feedback.

@krikun98
Copy link
Contributor

I really have to get #778 up to snuff, don't I. Great work here! I was planning to get to it eventually, but life got in the way and you beat me to it.

@infused-kim
Copy link
Contributor Author

infused-kim commented Apr 14, 2023

I really have to get #778 up to snuff, don't I. Great work here! I was planning to get to it eventually, but life got in the way and you beat me to it.

What do you think still needs to be done for #778? It looks pretty complete to me. Urob maintains a rebase of it on his fork and there are several people who are actively using it.

Maybe all of us can work on it together to get it to the finish line and merged.

@tinyboxxx
Copy link

Confirming it’s working on ZMK, using nrfmicro (nrf52840) and modified sofle.
83B4FE91-69CD-49B1-98E3-D388F34F8C30
3DB3D734-2134-41B5-90C8-09E1F62EB522

@krikun98
Copy link
Contributor

I really have to get #778 up to snuff, don't I. Great work here! I was planning to get to it eventually, but life got in the way and you beat me to it.

What do you think still needs to be done for #778? It looks pretty complete to me. Urob maintains a rebase of it on his fork and there are several people who are actively using it.

Maybe all of us can work on it together to get it to the finish line and merged.

The parts relevant to this PR may look more or less complete, but mouse movement emulation needs a rewrite. I haven't looked at it in... a while, but from what I remember - it's using Zephyr's work queues to emulate ticks for the movement. They can get de-prioritized fairly easily, resulting in choppy movement over Bluetooth. I was planning to split up the PR and rewrite that part to utilize bare threads instead.

@infused-kim
Copy link
Contributor Author

The parts relevant to this PR may look more or less complete, but mouse movement emulation needs a rewrite. I haven't looked at it in... a while, but from what I remember - it's using Zephyr's work queues to emulate ticks for the movement. They can get de-prioritized fairly easily, resulting in choppy movement over Bluetooth. I was planning to split up the PR and rewrite that part to utilize bare threads instead.

That makes sense and doesn’t sound like too much work. We should probably also check with Pete and the other maintainers to see what other requirements they might have.

I also briefly looked at integrating my mouse code with your mouse acceleration, but I didn’t want to intertwine the code too much.

So perhaps we could refactor that part a little too.

And maybe you could give me some guidance on how to send mouse events through the event system to the central half so that we can put the trackpoint on the peripheral side.

@krikun98
Copy link
Contributor

The parts relevant to this PR may look more or less complete, but mouse movement emulation needs a rewrite. I haven't looked at it in... a while, but from what I remember - it's using Zephyr's work queues to emulate ticks for the movement. They can get de-prioritized fairly easily, resulting in choppy movement over Bluetooth. I was planning to split up the PR and rewrite that part to utilize bare threads instead.

That makes sense and doesn’t sound like too much work. We should probably also check with Pete and the other maintainers to see what other requirements they might have.

I also briefly looked at integrating my mouse code with your mouse acceleration, but I didn’t want to intertwine the code too much.

So perhaps we could refactor that part a little too.

And maybe you could give me some guidance on how to send mouse events through the event system to the central half so that we can put the trackpoint on the peripheral side.

I knew the acceleration wasn't on in that demo video! I also wanted to make that part more extensible and easily configurable, maybe with on-the-fly profile tuning as in your trackpoint code.
With mouse movement emulation having mouse movement keys on the peripheral is trivial - those were just new behaviours that plugged in to the matrix code. Here, it's a bit more difficult - I would imagine this would require somehow incorporating the trackpoint events into the central-peripheral communication. I never really investigated that part much, Pete or the people experimenting with wired split would surely know more.

@xudongzheng
Copy link
Contributor

And maybe you could give me some guidance on how to send mouse events through the event system to the central half so that we can put the trackpoint on the peripheral side.

It's worth considering sending mouse data the same way as the encoder stuff in #728.

@infused-kim
Copy link
Contributor Author

It's worth considering sending mouse data the same way as the encoder stuff in #728.

Yeah that would be a good starting point. Although, I think one of the reasons the split side encoder stuff hasn’t been merged is that Pete didn’t think the communication was implemented the right way.

I think he is working right now on refactoring it. So perhaps it’s best to wait for those changes.

@GregoryBai
Copy link

Wow, looks cool! Any updates? I will try to test it myself. I've got Cirque & Azteq touchpads if that helps anyone.

@infused-kim
Copy link
Contributor Author

Wow, looks cool! Any updates? I will try to test it myself. I've got Cirque & Azteq touchpads if that helps anyone.

A couple of people have tried it and it seems to be working well enough for them. I am still in the process of designing my own keyboard with a trackpoint mount.

So I haven't had the opportunity to test it in real life and develop more new features.

I am not familiar with the circque & aztec touchpads, but if they use PS/2 I would love it if you tested them with this PR and gave feedback.

@GregoryBai
Copy link

@infused-kim

As of now, I have tried soldering the pins and didn't get it working (Although, I am not sure whether I soldered the GND pin correctly or not). I will try again with another trackpad.

It'd be great if you could take a look the specs. The official site of it is cirque.com, though.

I've seen some post on Reddit / Discord of people having those working but in wired mode though.

@infused-kim
Copy link
Contributor Author

@infused-kim

As of now, I have tried soldering the pins and didn't get it working (Although, I am not sure whether I soldered the GND pin correctly or not). I will try again with another trackpad.

It'd be great if you could take a look the specs. The official site of it is cirque.com, though.

I've seen some post on Reddit / Discord of people having those working but in wired mode though.

It seems like it uses i2c or spi

@GregoryBai
Copy link

GregoryBai commented May 8, 2023

@infused-kim Yup! I followed the guide and removed the R1 resistor to enable i2c instead of SPI, but failed. Does the PR at its state imply that any I2C trackpad would be working by default?

P.S.

I got it. It wasn't supposed T_T

@infused-kim
Copy link
Contributor Author

@infused-kim Yup! I followed the guide and removed the R1 resistor to enable i2c instead of SPI, but failed. Does the PR at its state imply that any I2C trackpad would be working by default?

P.S.

I got it. It wasn't supposed T_T

Yeah, unfortunately it’s for PS/2 pointing devices only. :)

@GregoryBai
Copy link

@infused-kim Do you think it is going to be a lot of work to add the I2C support? A lot of users hope for that, as you've noticed on reddit :P

@infused-kim
Copy link
Contributor Author

@infused-kim Do you think it is going to be a lot of work to add the I2C support? A lot of users hope for that, as you've noticed on reddit :P

I haven’t looked into it. So I can’t really say with any certainty.

What I can say with certainty though is that I am not interested in trackpads and that I won’t be working on that.

Sorry.

@artggd
Copy link

artggd commented May 9, 2023

Hello everyone. I've been able to mount a T440 trackpoint to my keyboard running on a pair of nice!nano microcontrollers. Here's a picture of the finished build:
B05yTId

I have simply followed the steps in this PR and everything compiled, no issue.

So far, it seems to be useable enough for it to be my main way of moving the cursor 🎉

Following are the issues I've encountered for a day a half working with it:

  • Firmware crashes every time I hold any key on the main side and moves the cursor for 6 seconds. This prevents using drag'n'drop or scrolling with btn3 pressed down. To recover from it, I have to reset the microcontroller. I had to move my mouse keys to the secondary side to avoid crashes, but it's actually more ergonomic that way, so I'm happy with that.
  • I do encounter a few random crashes from time to time. Either it's the whole mcu or just the trackpoint that stops working. This is usually fixed by rebooting the mcu.
  • I noticed that when I click a mouse button (any), the cursor moves slightly. It moves in the opposite direction from the last movement I made with the trackpoint. This prevents me from clicking on a draggable link, since it then moves the content.
  • The trackpoint seems to drain my battery faster, but I have no good proof to show, apart from charging the keyboard overnight and wait for a full discharge. The battery usage doesn't show up any more in my computer (Manjaro Linux, Gnome, running on X11), but this has always been buggy...
  • CONFIG_ZMK_MOUSE_PS2_TP_TAP_TO_SELECT=y doesn't seem to work at all for the T440, but I do sometimes get a random right click event happening. Though, this is very rare and might not be related.

I'd be nice if moving the cursor activated a layer dedicated to mouse clicks.

Again, beside the few issues I had, this is already very useable.

@infused-kim
Copy link
Contributor Author

Hello everyone. I've been able to mount a T440 trackpoint to my keyboard running on a pair of nice!nano microcontrollers. Here's a picture of the finished build: B05yTId

I have simply followed the steps in this PR and everything compiled, no issue.

So far, it seems to be useable enough for it to be my main way of moving the cursor 🎉

Following are the issues I've encountered for a day a half working with it:

  • Firmware crashes every time I hold any key on the main side and moves the cursor for 6 seconds. This prevents using drag'n'drop or scrolling with btn3 pressed down. To recover from it, I have to reset the microcontroller. I had to move my mouse keys to the secondary side to avoid crashes, but it's actually more ergonomic that way, so I'm happy with that.
  • I do encounter a few random crashes from time to time. Either it's the whole mcu or just the trackpoint that stops working. This is usually fixed by rebooting the mcu.
  • I noticed that when I click a mouse button (any), the cursor moves slightly. It moves in the opposite direction from the last movement I made with the trackpoint. This prevents me from clicking on a draggable link, since it then moves the content.
  • The trackpoint seems to drain my battery faster, but I have no good proof to show, apart from charging the keyboard overnight and wait for a full discharge. The battery usage doesn't show up any more in my computer (Manjaro Linux, Gnome, running on X11), but this has always been buggy...
  • CONFIG_ZMK_MOUSE_PS2_TP_TAP_TO_SELECT=y doesn't seem to work at all for the T440, but I do sometimes get a random right click event happening. Though, this is very rare and might not be related.

I'd be nice if moving the cursor activated a layer dedicated to mouse clicks.

Again, beside the few issues I had, this is already very useable.

Thank you for the detailed report. Very helpful.

I will look into all of these issues eventually. Especially the holding a key whole moving the mouse. If I can reproduce it, it should be easy to fix.

Unfortunately, at the moment I am busy with other stuff. So it might be awhile until I get to it.

@artggd
Copy link

artggd commented May 9, 2023

Unfortunately, at the moment I am busy with other stuff. So it might be awhile until I get to it.

My C skills are a bit rusty but I'd be more than happy to contribute! I'll try and see how it goes as soon as I have some free time :)

@infused-kim
Copy link
Contributor Author

Unfortunately, at the moment I am busy with other stuff. So it might be awhile until I get to it.

My C skills are a bit rusty but I'd be more than happy to contribute! I'll try and see how it goes as soon as I have some free time :)

Sweet. I would appreciate all the help I can get. Feel free to join #pointing-devices on the zmk discord and I’ll be happy to give you guidance.

@infused-kim
Copy link
Contributor Author

Hey guys,

a few updates...

1. Rebase to latest zmk main

I have rebased this PR and the pr-testing branch onto the latest zmk main.

So, now you can enjoy your trackpoints as well as the latest zmk improvemements, such as split side encoders, timerless homerow mods, nice!view displays and much more.

2. My TP build

I have also finally finished building my own keyboard with a TP:
IMG_4085

3. TP support improvements...

Since I now have a TP on my keyboard, I have a lot more motivation to work on the code. I am not ready to release it yet, but there is some exciting stuff coming up.

I am currently testing a new PS/2 driver that is using the nrf52's UART chip instead of GPIO interrupt. This should drastically reduce jumps and cutouts of the mouse movement.

And I am testing an auto-layer on mouse movement, which lets you turn your thumb cluster into mouse keys automatically when you are moving the trackpoint.

Both are working fairly well now, but not good enough for release.

If any of you are interested in testing it soon, let me know here or in the zmk discord #pointing-devices channel.

@myst729
Copy link

myst729 commented Oct 10, 2023

Wow~ this looks great. I was a long-time ThinkPad user and always wanted a track point on my custom keyboard. I've done that with QMK but unfortunately it can't work without wire. Very excited to see this. 😆

@liljenstolpe
Copy link

liljenstolpe commented Oct 11, 2023

Hey guys,

a few updates...
[clip]
=

Is this still restricted to the central side only?

@FearlessSpiff
Copy link

I am just playing with a cirque trackpad on i2c with QMK. It works really well! Much better than the pimoroni trackball. Hope this or something in the future will also work with i2c. I miss wireless on QMK. ;)

@myst729
Copy link

myst729 commented Nov 3, 2023

@infused-kim @tinyboxxx Just want to know where do you get the track point module? I have used one from T400 running QMK. The module uses an NXP chip which requires >4.5V power supply. So what types are capable under 3V power? I see @artggd mentioned T440, I will give it a try. I also notice ThinkPad uses a now module type starting from T440. Any other advices? Thank you.

@infused-kim
Copy link
Contributor Author

@infused-kim @tinyboxxx Just want to know where do you get the track point module? I have used one from T400 running QMK. The module uses an NXP chip which requires >4.5V power supply. So what types are capable under 3V power? I see @artggd mentioned T440, I will give it a try. I also notice ThinkPad uses a now module type starting from T440. Any other advices? Thank you.

Most Lenovo trackpoints seem to work just fine on 3v. And I convinced an AliExpress store to sells the trackpoints without a keyboard.

I posted details on discord:
https://discord.com/channels/719497620560543766/845285481888743434/1112605096262508645

caksoylar and others added 12 commits April 6, 2024 21:30
* Add ability to swap X/Y, invert X and Y values, and apply a
  scalar multiplier/divisor.
* Remove now-unused mouse work queue and related mouse main file.
* Move ticks config into a DTS property on the two axis input behavior.
* Corrected logging for two-axis input timestamps.
* Buffer data from input devices and only surface to HID once synd'd.
* Always import mouse keys behavior and their associated listeners.
* Tweak listener code to only add listener nodes when
  listener and the associated input device are enabled.
* Dedicated mouse source directory.
* Split mouse HID into dedicated USB endpoint and HoG service.
* Enable composite USB device automatically, tweak the
  various default sizes.
@infused-kim
Copy link
Contributor Author

Hey guys,

I have some news...

1. I created a zmk module of the PS/2 TrackPoint driver

This means you can get more up-to-date versions of zmk main without having to wait for me to rebase my fork anymore.

Unfortunately, for now, you won't be able to use it with zmk main, because it still depends on a fork with the mouse PR #2027.

But you can use it with pete's mouse PR branch, which he updates frequently with the new changes from zmk main.

You can find it here:
https://github.com/infused-kim/kb_zmk_ps2_mouse_trackpoint_driver

And I'd appreciate it if you could give it a star.

From now on, I will continue all development in the module and won't update the forks anymore. When we get closer to merging this into zmk, I will resume here again.

2. New documentation

I wrote brand new documentation that should make it much easier to get started and to troubleshoot potential issues.

You can find it in the readme of the repo above.

And to make it even easier to get started, I listened to the suggestion by Tobi and created...

3. A minimal example zmk-config

It guides you all the way from creating a fresh zmk-config to adding everything you need to get your trackpoint working.

I also published a pre-compiled firmware that is known to work. This should make it very easy to prototype and test your trackpoints either on a keyboard or breadboard without needing to figure out all the config details first.

You can find it here:
https://github.com/infused-kim/kb_zmk_ps2_mouse_trackpoint_driver-zmk_config

4. I also wrote a "blog post"

It's about the development journey and some implementation details that some of you might enjoy.

You can find it at the bottom of the driver module repo:

https://github.com/infused-kim/kb_zmk_ps2_mouse_trackpoint_driver#5-development-journey-blog--implementation-details

PS2 GPIO: Added initial structure

PS2 GPIO: Added interrupt handler and pin reading/writing helper functions

PS2 GPIO: Added reading mode

PS2 GPIO: Add zephyr PS2 API support for callback and read function

PS2 GPIO: Fix reading not working with real PS2 device

PS2 GPIO: Added write with separate edge rising callback

PS2 GPIO: Made single interrupt handler

PS2 GPIO: Switch to inactive and falling handler only

PS2 GPIO: Switch back to edge falling

PS2 GPIO: Working without writing

PS2 GPIO: Enable both edges (working)

PS2 GPIO: Somewhat working write

PS2 GPIO: Write with ack working, but read always wrong stop bit

PS2 GPIO: Cleanup code

PS2 GPIO: Fix write not working by increasing clock holding time

PS2 GPIO: Adde scl clock timeout for reads

PS2 GPIO: Add scl clock timeout for writes

PS2 GPIO: Moved pin config into separate functions (not working)

PS2 GPIO: Make read work by removing some logging

PS2 GPIO: Fixed memory issue with fifo

PS2 GPIO: Kind of working sometimes except for ack bit

PS2 GPIO: Working without ack

PS2 GPIO: Fix crash: Disable resend command in read abort

PS2 GPIO: Make write work (But next read fails)
PS2 GPIO:
PS2 GPIO: I think the next read fails because we swallow the first 0 bit when reading the ack.

PS2 GPIO: Disable logging go speed up processing

PS2 GPIO: Made writing mostly work; read afterwards inconsistent

PS2 GPIO: Added post write debug log

PS2 GPIO: Improved post-write reading success rate

PS2 GPIO: Adjust timings

PS2 GPIO: Added logging of time between interrupts

PS2 GPIO: Instead of sleep() for write line inhibition, do delayable work

PS2 GPIO: Compensate for missing stop bit in read when it happens directly after write

PS2 GPIO: Adding interrupt logging

PS2 GPIO: Fix set_scl() setting sda and set_sda() setting scl

PS2 GPIO: Make interrupt logging work for sends
PS2 GPIO:
PS2 GPIO: Add interrupt logging to read functions

PS2 GPIO: Add ability to disable interrupt callback during write inhibition phase

PS2 GPIO: Rename inhibit slc duration config option
PS2 GPIO:
PS2 GPIO: Add missing inhibit setting rename

PS2 GPIO: Improve handling of spurious interrupts after writing better
PS2 GPIO:
PS2 GPIO: Adjust scl timeouts

PS2 GPIO: Added logging helper
PS2 GPIO:
PS2 GPIO: Logging helper

PS2 GPIO: Start fixing original write functions

PS2 GPIO: Cleanup read code

PS2 GPIO: Cleanup write code

PS2 GPIO: Adjust read/write scl timeouts

PS2 GPIO: Disable logger

PS2 GPIO: Add custom interrupt logging

PS2 GPIO: Enable interrupt logger

PS2 GPIO: Added separate write scl timeout for the first clock

PS2 GPIO: Fix running out of memory due to read queue

PS2 GPIO: Allow disabling of interrupt logs

PS2 GPIO: Improve logging and comments

PS2 GPIO: Removed old interrupt code
PS2 GPIO:
PS2 GPIO: interrupt stuff
PS2 GPIO:
PS2 GPIO: x

PS 2 Mouse: Add initial mouse driver for testing PS2 driver

PS2 GPIO: Add ps2 device to callback

PS2 Mouse: Added initial mouse movement cmd parsing

PS2 Mouse: Added actual mouse movement
PS2 Mouse:
PS2 Mouse: Inverted y for zmk as that is what zmk thinks is up
PS2 Mouse:
PS2 Mouse: comm

PS2 Mouse: Changed config options to match zmk mouse naming

PS2 Mouse: Added options to invert x and y mouse movements

PS2 Mouse: Rename mouse movement to mouse activity

PS2 Mouse: Added mouse button support

PS2 GPIO: Small cleanup

PS2 GPIO: Move interrupt logging around

PS2 GPIO: Added write re-try on fail

PS2 GPIO: Initial attempt at write await response (not working)

PS2 GPIO: Revert "Initial attempt at write await response (not working)"
PS2 GPIO:
PS2 GPIO: This reverts commit 6e77047ba8f8537e076f3a583d03c6222874998c.

PS2 GPIO: Added reason do abort_read()

PS2 GPIO: Disable resend on read failure for debugging

PS2 GPIO: Initialize current write byte

PS2 GPIO: Added max read retry

PS2 GPIO: Disable log interrupt

PS2 GPIO: Made timings based on ps2 protocol defines
PS2 GPIO:
PS2 GPIO: Move settings

PS2 GPIO: Added write_byte_await_response()

PS2 GPIO: Fix write retry incorrect retry number log

PS2 Mouse: Sort defines

PS2 Mouse: Improve init sequence

PS2 Mouse: Removed unneeded stream mode enabling command

PS2 GPIO: Make no response in write_byte_await_response an failure

PS2 Mouse: During init send reset command on every 4th attempt

PS2 Mouse: Add sample rate setting support

PS2 Mouse: Start use mouse events for movements

PS2 GPIO: Change most logging to debug

PS2 Mouse: Change logs from inf to dbg

PS2 Mouse: Experiment with different mouse movement methods

PS2 Mouse: Report mouse activity through timer instead of as it arrives.

PS2 Mouse: Use zmk log level instead of ps2

PS2 GPIO: Moved retry logic into separate function that also resends on error resp from device
PS2 GPIO:
PS2 GPIO: Enable retry send for ps2 interface
PS2 GPIO:
PS2 GPIO: Fix write failure error code

PS2 GPIO: Cleanup write code slightly

PS2 GPIO: Disable interrupt logging

PS2 GPIO: Use just one function for parity checking

PS2 GPIO: Small cleanup of read and write
PS2 GPIO:
PS2 GPIO: Small comment improvement

PS2 GPIO: In read_interrupt, moved data bits to a more logical position
PS2 GPIO:
PS2 GPIO: Read interrupt fix

PS2 GPIO: Call zephyr PS2 callback from a worker

PS2 GPIO: Remove unneeded gpio config flags

PS2 GPIO: Allow to enable and disable interrupt log through kconfig

PS2 Mouse: Re-enable resend request due to packet misalignment

PS2 Mouse: Parse cmd buffer using a struct instead of pointers

PS2 Mouse: Cleanup and improve documentation

PS2 Mouse: Silence error on read data queue timeout

PS2 Mouse: Improved init sequence

PS2 Mouse: Move init functions around

PS2 Mouse: Added scroll mode

PS2 Mouse: Make scroll mode configurable

PS2 Mouse: Added functions to incr and decr sampling rate

PS2 GPIO: Update blocking timeout for retry changes

PS2 GPIO: Increased timeout for write_byte_await_response()

PS2 GPIO: Added custom work queue

PS2 Mouse: Added mouse settings behavior (with queue)

PS2 Mouse: Removed queue from mouse setting behavior
PS2 Mouse:
PS2 Mouse: A queue was added to the ps2 gpio driver itself. So there is no need for it here anymore.

PS2 GPIO: Don’t log error on timed out ps2_read()

PS2 Mouse: Added reset-on-power function for trackpoints

PS2 Mouse: Add command to check if device is trackpoint

PS2 Mouse: Added send_cmd function that can send multibyte commands and read response

PS2 Mouse: Added trackpoint press to select and sensitivity setting

PS2 Mouse: Add config option for tap to select

PS2 Mouse: Fixed power-on-reset

PS2 Mouse: Add trackpoint swap xy command

PS2 Mouse: Add swap xy setting

PS2 Mouse: Added option to invert x and y directly on the trackpoint

PS2 Mouse: Added negative inertia setting

PS2 Mouse: Added upper plateau speed (value6)

PS2 Mouse: Added press-to-select threshold

PS2 Mouse: Added keybindings for TP settings

PS2 GPIO: Increased log break time

PS2 GPIO: Re-enable re-send request on read error

PS2 GPIO: Fix outdated comment

PS2 GPIO: Reduce debug logging

PS2 Mouse: Reduce logging

PS2 Mouse: Added config option to disable clicking

PS2 Mouse: Added config option for sampling rate

PS2 GPIO: Automatically adjust BT priorities when PS2 GPIO driver is enabled

PS2 Mouse: Lower default priority of NRF IRQs

PS2 Mouse: Add out of alignment detection

PS2 Mouse: Simplify out of alignment check

PS2 GPIO: Fixed deadlock on write due to wrong wokr queue priorities

PS2 GPIO: Log reason for write finish

PS2 GPIO: Increase SCL timeout

PS2 GPIO: Add pos to abort read logging

PS2 GPIO: For writes read ack bit before interrupt logging

PS2 GPIO: Fix interrupt_log_add crash

PS2 GPIO: Make interrupt_log. log in background

PS2 GPIO: Decrease priority for inhibition wait thread

PS2 GPIO: Increase timeout for first write clock

PS2 GPIO: Add argument to interrupt_log

PS2 GPIO: Improve write debug logging

PS2 Mouse: Improve misalignment detection by checking if there was prev movement

PS2 Mouse: Fix crash caused by zephyr LOG bug due to transient strings
PS2 Mouse:
PS2 Mouse: zephyrproject-rtos/zephyr#44996

PS2 GPIO: Use main write function for 0xfe retry cmd

PS2 GPIO: Add resend callback to gpio driver

PS2 GPIO: Increase inhibition delay timer

PS2 GPIO: Properly fail write if semaphore times out

PS2 Mouse: Added resend callback

PS2 Mouse: Fix TP incr/decrease not remembering prev value

PS2 Mouse: Reset cmd buffer on misalignment

PS2 Mouse: Re-enable abort on sharp movement

PS2 Mouse: Fix mouse scrolling unintendedly if mouse mode is disabled

PS2 Mouse: Log button presses as info

PS2 GPIO: Re-enable scl interrupt on write failure

PS2 GPIO: Increase write max retry

PS2 Mouse: Make compatible with zephyr 3.2 update

PS2 GPIO: Make compatible with zephyr 3.2

PS2 GPIO: Use new zephyr gpio_*_dt apis

PS2 Mouse: Use new zephyr gpio_*_dt apis

PS2 GPIO: Prevent lockout on resend by using cb worker

PS2 GPIO: Add missing interrupt detection

PS2 GPIO: Added write error code for semaphore timeout

PS2 GPIO: Added mutex to prevent multiple threads writing at once

PS2 Mouse: Treat packet overflow as mistransmission

PS2 Mouse: If multiple mouse buttons are changed in one packet, treat it as mistransmission

PS2 Mouse: Fix value6 functions actually setting neg inertia

PS2 Mouse: Remove kconfig setting for mouse

PS2 Mouse: Reordered includes

PS2 Mouse: Switched tp sensitivity setting from float to int

PS2 Mouse: Added setting saving and restoring

PS2 Mouse: Created proper, automatic Kconfig

PS2 Mouse: Reorder includes

PS2 GPIO: Enable PS2 GPIO driver automatically if the devicetree node is present

PS2 Mouse: Rename ps2_mouse to mouse_ps2

PS2 Mouse: Fully rename ps2_mouse to mouse_ps2

PS2 Mouse: Cleanup of mouse_ps2 function  naming and order

PS2 Mouse: Renamed activity cmd to packet

PS2 Mouse: Removed sampling rate from being set from keypresses

PS2 Mouse: Remove redudant decr/incr functions

PS2 Mouse: Rename array functions

PS2 Mouse: Ensure all functions have common prefix zmk_mouse_ps2

PS2 Mouse: Reduce logging

PS2 GPIO: Make sure all functions have the ps2_gpio prefix

PS2 UART: Added initial UART PS2 driver

PS2 UART: Made uart PS2 read work

PS2 UART: Refactored uart read code

PS2 UART: Added parity error checking

PS2 UART: Initial attempt at pincntrl switching

Revert "PS2 UART: Initial attempt at pincntrl switching"

This reverts commit 5538731a90031b7607f05775135385bbe14464dc.

PS2 UART: Initial version of GPIO write (not working)

PS2 UART: Kindoff working

PS2 UART: Working

PS2 UART: Small cleanup

PS2 UART: Adjust timings and improve comments

PS2 Mouse: Increase init priority to be after UART drivers

PS2 UART: Attempt to not block interrupt (not working, worker never called)

Revert "PS2 UART: Attempt to not block interrupt (not working, worker never called)"

This reverts commit 6c37f9a236137cc0ca873c6ee4afef3f69eb5708.

PS2 Mouse: Add config option to disable additional error mitigation

Auto Layer: Add automatic layer toggling on mouse movement

PS2 Mouse: Log transmission error reason

PS2 UART: Check read error in received_byte

PS2 UART: Added blocking write

Auto Layer: Fixes

Scroll Mode: Initial implementation

PS2 Mouse: Auto lint code

PS2 Mouse: Scroll mode improvements (uncommitted changes from 3 month ago)

PS2 GPIO: Apply new auto-format to ps2_gpio

PS2 UART: Apply new auto-format

PS2 GPIO: Make write mutex names unique

PS2 UART: Make write mutex names unique

PS2 GPIO: Fix zephyr 3.5 compile errors

PS2 UART: Fix zephyr 3.5 compile errors

Mouse Settings & Scroll Mode: Apply new auto-formatting

Mouse Settings: Silence zephyr 3.5 compile warnings

PS2 Mouse: Move to zephyr 3.5 input system

PS2 Mouse: Add mouse ps2 work queue

PS2 Mouse: Increase err_msg buffer size for zmk_mouse_ps2_send_cmd_resp to avoid truncating

PS2 UART: Attempt adding pinctrl states dynamically (unsuccessful)

Revert "PS2 UART: Attempt adding pinctrl states dynamically (unsuccessful)"

This reverts commit 1a2c3f6f6832a95a828675a093bef7070fbe5bac.

PS2 UART: Fix zephyr 3.5 pinctrl change error

the issue was that I was using the sleep pinctrl state, because I couldn't figure out how to create a custom state.

And in zephyr 3.2 that worked fine, but in 3.5 it disables the sleep state if CONFIG_PM_DEVICE=n.

So the state was missing.

Mouse Settings & Scroll Mode: Update mouse settings and scroll mode behaviors to new labelless behavior init method

PS2 Mouse: Disable restore of mouse PS2 settings

Mouse Auto Layer: Added toggle delay to prevent accidental activations

PS2 Mouse: Added script to generate interrupt priority overrides for zmk-config

PS2 Mouse: Don’t override interrupt priorities in all zmk builds by default

PS2 GPIO: Potentially fix SCL not being pulled down during write

PS2 UART: Potentially fix inhibition not being done correctly

PS2 Mouse: Reset device if invalid self-test result is received

PS2 UART: Fix PM_DEVICE not enabling

PS2 GPIO: Improve PS2 GPIO kconfig documentation

PS2 UART: Lower BT interrupt priorities

PS2 Mouse: Delay PS2 init sequence to give mouse time to send init data

PS2 UART: Fix first write not working

PS2 UART: Add debug write function

PS2 UART: Disable debug write function

PS2 UART: Log received bytes

PS2 UART: Use delayable instead of k_busy_wait

Revert "PS2 UART: Use delayable instead of k_busy_wait"

Realized that this is pointless, since that’s a blocking function anyways.

We need to not block in the interrupt handler rather than here.

This reverts commit eee0c35793ccccf40d72742d19cc85762423a103.

PS2 UART: Move scl interrupt into worker

PS2 UART: Add cur write pos

Revert "PS2 UART: Move scl interrupt into worker"

This reverts commit ea24c68b1af0f6e4a367622829d92f380d92a27f.

PS2 UART: Added async interrupt writing mode

PS2 UART: Ignore framing errors for 0xfa received bytes

PS2 UART: Log write and received bytes only in DBG

PS2 Mouse: Remove kconfig settings that are handled by input-config

PS2 Mouse: Moved sampling rate and press to select from kconfig to device tree

PS2 Mouse: Log when settings were set successfully

PS2 Mouse: Added devicetree settings for TP settings

Correct val6 default value

PS2 Mouse: Enable settings restore again

PS2 Mouse: Don’t restore runtime settings if device config is present

Mouse Settings: Added log and reset behavior

Adjust settings log

PS2 Mouse: Fix wrong type for tp press to select var

PS2 Mouse: Replace kconfig enable clicking with devicetree disable clicking

PS2 Mouse: Disable error mitigations by default

PS2 Mouse: Moved axis options from kconfig to devicetree

PS2 Mouse: Removed scroll mode behavior

PS2 Mouse: Move scroll mode from kconfig to devicetree

PS2 Mouse: Removed movement buffer / queue

Remove butter kconfigs

PS2 Mouse: Moved mouse_ps2 driver to module

PS2 Mouse: Convert clicking to zephyr 3.5 input system

Mouse Auto Layer: Removed it in preparation for zephyr 3.5 module

PS2 UART: Fix Linking error due to missing config options if logging is disabled

PS2 UART: Replace data queue fifo with message queue to get rid of heap use

PS2 GPIO: Replace data queue fifo with message queue to get rid of heap use

Input Listener: Added logging

Auto Layer Toggle: Added initial version

Auto Layer Toggle: Changed how config is retrieved

Auto Layer Toggle: Adjust logging

Auto Layer Toggle: Remove dependency on urob’s “Zen display & battery tweaks”

That commit adds an additional parameter to the `zmk_keymap_layer_activate` function.

Need to find a better way to make the code compatible whether urob is used or not.

urob@5796aa0

PS2 GPIO & UART: Apply zmk auto-formatting

PS2 UART, GPIO, Mouse: Make resend callback optional to allow compilation without zephyr fork

PS2 Mouse & Auto Layer Toggle: Add urob compatibility option

PS2 Mouse: Remove unneeded file
@PikakueNNZ
Copy link

Thank you @infused-kim for enabling the integration of the trackpoint with zmk. I have successfully implemented the trackpoint on my wireless split keyboard. However, I have encountered an issue: when connected to the computer via Bluetooth, the mouse movement is slightly laggy, whereas it is exceptionally smooth when using a USB connection. It seems the polling rate is very low. but I have checked that the mouse polling rate remains at 90+ regardless of whether it is connected via Bluetooth or USB.

@infused-kim
Copy link
Contributor Author

You are very welcome @PikakueNNZ.

However, I have encountered an issue: when connected to the computer via Bluetooth, the mouse movement is slightly laggy, whereas it is exceptionally smooth when using a USB connection. It seems the polling rate is very low. but I have checked that the mouse polling rate remains at 90+ regardless of whether it is connected via Bluetooth or USB.

Can you check if mouse keys are also laggy on Bluetooth or if it is the same?

And can you try updating to the new module-based version that uses the latest mouse PR to see if it improves?

@PikakueNNZ
Copy link

You are very welcome @PikakueNNZ.

However, I have encountered an issue: when connected to the computer via Bluetooth, the mouse movement is slightly laggy, whereas it is exceptionally smooth when using a USB connection. It seems the polling rate is very low. but I have checked that the mouse polling rate remains at 90+ regardless of whether it is connected via Bluetooth or USB.

Can you check if mouse keys are also laggy on Bluetooth or if it is the same?

And can you try updating to the new module-based version that uses the latest mouse PR to see if it improves?

The mouse keys have no noticeable laggy, but the mouse movement doesn't feel smooth enough. I am now using your mouse_ps2_module_base_urob branch.

I have written a Python script to record mouse movement data and obtained the following results:
When using a USB connection, the time interval of mouse movement is basically around 0.01 seconds.
image

While using a Bluetooth connection, the mouse movement interval is around 0.015 seconds, with intermittent data less than 0.001 seconds. This may be the reason for the unsmooth mouse movement.
image

Here is the Python script I used to record mouse movement data.

from pynput import mouse
import time

# Define global variables to store the previous mouse position
prev_x, prev_y = 0, 0
prev_time = time.time()

# Define a callback function to handle mouse movement events
def on_move(x, y):
    global prev_x, prev_y, prev_time

    # Calculate the distance moved
    distance_x = x - prev_x
    distance_y = y - prev_y

    # Get the current time
    current_time = time.time()

    # Calculate the time interval of movement
    time_interval = current_time - prev_time

    # Print the movement distance and time interval
    print('Mouse moved distance - X: {0}, Y: {1}, time interval: {2}'.format(distance_x, distance_y, time_interval))

    # Update the previous position and time
    prev_x, prev_y = x, y
    prev_time = current_time

# Create a mouse listener
with mouse.Listener(on_move=on_move) as listener:
    listener.join()

infused-kim added a commit to infused-kim/kb_zmk_ps2_mouse_trackpoint_driver that referenced this pull request Apr 26, 2024
@infused-kim
Copy link
Contributor Author

infused-kim commented Apr 26, 2024

@PikakueNNZ thank you for the comprehensive issue description and the script. I love the idea and the ability to debug the issue in a more data driven way.

I have expanded the script and added the ability to print the median time between movements whenever you stop moving the mouse for 0.5s.

You can find it here:
https://github.com/infused-kim/kb_zmk_ps2_mouse_trackpoint_driver/blob/main/scripts/mouse_logger.py

I am on macOS 14.3.1 and get...

  • 10.02 ms on USB with my trackpoint
  • 11.23 ms on BT with my trackpoint
  • 11.33 ms on my Apple BT touchpad

I think 1.2 ms difference between USB and BT are fine.

That being said, I also noticed that on BT the performance is a little "jankier" even though the time intervals are acceptable.

I changed the following options:

# Enable experimental BLE settings to improve performance on BT
CONFIG_ZMK_BLE_EXPERIMENTAL_CONN=y
CONFIG_ZMK_BLE_THREAD_PRIORITY=3

And it seems to be smoother, but the time intervals are about the same as before. So, I can't be sure.

Can you please...

  • Tell me the OS you are on
  • Run the new script on USB and BT to see if there are any differences in the median values
  • Run it again after changing the settings to see if there is a difference
  • Check if the perceived lagginess improves

@PikakueNNZ
Copy link

@infused-kim I tried the new settings, but I didn't see any significant improvement.

My system is Windows 11 22H2.
When connected via USB, the median time is 10.97ms.
When connected via Bluetooth, the median time is 14.48ms.
I agree that 3.5ms difference is acceptable.

However, the unstable time intervals during Bluetooth connection can cause sudden changes in mouse movement speed, which causes the unsmooth mouse movement.
image

In this log, the mouse moved distance is basically the same in each movement, but there are three instances where the time interval is less than 1 ms. If these instances are merged with the previous mouse movement, the result is equivalent to doubling the mouse movement speed.

@infused-kim
Copy link
Contributor Author

@PikakueNNZ, I am not on my computer to compare this, but I am pretty sure I didn’t experience that yesterday.

First, let’s try setting the trackpoints acceleration settings to the original values to make sure that’s not causing issues.

In your mouse_tp.dtsi set:

    &mouse_ps2 {

      // How sensitive the TP is (Default: 128)
      tp-sensitivity = <128>;


      // The maximum mouse movement speed the TP will accelarate to
      // (Default: 97)
      tp-val6-upper-speed = <97>;


      // Negative Inertia (Default: 6)
      // Makes the TP feel more responsive and "overshoot" less.
      tp-neg-inertia = <6>;
}

This will override any custom config in the flash.

And test if the behavior is still happening.

If it is, then the next thing I would like to try is lowering the sample rate.

The sample rate defines how many times per second the trackpoint sends mouse movement to the nice!nano.

If the MCU struggles with keeping up with the TP transfer as well as sending the BT data, then it might not be able to keep sending the movements at a constant rate.

So we can try lowering the sample rate to see if it helps.

     &mouse_ps2 {

      // Set sampling rate
      //
      // The frequency at which the mouse samples and sends data (in hz).
      // The default rate is 100. You can try lowering it if you experience
      // connection errors.
      //
      // Higher values than 100 are no improvement in my experience, but will
      // increase battery life and potentially errors in transmissions.
      //
      // These values are allowed: 10,20,40,60,80,100,200
      sampling-rate = <80>;
  }

Try it without 60 and 80 please.

And could you tell me what TP you are using?

@PikakueNNZ
Copy link

@infused-kim After analyzing the code related to zmk and mouse movement, I found that the controller sends mouse movement data every 10ms through the bt_gatt_notify_cb function. This indicates that the issues is not related to the Trackpoint. I suspect it is related to the Bluetooth configuration.

By adding the following settings, I successfully forced the Bluetooth connection interval to be 10ms.

CONFIG_BT_PERIPHERAL_PREF_MIN_INT=8
CONFIG_BT_PERIPHERAL_PREF_MAX_INT=8

Finally, when connected via Bluetooth, although there are occasional mouse movements with a time interval of 20ms, the majority of the time intervals are 10 milliseconds.
image
The mouse movement is much smoother than before!

@infused-kim
Copy link
Contributor Author

@PikakueNNZ thank you! I tried the options and my performance is much better now too!

@myst729
Copy link

myst729 commented May 25, 2024

@infused-kim mouse keys cannot be used by encoders.

I defined several sensor behaviors like this:

  behaviors {
    volume_encoder: volume_encoder {
      compatible = "zmk,behavior-sensor-rotate";
      #sensor-binding-cells = <0>;
      bindings = <&kp C_VOL_UP>, <&kp C_VOL_DN>;
    };

    scroll_encoder: scroll_encoder {
      compatible = "zmk,behavior-sensor-rotate";
      #sensor-binding-cells = <0>;
      bindings = <&msc SCRL_UP>, <&msc SCRL_DOWN>;
    };

    rgb_encoder: rgb_encoder {
      compatible = "zmk,behavior-sensor-rotate";
      #sensor-binding-cells = <0>;
      bindings = <&rgb_ug RGB_BRI>, <&rgb_ug RGB_BRD>;
    };
  };

Both sensor-bindings = <&volume_encoder>; and sensor-bindings = <&rgb_encoder>; works, but sensor-bindings = <&scroll_encoder>; doesn't.

@infused-kim
Copy link
Contributor Author

@myst729 mouse scroll keys are implemented in #2027. So it’s best to raise the issue there.

But stay tuned. With the next release of the driver you won’t need encoders for scrolling.

@myst729
Copy link

myst729 commented Jun 3, 2024

Oh by the way. Where can I find a schematic for the TrackPoint module? Any type is OK. There's a lot of limits mounting the module because of its size. It'll be much easier if it could be break down and re-layout on the board.

@myst729
Copy link

myst729 commented Jun 3, 2024

Found the datasheet, on page 3. https://www.mikrocontroller.net/attachment/52583/tpm754.pdf

@infused-kim
Copy link
Contributor Author

@myst729 even if you find the schematic and can source all of the components, you will still have to write a custom firmware that handles drift correction, negative inertia and so on.

Not impossible, but something to consider.

You might also find this person's attempt at building their own open source TP useful:
https://www.gbryant.co.uk/posts/2021-07-17_pointing-stick/post.html

@myst729
Copy link

myst729 commented Jun 4, 2024

@infused-kim I'm not going to build my own module. That seems extremely low ROI or impossible, since I don't know where can I get the firmware, either binary or source code. I took the modules from old scrapped ThinkPads, and just want to re-layout the chips, for mounting flexibility. 😄 However, the schematic isn't the same as my modules (I have 3 different revisions, none of them fits). I guess it's feasible, but may take a lot of effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.