Cursorless

Cursorless itself is primarily 1) a language, and 2) a text manipulation engine implementing the semantics of that language. Tasks like communicating with an eye tracker, preforming speech recognition, etc, are things we leave to Talon.

Note that in both of the active integrations, we're relying on Talon to give us gaze information at the point that you say a word. We are happy to rely on Talon to provide that abstraction rather than trying to build it ourselves.

Another example of an active integration would be relying on gaze to allow us to reuse good hats. If two hats are not too close to each other, then it's ok if they're the same hat, because we can disambiguate based on which you're looking at.

For example, we could map a mark called "here", which refers to the place you're looking when you say that word. So then you could say "swap funk here with here" to swap two functions, making sure you look at the right function at the moment you say each "here"

An active integration is one that requires effort on our part to build. In general this would involve getting word-level timing from Talon so that we know where the user is looking at the moment they say a particular word.

So when he looks and pops, then all the hats he needs in that area are the good, one-syllable ones. In addition, he can refer to scopes like "funk", "class", etc without needing to mention a hat, because they are implicitly referring to the ones containing his cursor.

He will commonly look somewhere, then pop to move his cursor. He has Cursorless configured with the "greedy" hat allocation setting, which means that it tries really hard to give the best hats to those near the cursor.

Passive integrations are ones where a user benefits from using an eye tracker alongside Cursorless without any formal integration on our part. For example, one of our core maintainers uses an eye tracker, and makes a popping sound to click.

Cursorless itself is primarily 1) a language, and 2) a text manipulation engine implementing the semantics of that language. Tasks like communicating with an eye tracker, preforming speech recognition, etc, are things we leave to Talon.

Note that in both of the active integrations, we're relying on Talon to give us gaze information at the point that you say a word. We are happy to rely on Talon to provide that abstraction rather than trying to build it ourselves.

Another example of an active integration would be relying on gaze to allow us to reuse good hats. If two hats are not too close to each other, then it's ok if they're the same hat, because we can disambiguate based on which you're looking at.

For example, we could map a mark called "here", which refers to the place you're looking when you say that word. So then you could say "swap funk here with here" to swap two functions, making sure you look at the right function at the moment you say each "here"

An active integration is one that requires effort on our part to build. In general this would involve getting word-level timing from Talon so that we know where the user is looking at the moment they say a particular word.

So when he looks and pops, then all the hats he needs in that area are the good, one-syllable ones. In addition, he can refer to scopes like "funk", "class", etc without needing to mention a hat, because they are implicitly referring to the ones containing his cursor.

He will commonly look somewhere, then pop to move his cursor. He has Cursorless configured with the "greedy" hat allocation setting, which means that it tries really hard to give the best hats to those near the cursor.

Passive integrations are ones where a user benefits from using an eye tracker alongside Cursorless without any formal integration on our part. For example, one of our core maintainers uses an eye tracker, and makes a popping sound to click.

@eyewarebeam I’m not sure I’d say they figure out their multimodal approach individually. Cursorless is built on top of talon; anything multimodal will be handled by that layer of the stack

Our new site is live 🙌 cursorless.org/