Using a LeapMotion for Qt inputs

We are still working on our project Tepee3D and although we have not yet reached the popularity we hoped too, this project has allowed us to work on many interesting aspects, one of which is adding LeapMotion inputs to our application.
For those of you who don’t know what a LeapMotion is, it is a small sensor about the size of a lighter that you can use to track hands movement in space.
You can find more information about it on the LeapMotion web site.

Leap Motion

Leap Motion (Photo credit: khawkins04)

There are several ways to interact with an application using the LeapMotion sensor. However as Tepee3D has mainly been focused on touch and multitouch inputs, one simple way to use the LeapMotion was to convert fingers 3D position in Qt touch points so that they could then be used in QTouchEvent or QMouseEvents without having to rewrite the entire application.

For the specific gestures (swipe, circle, screentap, keytap) offered by the Leap SDK, creating a custom QtQuick/QWidget element that will handle these is a clean way to include the additional inputs of the LeapMotion without compromising touch and mouse inputs on platforms that do not have access to a LeapMotion.

Though originally developed for Tepee3D, this library is made to work with any Qt/QtQuick application.

Those not interested by the code explanation below, you can directly obtain the sources and a sample project here.

In order to run the project, you have to download the Leap SDK and copy the include directory in the Leap subdirectory in the project. This is clearly explained in the README. The Leap shared libraries are bundled in but to respect the SDK agreement, I cannot provide the headers.

You can download the SDK here (but you have to sign up)

We will start our LeapMotion library by creating LeapMotionController class. This will be used to initialize the LeapMotion sensors and allow QObject (QQuickItem or QWindow) to register to the inputs or gestures. There is no reason why it shouldn’t work with QWidget but I haven’t tested.

To be sure the objects registering to either inputs or gestures implement the right methods, they have to inherit from at least one of the interfaces below. For touch or mouse inputs there is nothing special about the interface. For gestures however, you need to implement callback methods for each of the gestures.

#### LeapMotionServiceUserInterface.h ####

#ifndef LEAPMOTIONSERVICEUSERINTERFACE_H
#define LEAPMOTIONSERVICEUSERINTERFACE_H

#include <QObject>
#include <QVector3D>
#include <QPointF>

namespace Services
{

class LeapMotionServiceInputUserInterface
{
public :
 // REGISTER SIGNAL
 virtual void registerToLeapMotionInputs(QObject *listener) = 0;
 virtual void unregisterFromLeapMotionInputs(QObject *listener) = 0;
};

class LeapMotionServiceGestureUserInterface
{
 // CALLBACK METHOD
public :
 enum GestureState
 {
 StartState = 0,
 UpdateState,
 DoneState
 };

 virtual void onCircleGestureCallBack(int gestureId,
 const QVector3D cicrcleCenter,
 const QVector3D circleNormal,
 const float circleRadius,
 const float circleTurns,
 const bool clockwise,
 const GestureState circleGestureState) = 0;
 virtual void onScreenTapGestureCallBack(int gestureId,
 const QVector3D screenTapDirection,
 const QVector3D screenTapPosition,
 const GestureState screenTapGestureState = DoneState) = 0;
 virtual void onKeyTapGestureCallBack(int gestureId,
 const QVector3D keyTapDirection,
 const QVector3D keyTapPosition,
 const GestureState keyTapGestureState = DoneState) = 0;
 virtual void onSwipeGestureCallBack(int gestureId,
 const QVector3D swipeDirection,
 const QVector3D swipePosition,
 const QVector3D swipeStartPosition,
 const float swipeSpeed,
 const GestureState swipeGestureState) = 0;

public :
 // REGISTER SIGNAL
 virtual void registerToLeapMotionGestures(QObject *listener) = 0;
 virtual void unregisterFromLeapMotionGestures(QObject *listener) = 0;
};

}
Q_DECLARE_INTERFACE(Services::LeapMotionServiceInputUserInterface, "com.tepee3d.Services.LeapMotionServiceInputUserInterface/1.0")
Q_DECLARE_INTERFACE(Services::LeapMotionServiceGestureUserInterface, "com.tepee3d.Services.LeapMotionServiceGestureUserInterface/1.0")

#endif // LEAPMOTIONSERVICEUSERINTERFACE_H

Note : All classes prefixed by Leap:: are part of the Leap SDK and are included by Leap.h. They are provided with the Leap SDK.

#### LeapMotionController.h #####


#ifndef LEAPMOTIONCONTROLLER_H
#define LEAPMOTIONCONTROLLER_H

#include <QObject>
#include "Leap.h"
#include "LeapMotionListener.h"

class LeapMotionController : public QObject
{
 Q_OBJECT

private :
 LeapMotionController(QObject *parent = 0);

 static LeapMotionController *instance;
 LeapMotionListener *leapListener;
 Leap::Controller *leapController;

public:
 ~LeapMotionController();

 static LeapMotionController *getInstance();

public slots:
 void registerTargetListenerToLeapMotionInputs(QObject *listener);
 void registerTargetListenerToLeapMotionGestures(QObject *listener);
 void unregisterTargetListenerFromLeapMotionInputs(QObject *listener);
 void unregisterTargetListenerFromLeapMotionGestures(QObject *listener);
};

#endif // LEAPMOTIONCONTROLLER_H

#### LeapMotionController.cpp #####


#ifndef LEAPMOTIONCONTROLLER_H
#define LEAPMOTIONCONTROLLER_H

#include <QObject>
#include "Leap.h"
#include "LeapMotionListener.h"

class LeapMotionController : public QObject
{
 Q_OBJECT

private :
 LeapMotionController(QObject *parent = 0);

 static LeapMotionController *instance;
 LeapMotionListener *leapListener;
 Leap::Controller *leapController;

public:
 ~LeapMotionController();

 static LeapMotionController *getInstance();

public slots:
 void registerTargetListenerToLeapMotionInputs(QObject *listener);
 void registerTargetListenerToLeapMotionGestures(QObject *listener);
 void unregisterTargetListenerFromLeapMotionInputs(QObject *listener);
 void unregisterTargetListenerFromLeapMotionGestures(QObject *listener);
};

#endif // LEAPMOTIONCONTROLLER_H

#### LeapMotionController.cpp ####

#include "LeapMotionController.h"

LeapMotionController * LeapMotionController::instance = NULL;

LeapMotionController::LeapMotionController(QObject *parent) : QObject(parent)
{
 this->leapController = new Leap::Controller();
 this->leapListener = new LeapMotionListener();
 // ADD LISTENER TO LEAP CONTROLLER
 this->leapController->addListener(*this->leapListener);
}

LeapMotionController::~LeapMotionController()
{
 this->leapController->removeListener(*this->leapListener);
 delete this->leapListener;
 delete this->leapController;
 LeapMotionController::instance = NULL;
}

LeapMotionController *LeapMotionController::getInstance()
{
 if (LeapMotionController::instance == NULL)
 LeapMotionController::instance = new LeapMotionController();
 return LeapMotionController::instance;
}

void LeapMotionController::registerTargetListenerToLeapMotionInputs(QObject *listener)
{
 this->leapListener->addInputListener(listener);
}

void LeapMotionController::registerTargetListenerToLeapMotionGestures(QObject *listener)
{
 this->leapListener->addGestureListener(listener);
}

void LeapMotionController::unregisterTargetListenerFromLeapMotionInputs(QObject *listener)
{
 this->leapListener->removeInputListener(listener);
}

void LeapMotionController::unregisterTargetListenerFromLeapMotionGestures(QObject *listener)
{
 this->leapListener->removeGestureListener(listener);
}

The LeapMotionController class is basically a wrapper around the LeapMotionListener class. How the Leap SDK works is by having you implement a Leap::Listener.
This listener runs in its own thread and has callbacks that will be triggered when the sensors state changes.

Note that we created a Leap::Controller instance that will be needed to instanciate our listener.

The LeapMotionListener is where the sensor’s data are received and converted to inputs events for our Qt application. To create your own listener, you need to subclass Leap::Listener and implement its callbacks method.


 void onInit(const Leap::Controller &);
 void onConnect(const Leap::Controller &);
 void onDisconnect(const Leap::Controller &);
 void onExit(const Leap::Controller &);
 void onFrame(const Leap::Controller &);
 void onFocusGained(const Leap::Controller &);
 void onFocusLost(const Leap::Controller &);

There are two callbacks methods that really matter : onInit(const Leap::Controller &) and onFrame(const Leap::Controller &)

onInit is called when the the sensors is initialized and this is where you configure what type of gestures you want to receive and what is required for them to be recognized.
onFrame is called for every frame of data (between 1 and 240 per second) that the sensor has processed. onFrame is where you will read the data and translate them to Qt Touch or Mouse events.

For each onFrame call, you are able to retrieve a Leap::Frame object which gives a simple access to various Leap objects. All inputs detected appears as Leap::Pointable. A Leap::Pointable can either be a Leap::Finger or a Leap::Tool (a tool might be a pencil that you put over the device instead of a finger).

To ease the process of recognizing pointables, the Leap::Hand object contains a list of Leap::Pointables associated with a given hand but you can also retrieve the whole list of pointables in a frame with the Leap::Frame object directly. The list class which contains Leap::Pointable(s) has convenience methods to retrieve the frontmost, rightmost and leftmost pointables of a hand or a frame.

Next, the Leap::Frame contains a list of Leap::Gesture which are the moment can either be a Swipe, Circle, Screen tap or Key Tap. To handle there, the LeapMotionListener class has a callback method for each of those gestures that extract the gesture’s data.

The documentation is surprisingly easy to read and if you’d like to know more about the various Leap objects here is a link to a list of the LEAP classes.

In addition to the callbacks, LeapMotionListener also contains two lists of QObject *, that will contain the objects that have registered to inputs and or gestures of our library.

For each frame that we receive, the data are processed in 3 pass. First we convert them to QMouseEvent in handleMouseEvents. Then we convert them to QTouchEvent in handleTouchEvents and then the gestures recognized by the Leap SDK are processed with the gesture handlers.

#### LeapMotionListener.h ####


#ifndef LEAPMOTIONLISTENER_H
#define LEAPMOTIONLISTENER_H

#include <QObject>
#include <QList>
#include <QCoreApplication>
#include <QGuiApplication>
#include <QScreen>
#include <QWindow>
#include <QTouchEvent>
#include <QMouseEvent>
#include <QHash>
#include <QQuickItem>
#include "LeapMotionTouchDevice.h"
#include "LeapMotionServiceUserInterface.h"
#include "Leap.h"
#include <qmath.h>

class LeapMotionListener : public QObject, public Leap::Listener
{
 Q_OBJECT

public :
 LeapMotionListener();
 ~LeapMotionListener();

private :
 QList<QObject*> inputListeners;
 QList<QObject*> gesturesListeners;
 QScreen *primaryScreen;
 QHash<Leap::Gesture::State, Services::LeapMotionServiceGestureUserInterface::GestureState> gestureStateMatcher;
 int savedMousePointableId;

 QList<QTouchEvent::TouchPoint> touchPoints;
 bool mousePressed;
 QPoint previousPos;
 QHash<Leap::Gesture::Type, void (LeapMotionListener::*)(const Leap::Gesture &gesture, const Leap::Frame &frame)> gestureHandlers;

 void swipeGestureHandler(const Leap::Gesture &gesture, const Leap::Frame & frame);
 void circleGestureHandler(const Leap::Gesture &gesture, const Leap::Frame & frame);
 void keyTapGestureHandler(const Leap::Gesture &gesture, const Leap::Frame & frame);
 void screenTapGestureHandler(const Leap::Gesture &gesture, const Leap::Frame & frame);

 void handleMouseEvents(const Leap::Frame &frame);
 void handleTouchEvents(const Leap::Frame &frame);
 QPointF convertHandPosToScreenPos(const Leap::InteractionBox &interactionBox,
 const Leap::Hand &hand);
 QPointF convertPointablePosToScreenPos(const Leap::InteractionBox &interactionBox,
 const Leap::Pointable &pointable);
 QPointF convertPointablePosToScreenPos(const Leap::Vector &normalizedPos);
 QPointF convertGlobalPosToLocalPos(QObject *container, const QPointF &globalPos);
 QVector3D convertVectorToNormalizedScreenVector(const Leap::InteractionBox &interactionBox,
 const Leap::Vector &rawPos);
public:
 // CALLBACKS FOR THE Leap::Listener
 void onInit(const Leap::Controller &);
 void onConnect(const Leap::Controller &);
 void onDisconnect(const Leap::Controller &);
 void onExit(const Leap::Controller &);
 void onFrame(const Leap::Controller &);
 void onFocusGained(const Leap::Controller &);
 void onFocusLost(const Leap::Controller &);

public:
 void addInputListener(QObject *target);
 void removeInputListener(QObject *target);
 void addGestureListener(QObject *listener);
 void removeGestureListener(QObject *listener);
};

#endif // LEAPMOTIONLISTENER_H

The LeapMotionListener.cpp contains a bit more code but should be really straightforward. If you don’t want to read it all jump to onFrame and then to the handleMouseEvents and handleTouchEvents methods and you’ll have an idea of how it works.

#### LeapMotionListener.cpp ####


#include "LeapMotionListener.h"
#include <QDebug>

LeapMotionListener::LeapMotionListener() : QObject(), Leap::Listener()
{
 this->savedMousePointableId = -1;
 this->touchPoints = QList<QTouchEvent::TouchPoint>();
 this->mousePressed = false;
 this->previousPos = QPoint();
 this->primaryScreen = QGuiApplication::primaryScreen();
 this->inputListeners = QList<QObject *>();
 this->gesturesListeners = QList<QObject *>();
 this->gestureStateMatcher[Leap::Gesture::STATE_START] = Services::LeapMotionServiceGestureUserInterface::StartState;
 this->gestureStateMatcher[Leap::Gesture::STATE_UPDATE] = Services::LeapMotionServiceGestureUserInterface::UpdateState;
 this->gestureStateMatcher[Leap::Gesture::STATE_STOP] = Services::LeapMotionServiceGestureUserInterface::DoneState;
}

LeapMotionListener::~LeapMotionListener()
{
 this->inputListeners.clear();
 this->gesturesListeners.clear();
}

void LeapMotionListener::swipeGestureHandler(const Leap::Gesture &gesture, const Leap::Frame &frame)
{
 Leap::SwipeGesture swipe = gesture;
 QVector3D swipeDirection(swipe.direction().x, swipe.direction().y, swipe.direction().z);
 QVector3D swipeGlobalPos = this->convertVectorToNormalizedScreenVector(frame.interactionBox(), swipe.position());
 QVector3D swipeStartPos = this->convertVectorToNormalizedScreenVector(frame.interactionBox(), swipe.startPosition());
 Services::LeapMotionServiceGestureUserInterface::GestureState state = this->gestureStateMatcher[gesture.state()];

 // IF SWIPE IS CONTAINED IN THE LISTENER AREA, THIS IS CHECKED BY LeapGestureArea IF THIS IS THE LISTENER
 foreach (QObject *listener, this->gesturesListeners)
 qobject_cast<Services::LeapMotionServiceGestureUserInterface *>(listener)->
 onSwipeGestureCallBack(swipe.id(),
 swipeDirection,
 swipeGlobalPos,
 swipeStartPos,
 swipe.speed(),
 state);
}

void LeapMotionListener::circleGestureHandler(const Leap::Gesture &gesture, const Leap::Frame &frame)
{
 Leap::CircleGesture circle = gesture;
 QVector3D circleCenter = this->convertVectorToNormalizedScreenVector(frame.interactionBox(), circle.center());
 QVector3D circleNormal = QVector3D(circle.normal().x, circle.normal().y, circle.normal().z);
 Services::LeapMotionServiceGestureUserInterface::GestureState state = this->gestureStateMatcher[gesture.state()];

 // IF SWIPE IS CONTAINED IN THE LISTENER AREA, THIS IS CHECKED BY LeapGestureArea IF THIS IS THE LISTENER
 foreach (QObject *listener, this->gesturesListeners)
 qobject_cast<Services::LeapMotionServiceGestureUserInterface *>(listener)->
 onCircleGestureCallBack(circle.id(),
 circleCenter,
 circleNormal,
 circle.radius(),
 circle.progress(),
 circle.pointable().direction().angleTo(circle.normal()) <= M_PI / 2,
 state);
}

void LeapMotionListener::keyTapGestureHandler(const Leap::Gesture &gesture, const Leap::Frame &frame)
{
 Leap::KeyTapGesture keyTap = gesture;
 QVector3D tapPos = this->convertVectorToNormalizedScreenVector(frame.interactionBox(), keyTap.pointable().stabilizedTipPosition());
 QVector3D tapDirection(keyTap.direction().x, keyTap.direction().y, keyTap.direction().z);
 // IF SWIPE IS CONTAINED IN THE LISTENER AREA, THIS IS CHECKED BY LeapGestureArea IF THIS IS THE LISTENER
 foreach (QObject *listener, this->gesturesListeners)
 qobject_cast<Services::LeapMotionServiceGestureUserInterface *>(listener)->
 onKeyTapGestureCallBack(keyTap.id(), tapDirection, tapPos);
}

void LeapMotionListener::screenTapGestureHandler(const Leap::Gesture &gesture, const Leap::Frame &frame)
{
 Leap::ScreenTapGesture screenTap = gesture;
 QVector3D tapDirection = QVector3D(screenTap.direction().x, screenTap.direction().y, screenTap.direction().z);
 QVector3D screenTap3DPos = this->convertVectorToNormalizedScreenVector(frame.interactionBox(), screenTap.pointable().stabilizedTipPosition());
 // QCursor ::setPos(globalPointerPos.toPoint());
 // IF SWIPE IS CONTAINED IN THE LISTENER AREA, THIS IS CHECKED BY LeapGestureArea IF THIS IS THE LISTENER
 foreach (QObject *listener, this->gesturesListeners)
 qobject_cast<Services::LeapMotionServiceGestureUserInterface *>(listener)->
 onScreenTapGestureCallBack(screenTap.id(), tapDirection, screenTap3DPos);
}

void LeapMotionListener::handleMouseEvents(const Leap::Frame &frame)
{
 /////// MOUSE EVENTS /////////
 // MOUSE BUTTON PRESSED
 // MOUSE BUTTON RELEASED
 // MOUSE MOVE

 if (this->inputListeners.empty())
 return ;

 Leap::Pointable pointer = frame.pointable(this->savedMousePointableId);
 if (!pointer.isValid())
 {
 pointer = frame.pointables().frontmost();
 this->savedMousePointableId = pointer.id();
 }

 bool forceRelease = (frame.pointables().count() == 0 && this->mousePressed);
 QMouseEvent::Type frameMouseEvent = QMouseEvent::None;
 QPointF globalPointerPos = this->convertPointablePosToScreenPos(frame.interactionBox(), pointer);
 Qt::MouseButton button = Qt::NoButton;
 Qt::MouseButtons buttons = Qt::NoButton;

 // FINGER TOUCHING AND NO PREVIOUS PRESS -> SETTING BUTTON PRESS
 if (pointer.touchDistance() <= 0 &&
 pointer.touchZone() == Leap::Pointable::ZONE_TOUCHING &&
 !this->mousePressed)
 {
 this->mousePressed = true;
 frameMouseEvent = QMouseEvent::MouseButtonPress;
 button = Qt::LeftButton;
 }
 else if (this->mousePressed && (pointer.touchDistance() > 0 ||
 pointer.touchZone() == Leap::Pointable::ZONE_NONE ||
 forceRelease)) // FINGER NOT TOUCHING AND PREVIOUS PRESS -> RELEASING BUTTON PRESS
 {
 frameMouseEvent = QMouseEvent::MouseButtonRelease;
 this->mousePressed = false;
 button = Qt::LeftButton;
 }
 else if (frameMouseEvent == QMouseEvent::None && // FINGER IN TOUCHING OR HOVERING ZONE AND NO BUTTON PRESS / RELEASE CHANGE -> MouseMove
 pointer.touchZone() != Leap::Pointable::ZONE_NONE
 && globalPointerPos.toPoint() != this->previousPos)
 {
 frameMouseEvent = QMouseEvent::MouseMove;
 this->previousPos = globalPointerPos.toPoint();
 QCursor::setPos(this->previousPos);
 }

 if (this->mousePressed)
 buttons |= Qt::LeftButton;

 if (frameMouseEvent != QMouseEvent::None)
 foreach (QObject *listener, this->inputListeners)
 QCoreApplication::postEvent(listener, new QMouseEvent(frameMouseEvent,
 this->convertGlobalPosToLocalPos(listener, globalPointerPos),
 globalPointerPos,
 button,
 buttons,
 Qt::NoModifier));
}

void LeapMotionListener::handleTouchEvents(const Leap::Frame &frame)
{
 if (this->inputListeners.empty())
 return ;
 /////// TOUCH EVENTS /////////
 // TOUCHBEGIN
 // TOUCHUPDATE
 // TOUCHEND
 // TOUCHCANCEL

 QTouchEvent::Type frameTouchEvent = QTouchEvent::None;
 Qt::TouchPointStates touchPointState = 0;
 QList<Leap::Pointable> touchingPointables;

 // RETRIEVE ONLY VALID AND TOUCHING POINTS
 for (int i = 0; i < frame.pointables().count(); i++)
 {
 const Leap::Pointable pointer = frame.pointables()[i];
 if (pointer.touchDistance() <= 0.5 && // IF TOO SENSISBLE CHANGE 0 FOR A VALUE LIKE 0.5 OR 0.75
 pointer.touchZone() != Leap::Pointable::ZONE_NONE &&
 pointer.isValid())
 touchingPointables.append(pointer);
 }

 int pointCountDiff = touchingPointables.count() - this->touchPoints.size();

 if (pointCountDiff == 0 && !this->touchPoints.empty()) // SAME AMOUT OF POINTS AS PREVIOUS FRAME AND MORE THAN 0 POINTS
 {
 // qDebug() << "Touch Update 1";
 frameTouchEvent = QTouchEvent::TouchUpdate;
 }
 else if (pointCountDiff < 0) // LESS POINTS PRESSED THAN PREVIOUS FRAME
 {
 if (touchingPointables.count() == 0) // NO MORE TOUCHING POINTS
 {
 // qDebug() << "Touch End";
 frameTouchEvent = QTouchEvent::TouchEnd;
 }
 else
 {
 // qDebug() << "Touch Update 2";
 frameTouchEvent = QTouchEvent::TouchUpdate;
 }
 // qDebug() << "Pointables " << touchingPointables.count() << " diff " << pointCountDiff;
 }
 else if (pointCountDiff > 0) // MORE POINTS IN CURRENT FRAME THAN PREVIOUS
 {
 if (this->touchPoints.empty())
 {
 frameTouchEvent = QTouchEvent::TouchBegin;
 // qDebug() << "Touch Begin";
 }
 else
 {
 frameTouchEvent = QTouchEvent::TouchUpdate;
 // qDebug() << "Touch Update 3";
 }
 }

 // ALL TOUCHPOINT ARE SET TO BE RELEASED, IF THEY ARE IN THE CURRENT FRAME, THERE STATE WILL BE UPDATED BELOW HENCE NO RELEASE
 for (int i = 0; i < this->touchPoints.size(); i++)
 {
 QTouchEvent::TouchPoint touchPoint = this->touchPoints.takeAt(i);
 touchPoint.setLastPos(touchPoint.pos());
 touchPoint.setLastScreenPos(touchPoint.screenPos());
 touchPoint.setLastNormalizedPos(touchPoint.normalizedPos());
 touchPoint.setState(Qt::TouchPointReleased);
 this->touchPoints.insert(i, touchPoint);
 }

 if (pointCountDiff < 0)
 touchPointState = Qt::TouchPointReleased;

 // SET TOUCH POINTS LIST IF TOUCHBEGIN OR TOUCHUPDATE
 if (frameTouchEvent != QTouchEvent::None) // NO NEED TO TEST FOR TOUCHEND AS touchPointables == 0 IN THAT CASE
 for (int i = 0; i < touchingPointables.count(); i++)
 {
 Leap::Pointable pointer = touchingPointables[i];
 Leap::Vector normalizedPointerPos = frame.interactionBox().normalizePoint(pointer.stabilizedTipPosition());
 QPointF globalPointerPos = this->convertPointablePosToScreenPos(normalizedPointerPos);
 QPointF localPointerPos = globalPointerPos;
 QTouchEvent::TouchPoint touchPoint(pointer.id());
 touchPoint.setState(Qt::TouchPointPressed);

 // RETRIEVE LAST TOUCH EVENT MATCHING POINTABLE ID IF IT EXISTS
 for (int j = 0; j < this->touchPoints.size(); j++)
 if (this->touchPoints[j].id() == pointer.id())
 {
 touchPoint = this->touchPoints.takeAt(j);
 touchPoint.setState(Qt::TouchPointMoved);
 break;
 }
 // SET ALL ATTRIBUTES THAT NEED TO BE SET OR UPDATED
 touchPoint.setPos(localPointerPos);
 touchPoint.setScreenPos(globalPointerPos);
// touchPoint.setPressure((pointer.touchDistance() - 1) / 2.0);
// if (this->savedMousePointableId == touchPoint.id())
// touchPoint.setPressure(1);
 touchPoint.setPressure(0);
 touchPoint.setNormalizedPos(QPointF(normalizedPointerPos.x, normalizedPointerPos.y));

 if (touchPoint.state() == Qt::TouchPointMoved && touchPoint.pos() == touchPoint.lastPos())
 touchPoint.setState(Qt::TouchPointStationary);

 touchPointState |= touchPoint.state();

 // IF START OF MOUVEMENT, SET STARTING POS OF TOUCH EVENT
 if (touchPoint.state() == Qt::TouchPointPressed)
 {
 touchPoint.setStartPos(localPointerPos);
 touchPoint.setStartScreenPos(globalPointerPos);
 touchPoint.setStartNormalizedPos(QPointF(normalizedPointerPos.x, normalizedPointerPos.y));
 }
 // FIRST TOUCH POINT TRANSLATED TO MOUSE EVENT SO WE MAKE SURE THIS IS THE SAME POINTER AS THE MOUSE
// if (this->savedMousePointableId != touchPoint.id())
 this->touchPoints.append(touchPoint);
 }

 // TRANSMIT EVENT TO TARGETLISTENER IF THE EVENT IS VALID
 if (frameTouchEvent != QTouchEvent::None)
 foreach (QObject *listener, this->inputListeners)
 QCoreApplication::postEvent(listener,
 new QTouchEvent(frameTouchEvent,
 LeapMotionTouchDevice::getInstance(),
 Qt::NoModifier,
 touchPointState,
 this->touchPoints));

 // CLEAR TOUCH POINTS ON TOUCHEND OR TOUCHCANCEL
 if (frameTouchEvent == QTouchEvent::TouchEnd || frameTouchEvent == QTouchEvent::TouchCancel)
 this->touchPoints.clear();

 // CLEAR POINT FROM LIST THAT HAVE BEEN SET TO RELEASED
 for (int i = 0; i < this->touchPoints.size(); i++)
 if (this->touchPoints[i].state() == Qt::TouchPointReleased)
 {
 // qDebug() << "Removing Point =======================================";
 this->touchPoints.removeAt(i);
 i = 0;
 }
}

QPointF LeapMotionListener::convertHandPosToScreenPos(const Leap::InteractionBox &interactionBox,
 const Leap::Hand &hand)
{
 Leap::Vector normalizedPos = interactionBox.normalizePoint(hand.palmPosition());
 return this->convertPointablePosToScreenPos(normalizedPos);
}

QPointF LeapMotionListener::convertPointablePosToScreenPos(const Leap::InteractionBox &interactionBox,
 const Leap::Pointable &pointable)
{
 Leap::Vector normalizedPos = interactionBox.normalizePoint(pointable.stabilizedTipPosition());
 return this->convertPointablePosToScreenPos(normalizedPos);
}

QPointF LeapMotionListener::convertPointablePosToScreenPos(const Leap::Vector &normalizedPos)
{
 return QPointF(normalizedPos.x * this->primaryScreen->geometry().width(),
 (1 - normalizedPos.y) * this->primaryScreen->geometry().height());
}

QVector3D LeapMotionListener::convertVectorToNormalizedScreenVector(const Leap::InteractionBox &interactionBox, const Leap::Vector &rawPos)
{
 Leap::Vector normalizedPos = interactionBox.normalizePoint(rawPos);
 QPointF screenPos = this->convertPointablePosToScreenPos(normalizedPos);
 QVector3D ret = QVector3D(screenPos.x(), screenPos.y(), normalizedPos.z);
 return ret;
}

QPointF LeapMotionListener::convertGlobalPosToLocalPos(QObject *container, const QPointF &globalPos)
{
 QWindow *win = NULL;
 if ((win = qobject_cast<QWindow *>(container)) != NULL)
 return QPointF(win->mapFromGlobal(globalPos.toPoint()));
 return globalPos;
}

void LeapMotionListener::onInit(const Leap::Controller &controller)
{
 qDebug() << "On Listener Init";
 // SET SUPPORTED GESTURES WE WANT TO HANDLE
 controller.enableGesture(Leap::Gesture::TYPE_CIRCLE, true);
 controller.enableGesture(Leap::Gesture::TYPE_SWIPE, true);
 controller.enableGesture(Leap::Gesture::TYPE_SCREEN_TAP, true);
 controller.enableGesture(Leap::Gesture::TYPE_KEY_TAP, true);

 this->gestureHandlers[Leap::Gesture::TYPE_SWIPE] = &LeapMotionListener::swipeGestureHandler;
 this->gestureHandlers[Leap::Gesture::TYPE_CIRCLE] = &LeapMotionListener::circleGestureHandler;
 this->gestureHandlers[Leap::Gesture::TYPE_KEY_TAP] = &LeapMotionListener::keyTapGestureHandler;
 this->gestureHandlers[Leap::Gesture::TYPE_SCREEN_TAP] = &LeapMotionListener::screenTapGestureHandler;
}

void LeapMotionListener::onConnect(const Leap::Controller &)
{
 qDebug() << "On Listener Connect";
}

void LeapMotionListener::onDisconnect(const Leap::Controller &)
{
 qDebug() << "On Listener Disonnect";
}

void LeapMotionListener::onExit(const Leap::Controller &)
{
 qDebug() << "On Listener Exit";
}

void LeapMotionListener::onFrame(const Leap::Controller &controller)
{
 Leap::Frame frame = controller.frame();

 if (frame.isValid())
 {
 // RETRIEVE FINGER INFORMATIONS AND CONVERT THEM TO MOUSE AND TOUCH EVENTS
 this->handleMouseEvents(frame);
 this->handleTouchEvents(frame);
 // ANALYSE FRAME GESTURES AND CALL TARGETS THAT ARE QTQUICK PLUGINS
 // CREATE QTQUICK PLUGINS LeapGestureArea (FOR GESTURES)
 // FOR EACH GESTURE IN A FRAME, CHECK IF QTQUICK PLUGIN HAS IMPLEMENTED GESTURE TYPE AND IF GESTURE OCCURED WITHIN THE QTQUICK PLUGIN AREA
 // IF SO TRIGGER CALLBACK IN QTQUICK PLUGIN
 /////// LEAP MOTION BUILT-IN GESTURES ////////
 // SCREEN TAP
 // SWIPE
 // CIRCLE
 // KEY TAP
 for (int i = 0; i < frame.gestures().count(); i++)
 {
 Leap::Gesture gesture = frame.gestures()[i];
 if (gesture.isValid() && this->gestureHandlers.contains(gesture.type()))
 (this->*this->gestureHandlers[gesture.type()])(gesture, frame);
 }

 /////// TEPEE3D CUSTOM GESTURES DETECTION //////////
 }
}

void LeapMotionListener::onFocusGained(const Leap::Controller &)
{
 qDebug() << "On Listener FocusGained";
}

void LeapMotionListener::onFocusLost(const Leap::Controller &)
{
 qDebug() << "On Listener FocusLost";
}

void LeapMotionListener::addInputListener(QObject *target)
{
 if ((qobject_cast<QQuickItem *>(target) != NULL ||
 qobject_cast<QWindow*>(target) != NULL) &&
 !this->inputListeners.contains(target))
 this->inputListeners.append(target);
}

void LeapMotionListener::removeInputListener(QObject *target)
{
 if (target != NULL && this->inputListeners.contains(target))
 this->inputListeners.removeAll(target);
}

void LeapMotionListener::addGestureListener(QObject *listener)
{
 if (qobject_cast<Services::LeapMotionServiceGestureUserInterface*>(listener) != NULL &&
 !this->gesturesListeners.contains(listener))
 this->gesturesListeners.append(listener);
}

void LeapMotionListener::removeGestureListener(QObject *listener)
{
 if (qobject_cast<Services::LeapMotionServiceGestureUserInterface*>(listener) != NULL &&
 this->gesturesListeners.contains(listener))
 this->gesturesListeners.removeAll(listener);
}

I won’t go into each method but only in handleMouseEvents and handleTouchEvents as they are the most complexed.

Mouse Events Handling

Mouse events could be handled several ways but I chose to use only the frontmost pointer/finger returned by the Leap as the mouse pointer and simulate only the left mouse button. The Leap SDK assigns each finger an unique ID that persits alongs frame as long as the finger is present.
So in order to handle composed mouvements, the ID of the frontmost finger is saved and we try to use the same finger for all subsequent frames. There are 3 types of mouse events : mouseButtonPress, mouseButtonRelease and mouseButtonMove.
The mouseButtonMove event type is created when there is no button press or release events and that the finger has changed position between the current and the previous frame. At the same time we update the mouse cursor to the corresponding position.

Here comes the part of how to detect button press and release events. There are several ways this could have been handled : pressing your thumb against the frontmost finger or detect when the frontmost finger is advanced a bit, closing you hand …
I chose the second option but if you think another pattern would be better, it shouldn’t be that hard to implement it. My choice was influence by the fact that the LEAP SDK simulates a virtual touch plane so you can detect when fingers are pressing, hovering or leaving the touch plane.

If the frontmost finger is detected touching the touch plane and is not already pressed we create a MouseButtonPress event.
On the other hand if the finger is already pressed and is detected as not touching the touch plane anymore, we create a MouseButtonRelease event.

When we have figured the event type, a QMouseEvent is then created.

QMouseEvent::QMouseEvent(Type type, const QPointF & localPos, const QPointF & screenPos, Qt::MouseButton button, Qt::MouseButtons buttons, Qt::KeyboardModifiers modifiers)

The part that made me lose quite some time figuring out why release events were not properly sent to the application was that I didn’t really understand the difference between button and buttons so they were set to the same value.

After reading carefully the doc if finally found out that  :
“The button that caused the event is given as a value from the Qt::MouseButton enum. If the event type is MouseMove, the appropriate button for this event is Qt::NoButton. buttons is the state of all buttons at the time of the event, modifiers the state of all keyboard modifiers.”

That means that when you send a MousePressEvent, button = buttons = Qt::LeftButton. However when the left button is released, button = Qt::LeftButton while buttons = Qt::NoButton.

Transmitting the event to our listeners is just a question of posting the event using QCoreApplication::postEvent, if the listener is a QWidget or a QWindow, the event will be received as if  it were from a real mouse.

Touch Events Handling

Touch events work basically the same way as mouse events with the exception of button press/release handling. By definition a touch point is always pressed so if a touch point is released it is removed from the list of touch points of a QTouchEvent.
In order to avoid an overly sensitive input. For touch point to be detected they have to be slightly more advanced/pressed toward the Leap virtual touch plane than the mouse finger.

Simply put, for each frame, we detect the difference of points touching the touch plane between the previous frame. From that we can deduce whether touch points were added or removed and send accordingly the right type of QTouchEvent :
– TouchBegin (no previous points and one or more points in current frame)
– TouchUpdate (points added or removed but still one or more points in current frame or same amount of points as before)
– TouchEnd (there were points in the previous frame, there are none in the current frame)

Each point of the current frame is saved in a list of QTouchEvent::TouchPoint. On the next frame, we can compare and then update touch points in that list. This also helps us detect which touch points were removed between the current and the previous frame.

QTouchEvent(QEvent::Type eventType, QTouchDevice * device = 0, Qt::KeyboardModifiers modifiers = Qt::NoModifier, Qt::TouchPointStates touchPointStates = 0, const QList<QTouchEvent::TouchPoint> & touchPoints = QList<QTouchEvent::TouchPoint> ())
You´ll notice that when posting a QTouchEvent, LeapMotionTouchDevice::getInstance() is called as the device, this is a simple class used to identify the device that initiated the event.

QTouchEvent and QMouseEvent are posted to objects that have registered themselves by emitting the register signal in LeapMotionServiceInputUserInterface. That means you can register an object that doesn’t implement the interface to mouse of touch inputs. For example you can create a wrapper around a QWindow (instead of subclassing it) and call the register signal by passing the QWindow instance as the object to register.However this is not the case for the gesture interface.

The code for LeapMotionTouchDevice is just below.

#### LeapMotionTouchDevice.h ####


#ifndef LEAPMOTIONTOUCHDEVICE_H
#define LEAPMOTIONTOUCHDEVICE_H

#include <QTouchDevice>

class LeapMotionTouchDevice : public QTouchDevice
{
public:
 static LeapMotionTouchDevice *getInstance();
 ~LeapMotionTouchDevice();

 QTouchDevice::Capabilities capabilities() const;
 QTouchDevice::DeviceType type() const;
 QString name() const;

private:
 static LeapMotionTouchDevice *instance;
 LeapMotionTouchDevice();

};

#endif // LEAPMOTIONTOUCHDEVICE_H

#### LeapMotionTouchDevice.cpp ####


#include "LeapMotionTouchDevice.h"

LeapMotionTouchDevice * LeapMotionTouchDevice::instance = NULL;

LeapMotionTouchDevice *LeapMotionTouchDevice::getInstance()
{
 if (LeapMotionTouchDevice::instance == NULL)
 LeapMotionTouchDevice::instance = new LeapMotionTouchDevice();
 return LeapMotionTouchDevice::instance;
}

LeapMotionTouchDevice::~LeapMotionTouchDevice()
{
 LeapMotionTouchDevice::instance = NULL;
}

QTouchDevice::Capabilities LeapMotionTouchDevice::capabilities() const
{
 return (QTouchDevice::Position|QTouchDevice::Pressure|QTouchDevice::NormalizedPosition);
}

QTouchDevice::DeviceType LeapMotionTouchDevice::type() const
{
 return QTouchDevice::TouchScreen;
}

QString LeapMotionTouchDevice::name() const
{
 return QString("Leap Motion");
}

LeapMotionTouchDevice::LeapMotionTouchDevice() : QTouchDevice()
{
}

It basically simulates a touch screen handling the detection of the position, pressure and normalized position (between 0 and 1) of touch points.
Furthermore, the device is given the name Leap Motion which might eventually help in the case of detectiong which QTouchDevice sent a QTouchEvent.

Gesture Handling

The code pretty much speaks for itself, a gesture handler is assigned for each type of gestures, it retrieves the data associated with the said gesture and calls a gesture callback that all QObject implementing the LeapMotionServiceGestureUserInterface have.

Below is the implementation of a custom QQuickItem that implements the LeapMotionServiceGestureUserInterace in order to receive each gestures. In additon 3 QObject subclasses to represent swipe, circle and taps gesture objects in Qml are also included.

#### LeapGestureArea.h ####


#ifndef LEAPGESTUREAREA_H
#define LEAPGESTUREAREA_H

#include <QQuickItem>
#include <QSGSimpleRectNode>
#include <ViewToModelMapper.h>
#include "LeapMotionServiceUserInterface.h"

namespace Tepee3DQmlExtensions
{

class LeapGesture
{
public :
 virtual int getId() const = 0;
 virtual void setId(int id) = 0;
};

class LeapCircleGesture : public QObject, public LeapGesture
{
 Q_OBJECT
 Q_PROPERTY(int id READ getId)
 Q_PROPERTY(bool clockwise READ getClockwise NOTIFY clockwiseChanged)
 Q_PROPERTY(QVector3D center READ getCenter NOTIFY centerChanged)
 Q_PROPERTY(QVector3D normal READ getNormal NOTIFY normalChanged)
 Q_PROPERTY(qreal radius READ getRadius NOTIFY radiusChanged)
 Q_PROPERTY(qreal turns READ getTurns NOTIFY turnsChanged)
 Q_PROPERTY(CircleGestureState state READ getState NOTIFY stateChanged)
 Q_ENUMS(CircleGestureState)

public:
 enum CircleGestureState {CircleGestureStart = 1, CircleGestureUpdated, CircleGestureDone };

private:
 int m_id;
 QVector3D m_center;
 QVector3D m_normal;
 qreal m_radius;
 qreal m_turns;
 CircleGestureState m_state;
 bool m_clockwise;

public :

 LeapCircleGesture();

 CircleGestureState getState() const;
 QVector3D getCenter() const;
 QVector3D getNormal() const;
 qreal getRadius() const;
 qreal getTurns() const;
 bool getClockwise() const;
 int getId() const;

 void setId(int id);
 void setClockwise(bool clockwise);
 void setState(CircleGestureState state);
 void setCenter(const QVector3D &center);
 void setNormal(const QVector3D &normal);
 void setRadius(qreal radius);
 void setTurns(qreal turns);

signals :
 void stateChanged();
 void centerChanged();
 void normalChanged();
 void radiusChanged();
 void turnsChanged();
 void clockwiseChanged();
};

class LeapSwipeGesture : public QObject, public LeapGesture
{
 Q_OBJECT
 Q_PROPERTY(int id READ getId)
 Q_PROPERTY(QVector3D direction READ getDirection NOTIFY directionChanged)
 Q_PROPERTY(QVector3D position READ getPosition NOTIFY positionChanged)
 Q_PROPERTY(QVector3D startPosition READ getStartPosition NOTIFY startPositionChanged)
 Q_PROPERTY(qreal speed READ getSpeed NOTIFY speedChanged)
 Q_PROPERTY(SwipeGestureState state READ getState NOTIFY stateChanged)
 Q_ENUMS(SwipeGestureState)

public :
 enum SwipeGestureState {SwipeGestureStart = 1, SwipeGestureUpdated, SwipeGestureDone };

private:
 SwipeGestureState m_state;
 QVector3D m_direction;
 QVector3D m_position;
 QVector3D m_startPosition;
 qreal m_speed;
 int m_id;

public :

 LeapSwipeGesture();

 SwipeGestureState getState() const;
 QVector3D getDirection() const;
 QVector3D getPosition() const;
 QVector3D getStartPosition() const;
 qreal getSpeed() const;
 int getId() const;

 void setId(int id);
 void setState(SwipeGestureState state);
 void setDirection(const QVector3D &direction);
 void setPosition(const QVector3D &position);
 void setStartPosition(const QVector3D &startPosition);
 void setSpeed(qreal speed);

signals :
 void directionChanged();
 void positionChanged();
 void startPositionChanged();
 void speedChanged();
 void stateChanged();
};

class LeapTapGesture : public QObject, public LeapGesture
{
 Q_OBJECT
 Q_PROPERTY(int id READ getId)
 Q_PROPERTY(QVector3D direction READ getDirection NOTIFY directionChanged)
 Q_PROPERTY(QVector3D position READ getPosition NOTIFY positionChanged)

private:
 int m_id;
 QVector3D m_direction;
 QVector3D m_position;

public :
 LeapTapGesture();

 int getId() const;
 QVector3D getDirection() const;
 QVector3D getPosition() const;

 void setId(int id);
 void setPosition(const QVector3D position);
 void setDirection(const QVector3D direction);

signals:
 void directionChanged();
 void positionChanged();
};

class LeapGestureArea : public QQuickItem, public Services::LeapMotionServiceGestureUserInterface
{
 Q_OBJECT
 Q_INTERFACES(Services::LeapMotionServiceGestureUserInterface)
// QQmlParserStatus interface
public:
 LeapGestureArea(QQuickItem *parent = 0);
 ~LeapGestureArea();

private :
 QHash<int, Tepee3DQmlExtensions::LeapGesture*> savedGestures;

protected :
 QSGNode *updatePaintNode(QSGNode *oldNode, UpdatePaintNodeData *data);
 // QQmlParserStatus interface
 void componentComplete();
 void itemChange(ItemChange, const ItemChangeData &);

 // LeapMotionServiceGestureUserInterface interface
public:
 void onCircleGestureCallBack(int gestureId,
 const QVector3D cicrcleCenter,
 const QVector3D circleNormal,
 const float circleRadius,
 const float circleTurns,
 const bool clockwise,
 const GestureState circleGestureState);
 void onScreenTapGestureCallBack(int gestureId,
 const QVector3D screenTapDirection,
 const QVector3D screenTapPosition,
 const GestureState screenTapGestureState);
 void onKeyTapGestureCallBack(int gestureId,
 const QVector3D keyTapDirection,
 const QVector3D keyTapPosition,
 const GestureState keyTapGestureState);
 void onSwipeGestureCallBack(int gestureId,
 const QVector3D swipeDirection,
 const QVector3D swipePosition,
 const QVector3D swipeStartPosition,
 const float swipeSpeed,
 const GestureState swipeGestureState);

signals:
 void registerToLeapMotionGestures(QObject *listener);
 void unregisterFromLeapMotionGestures(QObject *listener);
 void circleGesture(LeapCircleGesture *gesture);
 void swipeGesture(LeapSwipeGesture *gesture);
 void keyTapGesture(LeapTapGesture *gesture);
 void screenTapGesture(LeapTapGesture *gesture);

};
}

#### LeapGestureArea.cpp ####


#include "LeapGestureArea.h"

Tepee3DQmlExtensions::LeapGestureArea::LeapGestureArea(QQuickItem *parent) : QQuickItem(parent)
{
}

Tepee3DQmlExtensions::LeapGestureArea::~LeapGestureArea()
{
 // DISCONNECT THE OBJECT LEAP MOTION GESTURES SERVICE
 // emit unregisterFromLeapMotionGestures(this);
}

void Tepee3DQmlExtensions::LeapGestureArea::componentComplete()
{
 QQuickItem::componentComplete();
}

void Tepee3DQmlExtensions::LeapGestureArea::itemChange(QQuickItem::ItemChange, const QQuickItem::ItemChangeData &)
{
 // WHEN A WINDOW IS ASSIGNED TO THE QQUICKITEM, WE REGISTER TROUGHT THE WINDOW TO THE LEAP MOTION SERVICE
 if (this->window())
 {
 qDebug() << "Connecting LeapGestureArea to LeapGestureController";
 // CONNECT TO CUSTOM WINDOW SIGNALS
 QObject::connect(this, SIGNAL(registerToLeapMotionGestures(QObject*)),
 (QObject *)this->window(), SIGNAL(registerQQuickItemToLeapMotionGesturesInput(QObject *)));
 QObject::connect(this, SIGNAL(unregisterFromLeapMotionGestures(QObject*)),
 (QObject *)this->window(), SIGNAL(unregisterQQuickItemFromLeapMotionGesturesInput(QObject *)));
 // CONNECT THE OBJECT AS A LISTENER TO LEAP MOTION GESTURES
 emit registerToLeapMotionGestures(this);
 }
}

QSGNode * Tepee3DQmlExtensions::LeapGestureArea::updatePaintNode(QSGNode *oldNode, UpdatePaintNodeData *data)
{
 Q_UNUSED(data);
 qDebug() << "Update Paint Node";
 QSGSimpleRectNode *rectangle = static_cast<QSGSimpleRectNode *>(oldNode);
 if (!rectangle)
 rectangle = new QSGSimpleRectNode();
 rectangle->setRect(QRectF(0, 0, width(), height()));
 rectangle->setColor(QColor(255, 0, 0, 50));
 this->update();
 return rectangle;
}

void Tepee3DQmlExtensions::LeapGestureArea::onCircleGestureCallBack(int gestureId,
 const QVector3D circleCenter,
 const QVector3D circleNormal,
 const float circleRadius,
 const float circleTurns,
 const bool clockwise,
 const Services::LeapMotionServiceGestureUserInterface::GestureState circleGestureState)
{
 // qDebug() << "LeapGextureArea Circle " << circleCenter;

 if (this->contains(QPointF(circleCenter.x(), circleCenter.y())))
 {
 Tepee3DQmlExtensions::LeapCircleGesture *gesture = NULL;
 if (this->savedGestures.contains(gestureId))
 {
 gesture = static_cast<Tepee3DQmlExtensions::LeapCircleGesture *>(this->savedGestures.take(gestureId));
 if (circleGestureState == Services::LeapMotionServiceGestureUserInterface::DoneState)
 gesture->setState(Tepee3DQmlExtensions::LeapCircleGesture::CircleGestureDone);
 else
 gesture->setState(Tepee3DQmlExtensions::LeapCircleGesture::CircleGestureUpdated);
 }
 else
 {
 gesture = new Tepee3DQmlExtensions::LeapCircleGesture();
 gesture->setId(gestureId);
 }
 gesture->setCenter(circleCenter);
 gesture->setNormal(circleNormal);
 gesture->setRadius(circleRadius);
 gesture->setTurns(circleTurns);
 gesture->setClockwise(clockwise);
 if (circleGestureState != Services::LeapMotionServiceGestureUserInterface::DoneState)
 this->savedGestures[gestureId] = gesture;
 emit circleGesture(gesture);
 }
}

void Tepee3DQmlExtensions::LeapGestureArea::onScreenTapGestureCallBack(int gestureId,
 const QVector3D screenTapDirection,
 const QVector3D screenTapPosition,
 const Services::LeapMotionServiceGestureUserInterface::GestureState screenTapGestureState)
{
 Q_UNUSED(screenTapGestureState)
 if (this->contains(QPointF(screenTapPosition.x(), screenTapPosition.y())))
 {
 Tepee3DQmlExtensions::LeapTapGesture *gesture = new Tepee3DQmlExtensions::LeapTapGesture();
 gesture->setId(gestureId);
 gesture->setDirection(screenTapDirection);
 gesture->setPosition(screenTapPosition);
 emit screenTapGesture(gesture);
 }
}

void Tepee3DQmlExtensions::LeapGestureArea::onKeyTapGestureCallBack(int gestureId,
 const QVector3D keyTapDirection,
 const QVector3D keyTapPosition,
 const Services::LeapMotionServiceGestureUserInterface::GestureState keyTapGestureState)
{
 Q_UNUSED(keyTapGestureState)
 if (this->contains(QPointF(keyTapPosition.x(), keyTapPosition.y())))
 {
 Tepee3DQmlExtensions::LeapTapGesture *gesture = new Tepee3DQmlExtensions::LeapTapGesture();
 gesture->setId(gestureId);
 gesture->setDirection(keyTapDirection);
 gesture->setPosition(keyTapPosition);
 emit keyTapGesture(gesture);
 }
}

void Tepee3DQmlExtensions::LeapGestureArea::onSwipeGestureCallBack(int gestureId,
 const QVector3D swipeDirection,
 const QVector3D swipePosition,
 const QVector3D swipeStartPosition,
 const float swipeSpeed,
 const Services::LeapMotionServiceGestureUserInterface::GestureState swipeGestureState)
{
 if (this->contains(QPointF(swipeStartPosition.x(), swipeStartPosition.y()))
 || this->contains(QPointF(swipePosition.x(), swipePosition.y())))
 {
 Tepee3DQmlExtensions::LeapSwipeGesture *gesture = NULL;
 if (this->savedGestures.contains(gestureId))
 {
 gesture = static_cast<Tepee3DQmlExtensions::LeapSwipeGesture *>(this->savedGestures.take(gestureId));
 if (swipeGestureState == Services::LeapMotionServiceGestureUserInterface::DoneState)
 gesture->setState(Tepee3DQmlExtensions::LeapSwipeGesture::SwipeGestureDone);
 else
 gesture->setState(Tepee3DQmlExtensions::LeapSwipeGesture::SwipeGestureUpdated);
 }
 else
 {
 gesture = new Tepee3DQmlExtensions::LeapSwipeGesture();
 gesture->setId(gestureId);
 }
 gesture->setDirection(swipeDirection);
 gesture->setPosition(swipePosition);
 gesture->setStartPosition(swipeStartPosition);
 gesture->setSpeed(swipeSpeed);
 if (swipeGestureState != Services::LeapMotionServiceGestureUserInterface::DoneState)
 this->savedGestures[gestureId] = gesture;
 emit swipeGesture(gesture);
 }
}

Tepee3DQmlExtensions::LeapCircleGesture::LeapCircleGesture() :
 QObject(),
 m_id(0),
 m_center(0,0,0),
 m_normal(0,0,0),
 m_radius(0),
 m_turns(0),
 m_state(Tepee3DQmlExtensions::LeapCircleGesture::CircleGestureStart),
 m_clockwise(true)
{
}

Tepee3DQmlExtensions::LeapCircleGesture::CircleGestureState Tepee3DQmlExtensions::LeapCircleGesture::getState() const
{
 return this->m_state;
}

QVector3D Tepee3DQmlExtensions::LeapCircleGesture::getCenter() const
{
 return this->m_center;
}

QVector3D Tepee3DQmlExtensions::LeapCircleGesture::getNormal() const
{
 return this->m_normal;
}

qreal Tepee3DQmlExtensions::LeapCircleGesture::getRadius() const
{
 return this->m_radius;
}

qreal Tepee3DQmlExtensions::LeapCircleGesture::getTurns() const
{
 return this->m_turns;
}

bool Tepee3DQmlExtensions::LeapCircleGesture::getClockwise() const
{
 return this->m_clockwise;
}

int Tepee3DQmlExtensions::LeapCircleGesture::getId() const
{
 return this->m_id;
}

void Tepee3DQmlExtensions::LeapCircleGesture::setId(int id)
{
 this->m_id = id;
}

void Tepee3DQmlExtensions::LeapCircleGesture::setClockwise(bool clockwise)
{
 if (clockwise != this->m_clockwise)
 {
 this->m_clockwise = clockwise;
 emit clockwiseChanged();
 }
}

void Tepee3DQmlExtensions::LeapCircleGesture::setState(Tepee3DQmlExtensions::LeapCircleGesture::CircleGestureState state)
{
 if (state != this->m_state)
 {
 this->m_state = state;
 emit stateChanged();
 }
}

void Tepee3DQmlExtensions::LeapCircleGesture::setCenter(const QVector3D &center)
{
 if (center != this->m_center)
 {
 this->m_center = center;
 emit centerChanged();
 }
}

void Tepee3DQmlExtensions::LeapCircleGesture::setNormal(const QVector3D &normal)
{
 if (normal != this->m_normal)
 {
 this->m_normal = normal;
 emit normalChanged();
 }
}

void Tepee3DQmlExtensions::LeapCircleGesture::setRadius(qreal radius)
{
 if (radius != this->m_radius)
 {
 this->m_radius = radius;
 emit radiusChanged();
 }
}

void Tepee3DQmlExtensions::LeapCircleGesture::setTurns(qreal turns)
{
 if (turns != this->m_turns)
 {
 this->m_turns = turns;
 emit turnsChanged();
 }
}

Tepee3DQmlExtensions::LeapSwipeGesture::LeapSwipeGesture() :
 QObject(),
 m_state(Tepee3DQmlExtensions::LeapSwipeGesture::SwipeGestureStart),
 m_direction(0,0,0),
 m_position(0,0,0),
 m_startPosition(0,0,0),
 m_speed(0),
 m_id(0)

{
}

Tepee3DQmlExtensions::LeapSwipeGesture::SwipeGestureState Tepee3DQmlExtensions::LeapSwipeGesture::getState() const
{
 return this->m_state;
}

QVector3D Tepee3DQmlExtensions::LeapSwipeGesture::getDirection() const
{
 return this->m_direction;
}

QVector3D Tepee3DQmlExtensions::LeapSwipeGesture::getPosition() const
{
 return this->m_position;
}

QVector3D Tepee3DQmlExtensions::LeapSwipeGesture::getStartPosition() const
{
 return this->m_startPosition;
}

qreal Tepee3DQmlExtensions::LeapSwipeGesture::getSpeed() const
{
 return this->m_speed;
}

int Tepee3DQmlExtensions::LeapSwipeGesture::getId() const
{
 return this->m_id;
}

void Tepee3DQmlExtensions::LeapSwipeGesture::setId(int id)
{
 this->m_id = id;
}

void Tepee3DQmlExtensions::LeapSwipeGesture::setState(Tepee3DQmlExtensions::LeapSwipeGesture::SwipeGestureState state)
{
 if (state != this->m_state)
 {
 this->m_state = state;
 emit stateChanged();
 }
}

void Tepee3DQmlExtensions::LeapSwipeGesture::setDirection(const QVector3D &direction)
{
 if (direction != this->m_direction)
 {
 this->m_direction = direction;
 emit directionChanged();
 }
}

void Tepee3DQmlExtensions::LeapSwipeGesture::setPosition(const QVector3D &position)
{
 if (position != this->m_position)
 {
 this->m_position = position;
 emit positionChanged();
 }
}

void Tepee3DQmlExtensions::LeapSwipeGesture::setStartPosition(const QVector3D &startPosition)
{
 if (startPosition != this->m_startPosition)
 {
 this->m_startPosition = startPosition;
 emit startPositionChanged();
 }
}

void Tepee3DQmlExtensions::LeapSwipeGesture::setSpeed(qreal speed)
{
 if (speed != this->m_speed)
 {
 this->m_speed = speed;
 emit speedChanged();
 }
}

Tepee3DQmlExtensions::LeapTapGesture::LeapTapGesture() :
 QObject(),
 m_id(0),
 m_direction(0,0,0),
 m_position(0,0,0)
{
}

int Tepee3DQmlExtensions::LeapTapGesture::getId() const
{
 return this->m_id;
}

QVector3D Tepee3DQmlExtensions::LeapTapGesture::getDirection() const
{
 return this->m_direction;
}

QVector3D Tepee3DQmlExtensions::LeapTapGesture::getPosition() const
{
 return this->m_position;
}

void Tepee3DQmlExtensions::LeapTapGesture::setId(int id)
{
 this->m_id = id;
}

void Tepee3DQmlExtensions::LeapTapGesture::setPosition(const QVector3D position)
{
 if (position != this->m_position)
 {
 this->m_position = position;
 emit positionChanged();
 }
}

void Tepee3DQmlExtensions::LeapTapGesture::setDirection(const QVector3D direction)
{
 if (direction != this->m_direction)
 {
 this->m_direction = direction;
 emit directionChanged();
 }
}

If you have a Leap Motion at hand and would like to try this out, I posted a small project on github that should work painlessly once you have included the Leap headers.

Advertisements

8 thoughts on “Using a LeapMotion for Qt inputs

  1. I have enabled Leap Motion in my Qt 5 program Shotcut (http://www.shotcut.org/). First, I did it through the SDK like you did, but I ran into problems at build-time on Windows because I cross-compile for Windows via mingw, and the Leap.dll is not compatible with the mingw compilers. Then, I ran into a run-time problem on Linux where it seems something statically linked into libLeap.so is conflicting with QtWebKit’s HTTP handler. So, I ended up using QWebSockets to connect to leapd on Linux and Windows. (I have yet to announce this feature yet and provide info on how to use it in Shotcut.)

  2. I guess I was lucky because I tried without QtWebKit and on Windows I compiled with msvc. Still, I’d really like to see how you used leapd with QWebSockets as I believe you can achieve a complete cross platform support that way. Additionally that allows to use a LeapMotion on Android devices or a RaspberryPi as long as the LeapMotion is plugged on a supported platform that can stream back the Leap’s data.

  3. Hi Lemire,
    I just happened upon your article about using hierarchical models in QML and ended back here. 🙂
    I used the QWebSockets project on github here:
    https://github.com/KurtPattyn/QWebSockets

    And here is my code for a “Network” LeapListener:
    https://github.com/mltframework/shotcut/blob/master/src/leapnetworklistener.h
    https://github.com/mltframework/shotcut/blob/master/src/leapnetworklistener.cpp

    Hopefully this many links will not flag this as a spam comment.

  4. Hello,
    I try to lauch the project in QtCreator2.7.0 based on Qt 5.0.2 64 bit, I followed your guide, but i have a error of undefined references:
    QtLeapMotionLibrary-master/QtLeapMotionExamples/LeapSwipeGesture/main.cpp:33: error: undefined reference to `QtLeapMotion::QtLeapMotionQQuickView::QtLeapMotionQQuickView()’

    What could be the problem ?

    thank you for your library.

    F.

    • Hi,
      I’m not really sure what could be the problem. Have you compiled the QtLeapMotion library first, the project file is in the QtLeapMotion directory, the examples need the library to compile. Are you on Windows by any chance ? You could also try disabling shadow building in the project tab of QtCreator.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s