Loki: software
AvatarFacialExpressions
The control of an avatar’s facial expressions in virtual reality is mainly based on the automated recognition and transposition of the user’s facial expressions. These isomorphic techniques are limited to what users can convey with their own face and have recognition issues. To overcome these limitations, non-isomorphic techniques rely on interaction techniques using input devices to control the avatar’s facial expressions. Such techniques need to be designed to quickly and easily select and control an expression, and not disrupt a main task such as talking. We present the design of a set of new non- isomorphic interaction techniques for controlling an avatar facial expression in VR using a standard VR controller. These techniques have been evaluated through two controlled experiments to help designing an interaction technique combining the strengths of each approach. This technique was evaluated in a final ecological study showing it can be used in contexts such as social applications.
BoxingCadence
The video time can be controlled through the mouse or keyboard with a frame granularity. Annotations for each athlete can be associated to the current frame by pressing a keyboard key. The tool visualizes all annotations on a timeline below the video, and clicking on one of them jumps to the associated frame in the video.
Chameleon
Chameleon is a system-wide tool that combines computer vision algorithms used for image identification with an open database format to allow for the layering of dynamic content. Using Chameleon, static documents can be easily upgraded by layering user-generated interactive content on top of static images, all while preserving the original static document format and without modifying existing applications.
ClimbingAnalysis
This tool has been designed in collaboration with the Fédération Française de la Montagne et de l'Escalade (FFME) to analyze data from high-level athletes captured using a video annotation tool. It enables data to be visualized using a number of graphs focusing on different types of data, such as hold times and climbing speed. Depending on the type of analysis required, the tool can be used to compare several athletes on the same route, or to analyze an athlete's performances over several competitions.
ClimbingAnnotation
The tool enables to view a video and to add frame-precise annotations to specify the start and end of significant actions. It was designed originally to study lead climbing videos and enables to annotate actions such as grasping or releasing a hold, the athlete's energy consumption, or athletes' and trainers' comments. The annotations can be entered through a sequencer buttons or using keyboard shortcuts. As soon as annotations are entered, the tool aggregate them automatically using plots to visualize them and facilitate the interpretation of a performance results. Plots depict, for instance, the average holding time per hand or the evolution of the score over a time interval.
ClimbingRouteCharacterization
This tool makes it easy to characterize sections of a track, so you can record the characteristics of the wall (size, number of movements, inclination), the people who designed the track, and indicate the technical characteristics of the sections, such as the holds used, physical or mental demands, etc.
Esquisse
Esquisse is a software tool designed to facilitate the production of vector-based illustrative figures of interactive scenarios. In that respect, it relies on a 3D scene where the user imports the necessary elements (interactive devices and characters that interact with these devices), stage the scene by modifying the position and posture of these elements, adjust the virtual camera of the scene, and finally export the view from that camera as a vector-based trace figure (static illustrative figures created to capture the essence of a situation, removing unnecessary details by limiting the graphical representation to the most important contours/outlines of the shown objects and people). Esquisse was built as a web application implemented with React, Typescript, WebAssembly and three.js. The vector-based rendering of the 3D scene is produced thanks to our own implementation of state-of-the-art non-photorealistic rendering algorithms, adapted to the specific needs of Esquisse.
EsquisseBlender
Trace figures are contour drawings of people and objects that capture the essence of scenes without the visual noise of photos or other visual representations. Their focus and clarity make them ideal representations to illustrate designs or interaction techniques. In practice, creating those figures is a tedious task requiring advanced skills, even when creating the figures by tracing outlines based on photos. To mediate the process of creating trace figures, we introduce the open-source tool Esquisse. Informed by our taxonomy of 124 trace figures, Esquisse provides an innovative 3D model staging workflow, with specific interaction techniques that facilitate 3D staging through kinematic manipulation, anchor points and posture tracking. Our rendering algorithm (including stroboscopic rendering effects) creates vector-based trace figures of 3D scenes. We validated Esquisse with an experiment where participants created trace figures illustrating interaction techniques, and results show that participants quickly managed to use and appropriate the tool.
ImageTaxonomy
Software suite implemented for the IllustrationTaxonomy project. It is composed of the following tools: - Coding tool: standalone PyQt application that can be used to code a set of images - Exploration tool: web-based application that facilitates the exploration of coded images - Visualisation tool: web-based application that can be used to browse the hierarchy of codes
Lagmeters
The first method works with most optical mice and allows accurate and real time latency measures up to 5 times per second. In addition, the technique allows easy insertion of probes at different places in the system -- i.e. mouse events listeners -- to investigate the sources of latency. The second method relies on a vibration sensor attached to a finger and a photo-diode to detect the screen response. Both are connected to a micro-controller connected to a host computer using a low-latency USB communication protocol in order to combine software and hardware probes to help determine where the latency comes from. We provide source code and materials to replicate both the hardware and software.
ParkEvolution
The originality of this application relies on the acquisition of data in an ecological context. Thanks to this application that patients download on their personal computer, the data corresponding to cursor displacement on screen and raw input from pointing devices are collected, encrypted and sent to a server. The analysis of this data allows to compute a motor score according to the parameters of movement, in order to identify alterations in fine motor control. We ensure a realistic score based on the important quantity of data collected. This software is written in C++ and runs on Windows. It uses the libpointing library to access raw data from pointing devices.
Polyphony
Polyphony is an experimental toolkit demonstrating the use of Entity-Component-System (ECS) to design Graphical User Interfaces (GUI) on web technologies (HTML canvas or SVG). It also extends the original ECS model to support advanced interfaces.
RayCursor
This is a Unity Project containing the source code and prefab for the pointing technique RayCursor to be easily integrated in other Unity Projects.
Spatial Jitter
Source code for a simulator allowing to measure the spatial jitter introduced by different output frequencies. Participants' anonymous data is also available to test the simulator. The source code allowing to generate the figures presented in the article https://dx.doi.org/10.1145/3379337.3415833 is also available.
TransferFunctionTools
Software tools developed for the research project on hardware-independenent transfer functions in order to allow a developer or an end user to have the same pointing transfer function even if the hardware configurations differ (in terms of size, resolution, etc.).
TurboMouse
End-to-end latency corresponds to the temporal difference between a user input and the corresponding output from a system. It has been shown to degrade user performance in both direct and indirect interaction. If it can be reduced to some extend, latency can also be compensated through software compensation by trying to predict the future position of the cursor based on previous positions, velocities and accelerations. In this paper, we propose a hybrid hardware and software prediction technique specifically designed for partially compensating end-to-end latency in indirect pointing. We combine a computer mouse with a high frequency accelerometer to predict the future location of the pointer using Euler based equations.
WhichFingers
HCI researchers lack low-latency and robust systems to support the design and development of interaction techniques using finger identification. We developed a low-cost prototype using piezo-based vibration sensors attached to each finger. By combining the events from an input device with the information from the vibration sensors we demonstrate how to achieve low-latency and robust finger identification. Our prototype was evaluated in a controlled experiment, using two keyboards and a touchpad, showing single-touch recognition rates of 98.2\% for the keyboard and 99.7\% for the touchpad, and 94.7\% for two simultaneous touches. These results were confirmed in an additional laboratory-style experiment with ecologically valid tasks. Last we present new interaction techniques made possible using this technology.
arrangement-2d-js
Typescript port of the CGAL 2d Arrangement package with WebAssembly.
fast-triangle-triangle-intersection
In order to detect possible intersections between 3D meshes, it is necessary to detect possible intersections between the triangles that make up this or these meshes. This software allows to compute these intersections based on orientation calculations of the triangles, and to recover the geometrical shapes (point, segment, polygon) corresponding to the possible intersections. This tool is implemented in Typescript. It is based on the algorithm described in the article Faster Triangle-Triangle Intersection Tests by Devillers and Guigue, and extends it to handle situations of coplanar triangle intersections that were not handled by the original algorithm.
libParamTuner
libParamTuner provides a lightweight syntax to bind some variables of an application to the parameters defined in an XML file. Each modification of the XML file updates in real time the associated parameters in the application. A graphical interface allows editing the XML file, using interactive controls dynamically created for each parameter.
liblag
The library comprises the management of a set of multitouch input devices, the implementation of latency compensation techniques from the state-of-the-art and new latency compensation techniques developed in the project, and a system to handle artificial latency. The library is developed in C++ using the Qt framework to allow compiling the same code on a wide range of devices and platforms.
libpointing
Libpointing is a software toolkit that provides direct access to HID pointing devices and supports the design and evaluation of pointing transfer functions. The toolkit provides resolution and frequency information for the available pointing and display devices and makes it easy to choose between them at run-time through the use of URIs. It allows to bypass the system's transfer functions to receive raw asynchronous events from one or more pointing devices. It replicates as faithfully as possible the transfer functions used by Microsoft Windows, Apple OS X and Xorg (the X.Org Foundation server). Running on these three platforms, it makes it possible to compare the replicated functions to the genuine ones as well as custom ones. The toolkit is written in C++ with Python, Java and Node.js bindings available (about 49,000 lines of code in total). It is publicly available under the GPLv2 license.
three-mesh-halfedge
This software implements in Typescript and for threeJS geometries the solution described by Kalle Rutanen on his post about half-edge structure. It can be used to navigate through edges and vertices of a 3D mesh, regardless of whether this mesh is one or two-manifold (that is, that the mesh can be split along its various edges and subsequently unfolded so that the mesh lays flat without overlapping pieces). These structures are used as base structure for the nonphotorealistic SVG rendering algorithm used in the Esquisse software. This implementation typically support several contexts that are not handheld by other implementations, for instance if the mesh has isolated polygons, vertices or edges, or if there are multiple edges between the same vertices.
three-svg-renderer
Standalone implementation of a nonphotorealistic rendering algorithm that can be used to render a 3D scene as a SVG vector graphic file. This algorithm is used in the software tool Esquisse. It supports 3D scene with intersection between 3D objects to some extent, and can render visible contour, invisible contour and hidden contours. It also generates lines to emphasize creasings over a certain angle. It also fill each region of a 3D object with a single solid color that corresponds to the color of the corresponding region of this 3D object, ignoring its texture. Some rendering problems might be observed when 3D objects intersect. In order to achieve this result, the renderer analyzes the geometry of all objects in the scene, builds a viewmap of the mesh edges and computes the visibility of each contour in the scene. It then produces a SVG file based on all these information. This algorithm was implemented in three.js with Typescript.
