Locomotion and haptic interfaces for VR exploration
Prof. Alessandro De Luca
Dipartimento di Ingegneria informatica, automatica e gestionale Antonio Ruberti (DIAG)
Sapienza Università di Roma
Via Ariosto 25, 00185 Roma, Italy
tel: +39 06 77274 052
email: deluca [at] diag [dot] uniroma1 [dot] it
Academic year 2016/17 (first semester)
Aims: This module presents locomotion and haptic interfaces for VR exploration, giving special emphasis to control issues. It introduces in particular two haptic devices (Geomagic Touch and Cyberith Virtualizer/Oculus Rift) available in the DIAG Robotics Laboratory, with some applications. It summarizes also the research results obtained by our Robotics group (see here) within the European project Cyberwalk.
Detailed contents: General introduction to haptic and locomotion interfaces with several illustrative examples. Two specific hardware devices: the Geomatic Touch haptic interface; the Cyberith Virtualizer locomotion interface (with the Oculus Rift HMD). Possible applications of these interfaces. Design, construction, actuation, sensing, modelling, and system issues for two locomotion platforms for VR exploration, developed within the CyberWalk project: the CyberCarpet (ball-array) and the CyberWalk platform (2D, omni-directional). Control design and experimental validation for the CyberCarpet. Control design and experimental validation for a 1D treadmill. Control design, experimental validation, and perceptual evaluation for the 2D CyberWalk platform.
Prerequisites: This module has no strict prerequisites. However, it is suggested to have acquaintance with the basic topics of Robotics 1 and Robotics 2, or with those treated in Autonomous and Mobile Robotics.
ECTS credits for this module: 3 credits (out of 12 credits for the whole course).
Lectures for this module
Period: first part of first semester (end of September-early November 2016; 4 to 6 weeks)
Begin: Wednesday, October 5, 2016, 17:30-19:15, in room A5 (as a one-time, extra slot)
End: Monday, November 7, 2016
|Monday||15:45-17:15 (room A7; DIAG, Via Ariosto 25)|
|Friday||15:45-17:15 (room A2; DIAG, Via Ariosto 25)|
Specials: NO class on Monday, October 10; Lab visit (room S-009, in basement) on Friday, October 15
Distribution of students according to the course of study:
A.A. 2016/17: 19 MARR (4 international), 2 MCER (2 international), 1 Erasmus (international); Total: 22 students (21 present at first lecture)
A.A. 2015/16: 8 MARR, 1 MCER, 1 PhD ABRO; Total: 10 students (10 present at first lecture)
A.A. 2014/15: 9 MARR; Total: 9 students (8 present at first lecture)
A.A. 2013/14: module not offered
A.A. 2012/13: 19 MARR (7 international), 1 Erasmus (international), 1 PhD student; Total: 21 students (21 present at first lecture)
A.A. 2011/12: 17 MARR (4 international); Total: 17 students (14 present at first lecture)
A.A. 2010/11: 14 MARR (5 international), 5 MELR, 2 MSIR; Total: 21 students (15 present at first lecture)
Teaching material (PDF of the lecture slides, videos, technical papers, etc.) is available hereafter:
#1 Haptic & LocomotionInterfaces_Intro.pdf (63 slides; last update on April 4, 2016)
19 videos to slides #1 (same order as in the slides)
- ImmersiveTouch (.wmv, 35MB) - Immersive Touch
- Telemanipulation_Robocast (.mpg, 8.7MB) - SIRSLab, University of Siena (D. Prattichizzo)
- HapticAugmentedReality_RoMan10 (.mp4, 18.8MB) - SIRSLab, University of Siena (S. Scheggi, G. Salvietti, D. Prattichizzo)
- Bilateral_Teleoperation_of_Multiple_UAVs_with_Decentralized_Connectivity_Maintenance (.mp4, 10.9MB) - Max Planck Inst. Biological Cybernetics, Tübingen (P. Robuffo Giordano)
- TeleKyb_Framework (.mp4, 7.1MB) - Max Planck Inst. Biological Cybernetics, Tübingen (A. Franchi)
- Berkeley Bionics Human Exoskeleton, available on YouTube
- ICRA10_KUKATitan_Ferrari (.mp4, 15.7MB) - Max Planck Inst. Biological Cybernetics, Tübingen (P. Robuffo Giordano)
- SarcosTreadport (.mp4, 3.6MB) on KSL 5 TV; University of Utah (J. Hollerbach)
- TorusTreadmill (.mp4, 280KB) - VR Lab, University of Tsukuba (H. Iwata)
- ODT_May05 (.wmv, 840KB) - Virtual Space Devices (D. Carmein)
- ODT_May06 (.wmv, 520KB) - Virtual Space Devices (D. Carmein)
- VS_Presentation (.mpeg, 50.8MB) - Virtusphere
- Cyberith Virtualizer Demonstration (.mov, 10.1MB) - edited from Cyberith.com
- Cyberith Virtualizer and Virtuix Omni platforms (.avi, 13.5MB) - edited from Cyberith.com and Virtuix.com
- VR2005_BAT_Simulation (.avi, 26.6MB) - Kogakuin University
- VR2005_BAT Experiment (.avi, 23MB) - Kogakuin University
- CirculaFloor_Siggraph04 (.mp4, 8.8MB) - VR Lab, University of Tsukuba (H. Iwata)
- CirculaFloor (.mp4, 8.8MB) - VR Lab, University of Tsukuba (H. Iwata)
- PoweredShoes_Siggraph06 (.mpeg, 30.8MB) - VR Lab, University of Tsukuba (H. Iwata)
#2 MotionControl_CyberwalkPlatforms_PartI.pdf (48 slides; last update on April 4, 2016)
15 videos to slides #2 (same order as in the slides)
- smoot_feedfw_datiETH (.avi, 2MB) [video repeated also on slide 27]
- MVI_7458_good (.avi, 2.4MB)
- MVI_7477_increasingspeed (.avi, 13MB)
- walker_still_type1 (.avi, 9.5MB)
- experiment5 (.mpeg, 6.5MB)
- experiment4 (.mpeg, 6.6MB)
- walker_still (.avi, 2.7MB)
- walker_straight_line_no_ff (.avi, 5.1MB)
- walker_straight_line_ff (.avi, 4.1MB)
- walker_circ_path_no_ff (.avi, 7.8MB)
- walker_circ_path_ff (.avi, 8.6MB)
- walker_square_path_no_ff (.avi, 6.5MB)
- walker_square_path_ff (.avi, 8.5MB)
- ICRA07_CyberCarpet_singleview (.avi, 1.7MB)
- quadrato_completo_umanoide (.avi, 5.3MB)
#3 MotionControl_CyberwalkPlatforms_PartII.pdf (36 slides; last update on April 4, 2016)
9 videos to slides #3 (same order as in the slides)
- CW_Mechanics (.mp4, 2.6MB)
- Movie1 (.mp4, 43.9MB)
- projects_cyberwalk_preintegration.01.edited (.avi, 26.9MB)
- video2 (.avi, 1.1MB)
- Cyberwalk-o_ton (.mov, 5.6MB)
- MVI_7625 (.avi, 26MB)
- CW_gain_orientation (.avi, 0.9MB)
- CW_gain_orientation_Paolo (.avi, 1.8MB)
- CW_IROS09_new (.avi, 4.6MB)
#4 GeomagicTouch.pdf (25 slides; last update on April 4, 2016)
1 video to slides #4 (same order as in the slides)
- SensAble PHANTOM Omni Haptic Device In Action!, available on YouTube
After attending classes, students should either give a presentation with slides on a certain topic (based on technical papers) or develop a small project (in general, involving simulations). Work can be done alone or in groups, typically by two students for presentations and three students for small projects. Presentations should be given in class by the end of December 2016, according to a schedule defined with the assignments. Projects should be completed and presented in class by the end of July 2017.
Note that in order to obtain the 12 credits for Elective in Robotics, it is necessary to complete four modules (each of 3 credits). Up to two modules can be taken from the parallel course of Elective in AI. Altogether, each student should give two (2) presentations and complete two (2) small projects. On request, the two projects of different modules may be merged into a single larger one. The final grade will be assigned based on the average of the single grades (weighted by the credits). For more details, see http://www.diag.uniroma1.it/~lanari/EIR/ElectiveinRobotics.html.
Projects and presentations 2015/16
i) Proposals for 2 small projects (info distributed by mail to the participating groups on May 23, 2016)
- Haptic rendering with the Geomagic Touch
- Kinematic models of the Geomatic Touch
ii) Subjects for 3 presentations (click on titles to download the (zipped) material; online since May 24, 2016)
- A Passive Guidance System for a Robotic Walking Assistant (2.5 MB)
- C. Huang, G. Wasson, M. Alwan, P. Sheth, and A. Ledoux, "Shared navigational control and user intent detection in an intelligent walker" AIAA Fall Symposium, 2005.
- Y. Hirata, A. Hara, and K. Kosuge, "Motion control of passive intelligent walker using servo brakes," IEEE Trans. on Robotics, vol. 23, no. 5, pp. 981-990, 2007.
- C.-H. Ko, K.-Y. Young, Y.-C. Huang, and S.K. Agrawal, "Walk-Assist Robot: A novel approach to gain delection of a braking controller using differential flatness," IEEE Trans. on Control Systems Technology, vol. 21, no. 6, pp. 2299-2305, 2013.
- D. Fontanelli, A. Giannitrapani, L. Palopoli, and D. Prattichizzo, "Unicycle steering by brakes: A passive guidance support for an assistive cart," Proc. 52nd IEEE Conf. on Decision and Control, pp. 2275-2280, 2013.
- D. Fontanelli, A. Giannitrapani, L. Palopoli, and D. Prattichizzo, "A passive guidance system for a robotic walking assistant using brakes," Proc. 54th IEEE Conf. on Decision and Control, pp. 829-834, 2015.
- The Soft Sixth-Finger Device (7.9 MB)
- I. Hussain, L. Meli, C. Pacchierotti, G. Salvietti, and D. Prattichizzo, "Vibrotactile haptic feedback for intuitive control of robotic extra fingers," Proc. 2015 IEEE World Haptics Conf., pp. 394-399, 2015.
- I. Hussain, G. Salvietti, L. Meli, C. Pacchierotti, D. Cioncoloni, S. Rossi, and D. Prattichizzo, "Using the robotic sixth finger and vibrotactile feedback for grasp compensation in chronic stroke patients," Proc. 2015 IEEE Int. Conf. on Rehabilitation Robotics, pp. 67-72, 2015.
- I. Hussain, G. Salvietti, M. Malvezzi, and D. Prattichizzo, "Design guidelines for a wearable robotic extra-finger," 1st Int. Forum on Research and Technologies for Society and Industry, Torino, 16-18 September 2015.
- G. Salvietti, I. Hussain, D. Cioncoloni, S. Taddei, S. Rossi, and D. Prattichizzo, "Compensating hand function in chronic stroke patients through the robotic sixth finger," IEEE Trans. on Neural Systems and Rehabilitation Engineering, DOI:10.1109/TNSRE.2016.2529684, 2016.
- I. Hussain, G. Salvietti, G. Spagnoletti, and D. Prattichizzo, "The Soft-SixthFinger: A wearable EMG controlled robotic extra-finger for grasp compensation in chronic stroke patients," IEEE Robotics and Automation Lett., vol. 1, no. 2, pp. 1000-1006, 2016.
- Hardware and Software Description of the Cyberith Virtualizer and Its Possible Applications (0.7 MB)
Projects and presentations 2014/15
i) Proposals for 3 small projects (click on titles to download the zipped proposal and related material; online since April 2, 2015)
- Analysis and control of 1-dof telemanipulation systems (2.7 MB)
- Evaluation of user experience in human-robot collision tests (6.6 MB)
- The MPI motion emulators (4.5 MB)
ii) Subjects for 4 presentations (click on titles to download the zipped material, if any; online since April 2, 2015)
BLEEX: Berkeley Lower Extremity Exoskeleton (8.3 MB)
- H. Kazerooni, L. Huang, J.L. Racine, and R. Steger "On the control of Berkeley Lower Extremity Exoskeleton (BLEEX)," Proc. IEEE Int. Conf. on Robotics and Automation, pp. 4364-4371, 2005.
- H. Kazerooni, R. Steger, and L. Huang, "Hybrid control of the Berkeley Lower Extremity Exoskeleton," Int. J. of Robotics Research, vol. 25, no. 5-6, pp. 561-573, 2006.
- H. Kazerooni, A. Chu, and R. Steger, "That which does not stabilize, will only make us stronger," Int. J. of Robotics Research, vol. 26, no. 1, pp. 75-89, 2007.
The Body Extender (7.3 MB)
- M. Fontana, R. Vertechy, S. Mareschi, F. Salsedo, and M. Bergamasco, "The Body Extender: A full-body exoskeleton for the transport and handling of heavy loads," Robotics and Automation Mag., vol. 21, no. 4, pp. 34-44, 2014.
- Special Issue of the Robotics and Automation Magazine on "Wearable Robotics," vol. 21, no. 4, December 2014.
Gravity Compensation for Haptic Devices (0.5 MB)
- A. Formaglio, S. Mulatto, and D. Prattichizzo, "Iterative estimation of the end-effector apparent gravity force for 3DoF impedance haptic devices," Proc. European Control Conf., pp. 537 - 542, 2009.
- M.H. Koul, P. Kumar, P.K. Singh, M. Manivannan, and S.K. Saha, "Gravity compensation for PHANToM Omni haptic interface," 1st Joint Int. Conf. on Multibody System Dynamics, Lappeenranta, Finland, May 2010.
- Small omnidirectional platforms: Cyberith Virtualizer and Virtuix Omni
Projects and presentations 2012/13
i) Proposals for 3 small projects (click on titles to download the zipped proposal and related material; online since November 23, 2012)
- Re-centering motion controller for the CyberCarpet considering also walker orientation (3.8 MB)
- Smooth motion controller for the CyberCarpet considering a dead-zone around the center (3.3 MB)
- A new omnidirectional treadmill (Virtual Reality Floor): Modeling and control of the device (29.7 MB)
ii) Materials for 6 presentations (click on titles to download the zipped material; online since November 23, 2012)
How does a CAVE work (3.8 MB)
- C. Cruz-Neira, D.J. Sandin, T.A. DeFanti, R.V. Kenyon, and J.C. Hart, "The CAVE: Audio visual experience automatic virtual environment," Comm. of the ACM, vol. 35, no. 6, pp. 64-72, 1992.
- C. Cruz-Neira, D.J. Sandin, and T.A. DeFanti, "Surround-screen projection-based virtual reality: The design and implementation of the CAVE," Proc. of SIGGRAPH '93, pp. 135-142, 1993.
The Torus treadmill (10.7 MB)
- H. Iwata, "Walking about virtual environments on an infinite floor," IEEE Virtual Reality Conf., 1999.
The Gaitmaster locomotion interface (20.9 MB)
- H. Yano, H. Noma, H. Iwata, and T. Miyasato, "Shared walk environment using locomotion interfaces," Proc. of ACM Conf. on Computer Supported Cooperative Work (CSCW'00), pp. 163-170, 2000.
- H. Yano, K. Kasai, H. Saitou, and H. Iwata, "Development of a gait rehabilitation system using a locomotion interface," J. of Visualization and Computer Animation, vol. 14, no. 5, pp. 243-252, 2003.
- H. Yano, S. Tamefusa, N, Tanaka, H. Saitou, and H. Iwata, "Gait rehabilitation system for stair climbing and descending," IEEE World Haptics Symp., 2010.
Passivity-based control of haptic interfaces (0.4 MB)
- R.J. Adams and B. Hannaford, "Control law design for haptic interfaces to virtual reality," IEEE Trans. on Control Systems Technology, vol. 10, no. 1, pp. 3-13, 2002.
- B. Hannaford and J.-H. Ryu, "Time-domain passivity control of haptic interfaces," IEEE Trans. on Robotics and Automation, vol. 18, no. 1, pp. 1-10, 2002.
Control schemes for kinesthetic haptic interfaces (11 MB)
- M.-W. Ueberle and M. Buss, "Control of kinesthetic haptic interfaces," TUM Internal Report, 2004.
- M.-W. Ueberle, "Design, Control, and Evaluation of a Family of KinestheticHaptic Interfaces," PhD Thesis, TU M&uunchen, 2006 (mainly Chapter 3).
How to buy a haptic device for use in robotics (11.8 MB)
- M.-W. Ueberle, "Design, Control, and Evaluation of a Family of Kinesthetic Haptic Interfaces," PhD Thesis, M&uunchen, 2006 (mainly Chapter 2).
- E. Samur, "Guidelines for haptic interface evaluation: Physical & psychophysical methods," Presentation at Haptic Systems Work. on Hardware Evaluation, 2012.
Projects and presentations 2011/12
i) Proposals for 5 small projects (click on titles to download the zipped proposal and related material; online since November 21, 2011)
- Bilateral Teleoperation with Delay (0.5 MB)
- Control and 3D Simulation of the Feet Followers Device (3.5 MB)
- CyberCarpet Alternative Controller (52.2 MB)
- Human Hand Tracking using Kinect (58.4 MB, two projects)
ii) Materials for 3 presentations (click on titles to download the zipped material; online since November 21, 2011)
CyberWalk Perceptual Evaluation (5.8 MB)
- J. Souman, P. Robuffo Giordano, M. Schwaiger, I. Frissen, T. Thümmel, H. Ulbrich, A. De Luca, H. Bülthoff, M. Ernst, "CyberWalk: Enabling Unconstrained Omnidirectional Walking Through Virtual Environments," ACM Trans. on Applied Perception, vol. 8, no. 4, pp. 24:1-24:22 (with Appendix), 2011.
- J. Souman, P. Robuffo Giordano, I. Frissen, A. De Luca, M. Ernst, "Making Virtual Walking Real: Perceptual Evaluation of a New Treadmill Control algorithm," ACM Trans. on Applied Perception, vol. 7, no. 2, pp. 11:1-11:14, 2010.
Mobile Haptic Interfaces (3.7 MB)
- A. Formaglio, D. Prattichizzo, F. Barbagli, and A. Giannitrapani, "Dynamic Performance of Mobile Haptic Interfaces," IEEE Trans. on Robotics, vol. 24, no. 3, pp. 2008.
- M. Ueberle, H. Esen, A. Peer, U. Unterhinninghofen, and M. Buss, "Haptic Feedback Systems for Virtual Reality and Telepresence Applications," Harmonic Drive Int. Symp., pp. 99-107, 2006.
- A. Peer, Mobile Manipulation in der Telerobotik (slides presentation).
- N. Nitzsche, U.D. Hanebeck, and G. Schmidt. "Design Issues of Mobile Haptic Interfaces," J. of Robotic Systems (renamed J. of Field Robotics), vol. 20, no. 9, pp. 549-556, 2003.
Treadport II (2 MB)
- J.M. Hollerbach, Y. Xu, R.R. Christensen, and S.C. Jacobsen, "Design Specifications for the Second Generation SARCOS Treadport Locomotion Interface," Haptics Symposium (Proc. of ASME Dynamic Systems and Control Division, DSC-vol. 69-2), pp. 1293-1298, 2000.
- J.M. Hollerbach, R. Mills, D. Tristano, R.R. Christensen, W.B. Thompson, and Y. Xu, "Torso Force Feedback Realistically Simulates Slope on Treadmill-Style Locomotion Interfaces," Int. J. of Robotics Research, vol. 20, pp. 939-952, 2001.
- R.C. Hayward and J.M. Hollerbach, "Implementing Virtual Stairs on Treadmills Using Torso Force Feedback," 2002 IEEE Int. Conf. on Robotics and Automation, pp. 586-591, 2002.
- J.M. Hollerbach, D. Checcacci, H. Noma, Y. Yanagida, and N. Tetsutani, "Simulating Side Slopes on Locomotion Interfaces using Torso Forces," 11th IEEE Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS'03), 2003.
Projects and presentations 2010/11
i) Proposals for 4 small projects (click on titles to download the proposal; on line since November 21, 2010)
- Dynamic effects on walker in the CyberCarpet (with additional material)
- Two simultaneous users on the CyberWalk platform
- Collision avoidance of multi-users using Powered Shoes in a VR room
- Dynamic analysis of the Cybersphere (with additional material)
ii) Materials for 5 presentations (click on titles to download the zipped material; online since November 21, 2010)
- H. Noma and T. Miyasato, "Design for Locomotion Interface in a Large Scale Virtual Environment - ATLAS: ATR Locomotion Interface for Active Self Motion," 7th Ann. Symp. on Haptic Interface for Virtual Environments and Teleoperated Systems, Proc. ASME-DSC, vol. 64, pp. 111-118, 1998.
- H. Noma and T. Miyasato, "A New Approach for Canceling Turning Motion in the Locomotion Interface, ATLAS", Proc. ASME-DSC, vol. 67, pp.405-406, 1999.
- J.M. Hollerbach, D. Checcacci, H. Noma, Y. Yanagida, and N. Tetsutani, "Simulating Side Slopes on Locomotion Interfaces using Torso Forces," 11th Symp. on Haptic Interfaces for Virtual Environment and Teleoperated Systems (HAPTICS'03), 2003.
- M. Schwaiger, H. Ulbrich, and T. Thümmel, "A Foot Following Locomotion Device with Force Feedback Capabilities," VIII Symp. on Virtual Reality, pp. 309-321, 2006.
- J. Yoon, J. Park. and J. Ryu, "Walking Control of a Dual-Planar Parallel Robot for Omni-directional Locomotion Interface," 2005 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS'05), pp. 1151-1156, 2005.
- J.-Y. Huang, "An Omnidirectional Stroll-Based Virtual Reality Interface and Its Application on Overhead Crane Training," IEEE Trans. on Multimedia, vol. 5, no. 1, pp. 39-51, 2003.
- J.-Y. Huang, W.-H. Chiu, Y.-T. Lin, M.-T. Tsai, H.-H. Bai, C.-F. Tai, C.-Y. Gau, and H.-T. Lee, "The Gait Sensing Disc - A Compact Locomotion Device for the Virtual Environment," 8th Int. Conf. on Computer Graphics, Visualization, and Computer Vision (WSCG'00), pp. 290-297, 2000.
- J.-Y. Huang, K.-P. Shih, Y.-T. Lin, C.-F. Tai, and C.-Y. Gau, "A Design of the Gait Sensing Algorithm for the Locomotion-based Virtual Reality System," Int. Computer Symp., 2006.
- H. Iwata, H. Yano, and M. Tomiyoshi, "String Walker," SIGGRAPH'07, 2007.
- "String Walker," VR Lab wep page, University of Tsukuba.
- Video: Introductory movie for SIGGRAPH'07 (.mpeg, 25.1MB)
- Z. Wang, K. Bauernfeind, and T. Sugar, "Omni-Directional Treadmill System," 11th Symp. on Haptic Interfaces for Virtual Environment and Teleoperated Systems (HAPTICS'03), 2003.
- Video: Omnidirectional Treadmill_Virtual Reality Floor (.fvl, 14.7MB)
- Video: Omnidirectional Treadmill_Virtual Reality Floor Update.flv (.fvl, 14.7MB)
Please check the information in the main page of the Elective in Robotics course. In order to get the final grade (for 12 credits) you need to register only once, when you have acquired the credits of all modules.