Luca Zappella, Ph.D.







Center for Imaging Science

Johns Hopkins University



+1 410 516 6736




324B Clark Hall

Johns Hopkins University

3400 North Charles Street

Baltimore, MD, 21218, US



zappella at cis dot jhu dot edu






Short Bio

I achieved my masters degree in computer science in Italy, at the University of Milan at the end of 2003. For three years I worked, first in a company then freelance (mainly in computer vision fields). I received the VIBOT masters degree (European Master in Vision and Robotics held in Edinburgh -UK-, Girona -Spain- and Le Creusot -France-) in 2008. In June 2011 I received the Ph.D. at the University of Girona -Spain- in the Computer Vision and Robotics Group with a thesis on motion segmentation and 3D structure from motion, under the supervision of Dr. Xavier Lladó and Dr. Joaquim Salvi.

From 2011 to 2013 I was a post-doctoral researcher in the VisionLab -Center for Imaging Science- of the Johns Hopkins University, under the supervision of Dr. René Vidal. Currently, I am working in a research team at Apple.


Research Interests and My Code

    I am interested in everything related with computer vision and image processing. Specifically, at the moment I am actively researching in the following fields.

  • Gestures Segmentation and Recognition (Language of Surgery)
  • The main project I am currently working on can be described as "gestures segmentation and recognition applied to Robotic Minimal Invasive Surgery (RMIS)". RMIS has been shown to have many benefits but also to have a very steep learning curve that limits its efficacy. The goal of our project is to develop objective and precise skill evaluation algorithms that can help trainees to improve their RMIS skills. Given the video data of a surgery, and possibly the kinematic data of the robot arms, the goal is to temporally segment the different actions performed by the surgeon, recognize each action, and assess the quality of the execution. In order to tackle this problem we take advantage of Hidden Markov Models, Conditional Random Fields, Linear Dynamical Systems and Multiple Kernel Learning techniques.

  • Motion Segmentation and Structure from Motion
    • Joint Estimation of Segmentation and Structure from Motion (JESS) [CVIU 2013]

      JESS is a novel optimisation framework for the estimation of the multi-body motion segmentation and 3D reconstruction of a set of image trajectories in the presence of missing data. The proposed solution not only assigns the trajectories to the correct motion but it also solves for the 3D location of multi-body shape and fills the missing entries in the measurement matrix. Such a solution is based on two fundamental principles: each of the multi-body motions is controlled by a set of metric constraints that are given by the specific camera model, and the shape matrix that describes the multi-body 3D shape is generally sparse. We jointly include such constraints in a unique optimisation framework which iteratively enforces these set of constraints in three stages. First, metric constraints are used to estimate the 3D metric shape and to fill the missing entries according to an orthographic camera model. Then, wrongly segmented trajectories are detected by using sparse optimisation of the shape matrix. A final reclassification strategy assigns the detected points to the right motion or discards them as outliers.

      Matlab code of JESS is publicly available:

      JESS_v0.1 Matlab code. Download also BALM and SpaRSA algorithms and try JESS.

      bibtex pdf
    • Adaptive Subspace Affinity (ASA, aka PAC and SCbA) [ACCV 2010]

      This motion segmentation algorithm is a manifold clustering-based technique. The two most important steps are: the new rank estimation of the trajectory matrix (PAC) and the new similarity measure adopted (SCbA). Both these two techniques take into account the trend of the principal angles when the rank estimation of the trajectory matrix changes. The rank estimation is performed by analysing which rank leads to a configuration where small and large angles are best separated. The affinity measure is a new function automatically parametrised so that it is able to adapt to the actual configuration of the principal angles.

      Matlab code of PAC and SCbA motion segmentation algorithm is publicly available:

      ASA_v1.2 Matlab code. Download also the Hopkins155 database and try PAC and SCbA.

      bibtex pdf
    • Enhanced Local Subspace Affinity (ELSA) [PR 2011]

      ELSA is a new feature-based motion segmentation technique. Unlike LSA, ELSA is robust in a variety of conditions even without manual tuning of its parameters. This result is achieved thanks to two improvements. The first is the EMS+ technique for the estimation of the trajectory matrix rank. The second is an estimation of the number of motions based on the analysis of the eigenvalue spectrum of the Symmetric Normalized Laplacian matrix.

      Matlab code of ELSA motion segmentation algorithm is publicly available:

      ELSA v0.1 Matlab code. Download also the Hopkins155 database and try ELSA. Here there are some plots that can help to understand how and why ELSA works: ELSA trends

      bibtex pdf
    • Enhanced Model Selection (EMS) [ICIP 2009]

      EMS is a novel rank estimation technique for trajectory matrices that can be used within the Local Subspace Affinity (LSA) framework. EMS is based on the relationship between the rank estimated by a model selection technique and the affinity matrix built with LSA. The result is a more robust and precise model selection by which it is possible to automate LSA without requiring any a priori knowledge (about the kind of motion) and to improve the final segmentation.

      Matlab code of LSA with EMS rank estimation is publicly available:

      EMS v0.1 Matlab code. Download also the Hopkins155 database and try EMS. Our implementation of LSA is derived from LSA implementation by Tron and Vidal.

      bibtex pdf






  • [MedIA 2013]
    L. Zappella*, B. Béjar*, G. Hager, and R. Vidal.
    * Equal Contribution
    Surgical Gesture Cassification from Video and Kinematic Data.
    Medical Image Analysis, 2013.
    bibtex pdf
  • [CVIU 2013]
    L. Zappella, A. Del Bue, X. Lladó, J. Salvi.
    Joint Estimation of Segmentation and Structure from Motion.
    Computer Vision and Image Understanding, Vol. 117, Issue 2, Pages 113-129, 2013.
    bibtex pdf JESS v0.1 Matlab code
  • [PR 2011]
    L. Zappella, X. Lladó, E. Provenzi, J. Salvi.
    Enhanced Local Subspace Affinity for feature-based motion segmentation.
    Pattern Recognition, Vol. 44, Pages 454-470, 2011.
    bibtex pdf ELSA v0.1 Matlab code
  • [EL 2009]
    L. Zappella, X. Lladó, J. Salvi.
    Rank Estimation of Trajectory Matrix in Motion Segmentation.
    IET Electronics Letters, Vol. 45, No. 11, Pages 540-541, 21st May 2009.
    bibtex pdf

  Refereed Conferences

  • [IbPRIA 2015]
    Muhammad Habib Mahmood, Luca Zappella, Yago Diez, Joaquim Salvi and Xavier Llado.
    A new trajectory based motion segmentation benchmark dataset (UdG-MS15).
    Iberian Conference on Pattern Recognition and Image Analysis, Santiago de Compostela (Spain), June 17-19, 2015.
  • [MICCAI 2013]
    L. Tao, L. Zappella, G. Hager, and R. Vidal.
    Markov/semi-Markov Conditional Random Field for Gesture Recognition and Temporal Segmentation.
    Medical Image Computing and Computer-Assisted Intervention, Nagoya (Japan), September 22-26, 2013.
  • [ECCV 2012]
    A. Jain, L. Zappella, P. McClure, and R. Vidal.
    Visual Dictionary Learning for Joint Object Categorization and Segmentation.
    European Conference on Computer Vision, Florence (Italy), October 7-13, 2012.
  • [MICCAI 2012] (oral) Best paper award in medical robotics and CAI systems
    B. Béjar*, L. Zappella*, and R. Vidal.
    * Equal Contribution
    Surgical Gesture Classification from Video Data.
    Medical Image Computing and Computer-Assisted Intervention, Nice (France), October 1-5, 2012.
    bibtex pdf video
  • [WMVC 2011]
    L. Zappella, A. Del Bue, X. Lladó, J. Salvi.
    Simultaneous Motion Segmentation and Structure from Motion.
    Proc. of the IEEE International Conference on Motion and Video Computing, Kona (Hawaii, US), January 5-7, 2011.
    bibtex pdf
  • [ACCV 2010]
    L. Zappella, E. Provenzi, X. Lladó, J. Salvi.
    Adaptive Motion Segmentation Algorithm Based on the Principal Angles Configuration.
    The tenth Asian Conference on Computer Vision, Queenstown (New Zealand), November 8-12, 2010.
    bibtex pdf PCA_and_SCbA v1.2 Matlab code
  • [ICIP 2009]
    L. Zappella, X. Lladó, J. Salvi. Enhanced Model Selection For Motion Segmentation.
    IEEE International Conference on Image Processing, Cairo (Egypt), November 7-11, 2009.
    bibtex pdf EMS v0.1 Matlab code
  • [CCIA 2008]
    L. Zappella, X. Lladó, J. Salvi.
    Motion Segmentation: A Review.
    11th International Conference of the Catalan Association for Artificial Intelligence, CCIA'08, Sant Martí d'Empúries (Spain), October 22-24, 2008.


  • [PhD 2011]
    L. Zappella
    Manifold Clustering for Motion Segmentation.
    European Ph.D. Thesis in Technology of the University of Girona (Spain), June 2011.
  • [MSc 2008]
    L. Zappella
    Motion Segmentation From Feature Trajectories.
    VIBOT European Master Thesis: University of Girona (Spain), Heriot-Watt University of Edinburgh (UK), University of Burgundy (France), July 2008.
  • [MSc 2003]
    L. Zappella
    Tecniche di Computer Vision applicate alla video sorveglianza.
    Tesi di Master dell'Università degli Studi di Milano, Dipartimanto di Crema (CR), Italy, December 2003.


  • [LAP 2011]
    L. Zappella, X. Lladó, J. Salvi.
    Motion Segmentation From Tracked Features.
    Lectures in Mathematics.
    LAP LAMBERT Academic Publish. GmbH & Co. KGBirkhauser. ISBn: 978-3-8443-0060-4, 2011.
    bibtex pdf

  Book Chapters

  • [PR 2009]
    L. Zappella, X. Lladó, J. Salvi.
    New Trends in Motion Segmentation.
    Pattern Recognition book. In-TECH, Ed. ISBn: 978-953-307-014-8, pp.31-46, 2009.
    bibtex pdf

  Technical Reports

  • [VIBOT 2008]
    L. Zappella
    Motion Segmentation from Feature Trajectories.
    Proceedings of the 1st VIBOT DAY meeting, Le Creusot (France), September 09, 2008.







 Last updateOctober 2012