Hi, my name is Patrick Carlson and I'm a PhD graduate in Human-Computer Interaction (HCI) from Iowa State University. My major adviser was Dr. Judy M. Vance. My undergraduate education was at Simpson College where I received a Bachelor of Arts, cum laude, with a double major in Computer Science and Psychology. I work at Sandia National Labs as a Data Scientist.


Many of my interests revolve around combining Computer Science and Psychology.

  • Human-Computer Interaction
  • Datamining
  • Experimental Design
  • User Studies
  • Distributed Software Development
  • Statistics and Data Analysis
  • Linux and Open Source Software
  • Web Development
  • Computer Supported Collaboration
  • Social Network Analysis
  • Usability
  • Interface Testing
  • Artificial Intelligence


  • Following Open Source, web development, badminton, playing piano


Improving Open Source Software Development: Harnessing Social and Technical Datamining

My PhD dissertation topic involved datamining Mozilla and KDE, two large Open Source communities. Through analysis of Git version control history, email mailing lists, and source code files, this parsed information was stored in the graph database Neo4j. Using Social Technical Congruence (STC) as a core starting point, three algorithms were developed to help developers better understand the social and technical structure of software development. The field of Social Technical Congruence (Cataldo et al. 2006) attempts to understand and model the synergies between technical development and communication. It is able to identify gaps in the communication structure based on shared technical dependencies of developers to help minimize future bugs and keep up with changes. Three novel algorithms were developed to help developers:

  • Impactful Commits: This algorithm shows you code commits that have had a substantial impact to the technical dependency structure of the source code.
  • Developer Knowledge: This algorithm allows you to select a source code file to determine who has knowledge or expertise in the file. This is helpful if you have a question about a particular file or section but are unsure who to contact for help.
  • Communication Suggestions: This algorithm provides suggestions about who you (or other developers) should be increasing communication with on the mailing list based on work on shared source code.

The three algorithms were implemented in a program and website called Jamii. The program provides all backend processing of data from source code version control and email mailing lists as well as visualization. The Jamii website was then tested against the same two communities. Two surveys examining Mozilla and KDE were performed that gathered qualitative and quantitative data. The first survey gathered demographic and perceptual differences in socialization between beginner and veteran members. The second survey investigated the perceived usefulness of the website as well as the poential for improving the socialization process.

  • Carlson, Patrick. "Engaging developers in open source software projects: harnessing social and technical data mining to improve software development" (2015). Graduate Theses and Dissertations. Paper 14663.

Using Bimanual Haptics in Virtual Assembly Tasks

I've worked with Dr. Judy Vance and other graduate students on research using various configurations of bimanual haptic devices to examine their effectiveness in virtual assembly tasks. One of the haptic devices we use is the Geomagic Touch (formerly Sensable Phantom Omni). This research aims to improve the training of assembly tasks through the utilization of haptic devices, virtual reality, and accurate physics calculations. We've also examined how much learning transfer takes place in using this technology for virtual training. You can find more details, images, and videos here.


  • Patrick Carlson, Judy M. Vance, and Meisha Berg, An evaluation of asymmetric interfaces for bimanual virtual assembly with haptics, Virtual Reality, pages 1-9.
  • Patrick Carlson, Anicia Peters, Stephen B. Gilbert, Judy M. Vance, and Andy Luse, Virtual Training: Learning Transfer of Assembly Tasks, IEEE Transactions on Visualization and Computer Graphics, pages 770-782, Vol 21, Issue 6.
  • Mike Oren, Patrick Carlson, Stephen Gilbert, and Judy M. Vance, Puzzle Assembly Training: Real World vs. Virtual Environment, In Proceedings of the IEEE 2012 Virtual Reality Conference, pages 1–4, Orange County, California, IEEE.
  • Judy M. Vance, Stephen B. Gilbert, Michael Oren, Ryan Pavlik, and Patrick Carlson, GOALI: A Hybrid Method to Support Natural Interaction of Parts in a Virtual Environment, In NSF Engineering Research and Innovation Conference Proceedings, pages 1–4, Atlanta, Georgia.

Virtual Shopping Study

Over the summer of 2010, I coordinated and co-advised three undergraduate Research Experience for Undergraduates (REU) students in a project to create a physical mockup of a shopping cart. This hardware was then used in a virtual reality user study utilizing the C6 at ISU. We have since updated the hardware to use an Arduino and a linear encoder for user control. You can find more details, images, and videos here.


Undergraduate Psychology Capstone - Examining The Linux Desktop

My undergraduate Psychology capstone involved two semesters of work. The first semester was spent creating an experiment, gathering the required materials, and making sure the design was solid. The second semester I implemented the experiment and analyzed the data. The experiment focused on differences in usability and appeal between a 2D versus 3D Linux desktop. The hypothesis was that the 3D desktop would be more appealing as well as more usable. The data collection involved participants taking surveys about their prior technology experience and desktop usage and then performing a variety of tasks while being timed. Additional surveys were used to examine their thoughts on the differences between the two desktops. No usability difference was found between the 2D and 3D desktops, however, participants thought the 3D desktop was more visually appealing. The interesting result was that participants preferred the 3D desktop over the 2D desktop but only if they experienced the 2D desktop first. I was able to explain this result using cognitive load theory from my cognitive Psychology class. I presented a poster of my results at the Midwest Psychology Association conference in May of 2008.

Paper: Usability and Appeal of a 2D versus 3D Linux Operating System (.pdf) - 2007




If you can't find a paper and would like a PDF, please contact me.


Kaggle Data Science Competitions

Recently I've been having a lot of fun with Kaggle and playing with the large datasets available via the competitions. Kaggle is a website that hosts data science and data analytics competitions from companies and organizations around the world who sponsor and provide the data and challenge. The goal usually is to work with a training set of known data and then generate a prediction for a testing set. My Kaggle profile shows the competitions that I've participated in. Most of the programming to extract features and run machine learning algorithms is done using Python and libraries such as Pandas, scikit-learn, numpy, and SciPy.

OpenCV: Object Tracking

This was an individual homework assignment for the HCI 575: Computational Perception class. OpenCV (Open Source Computer Vision Library) is an extremely popular library for image analysis and manipulation. The homework assignment was to write code to track very small worms in various video clips. This was a real problem that was provided to the professor by graduate students in Biology at Iowa State. The source code can be found here on Github. The video below shows the original video on the left and the tracked worms and red "breadcrumb" trails on the right.

Nvidia CUDA: Partial Implementation of JSEG Image Segmentation

This was an individual project for the ComS 610: seminar class (Nvidia CUDA Programming). CUDA is Nvidia's library for doing high speed parallel calculations on graphics processing units (GPUs). Image segmentation is the process of simplifying an image to object boundaries. The source code can be found here on Github. An example of an original image and segmented image can be seen below.

Distributed Software Development: Android Application

This was a semester long group project for ComS 510X (Distributed Software Development). The project spanned three different countries and universities. Students from Iowa State University in the US, Jilin University in China, and the Federal Universidade da Bahia in Brazil participated. The project was to create an Android based facial recognition program for storing names and contact information. At Iowa State, we worked on the front-end Android application which interfaced with the backend facial recognition systems in China and Brazil. This project demonstrated the benefits and challenges of geographically distributed software development.


Web Development Design and Evaluation

As a group project for HCI 573X (User Interface Implementation for Web Applications), we developed a website called "Byen" that parses Git repositories and helps developers understand historical changes to files through charts and graphs. Byen means city/community in Norwegian. From the technical side, the class focused on teaching web languages and technologies such as HTML, PHP, Javascript, and MySQL. Using these, we built the website and then ran some usability tests. Tasks were developed and provided to users such as creating and analyzing new repositories and finding which files have been changed frequently. Low fidelity and high fidelity prototypes were tested and time on task as well as task success were measured. This work was helpful in providing ideas for my subsequent thesis topic.

ThinkSpace Redesign

ThinkSpace is an online collaboration and instructional system for students and teachers developed at Iowa State University. As part of a group project for HCI 596 (Emerging Practices in Human-Computer Interaction), we spent the semester getting an understanding of the users and how the navigation of the website could be improved. Low fidelity prototypes were mocked up on paper and tested in class. Medium fidelity prototypes were developed using the software Balsamiq. Evaluations were done with users (both teachers and students) along the way using UserZoom and Just in Mind software. Users were evaluated on a series of common tasks which were timed as well as measured for errors. The System Usability Scale (SUS) and other questions were gathered using Qualtrics. A final higher fidelity prototype was created with Omni Graffle and presented to the developers and management of ThinkSpace. Our group focused on improvements to the tabbed navigation to ensure page consistency.

Virtual Reality Software

While working with Dr. Judy Vance and other graduate students, I have contributed to the following Vance Group VR software projects:

Low-Cost Virtual Reality for Immersive Computing Environments

As VR and gaming hardware advances, various off-the-shelf hardware can be used as inexpensive tracking and interaction devices. The Razer Hydra uses magnetic tracking to identify the position and orientation of the controllers. By taking one controller apart and attaching it to a pair of glasses, a user can inexpensively create a head-tracked viewpoint and an interaction wand. The Kinect from Microsoft combines an infrared (IR) depth camera and RGB camera to create a 3D view of the user. This information can then be processed to identify the skeletal and head position and orientation. Lastly, the Nintendo Wii Remote (Wiimote) can be used with a pair of LED glasses to track the user's head position and orientation. This work combines a wide variety of software developed by graduate and undergraduate students in Dr. Judy Vance's lab. You can find more details here. This hardware and software was demonstrated at IDETC-CIE 2013.

Mathematical Contest in Modeling (MCM)

The last three years as an undergraduate, I participated in the annual Mathematical Contest in Modeling competition. Teams of three members chose between three different problems and modeled a solution for the problem in only four days. All three years the team I was on achieved honorable mention. The following is the problem we picked for 2007 and the paper we submitted.

PROBLEM B: The Airplane Seating Problem

"Airlines are free to seat passengers waiting to board an aircraft in any order whatsoever. It has become customary to seat passengers with special needs first, followed by first-class passengers (who sit at the front of the plane). Then coach and business-class passengers are seated by groups of rows, beginning with the row at the back of the plane and proceeding forward. Apart from consideration of the passengers' wait time, from the airline's point of view, time is money, and boarding time is best minimized. The plane makes money for the airline only when it is in motion, and long boarding times limit the number of trips that a plane can make in a day. The development of larger planes, such as the Airbus A380 (800 passengers), accentuate the problem of minimizing boarding (and deboarding) time. Devise and compare procedures for boarding and deboarding planes with varying numbers of passengers: small (85-210), midsize (210-330), and large (450-800). Prepare an executive summary, not to exceed two single-spaced pages, in which you set out your conclusions to an audience of airline executives, gate agents, and flight crews. Note: The 2 page executive summary is to be included IN ADDITION to the reports required by the contest guidelines. An article appeared in the NY Times Nov 14, 2006 addressing procedures currently being followed and the importance to the airline of finding better solutions." The article can be seen here.

"Too Much Personal Baggage" And Other Variables Affecting Airplane Boarding Times (.pdf) - MCM 2007

Research Experience for Undergraduates (REU)

I spent two summers as an undergraduate researching machine learning in New Mexico at the University of New Mexico under the direction of Dr. Terran Lane. The first summer I learned the basics of machine learning and made some simple programs to make sure I understood reinforcement learning (Q-Learning). One involved creating a program that had an agent traversing a grid world in order to get to a destination. My first summer I attended the Association for the Advancement of Artificial Intelligence (AAAI) conference in Boston.

After attending the conference, I chose a research problem to focus on. I was interested in networking technology and found a paper by Justin Boyan and Michael Littman entitled, "Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach", explaining how machine learning could be used to optimize packet routing in a computer network. After programming and doing research the subsequent summer, I was able to replicate their results. I attended the AAAI conference the second summer, this time in Vancouver.

The file below is a presentation I gave at Simpson college regarding the work.

REU Presentation: Can the Internet learn? (.pdf) - 2007

Graduate School Coursework

Fall 2013

  • ComS 610: Seminar (Advanced Topics in Repository Analysis and Code Search)

Spring 2012

  • HCI 573X: User Interface Implementation for Web Applications
  • HCI 596: Emerging Practices in Human-Computer Interaction

Fall 2011

  • ComS 510X: Distributed Software Development
  • Psych 516: Advanced Cognition

Spring 2011

  • Stat 401: Statistics Methods for Researchers

Fall 2010

  • ComS 610: Seminar (Nvidia CUDA Programming)
  • HCI 591: Seminar in Human-Computer Interaction

Spring 2010

  • HCI 575: Computational Perception
  • HCI 591: Seminar in Human-Computer Interaction

Fall 2009

  • HCI 521: Cognitive Psychology of Human-Computer Interaction
  • HCI 591: Seminar in Human-Computer Interaction
  • ComS/ME 557: Computer Graphics and Geometric Modeling

Spring 2009

  • Econ 308: Agent-Based Computational Economics
  • HCI 591: Seminar in Human-Computer Interaction
  • JLMC 598P: Communication Technology-Philosophy and Ethics

Fall 2008

  • ComS 587: Principles of Network and Distributed Programming
  • ComS 592X: Research Colloquia

If you would like a copy of my unofficial undergraduate or graduate transcript, please contact me.