Visual perception of surface properties through manipulation

Vision is a component of a perceptual system whose function is to support purposeful behavior. In this project we studied the perceptual system that supports the visual perception of surface properties through manipulation. Observers were tasked with finding dents in simulated flat glossy surfaces. The surfaces were presented on a tangible display system implemented on an Apple iPad, that rendered the surfaces in real time and allowed observers to directly interact with them by tilting and rotating the device. On each trial we recorded the angular deviations indicated by the device's accelerometer and the images seen by the observer. The data reveal purposeful patterns of manipulation that serve the task by producing images that highlight the dent features. These investigations suggest the presence of an active visuo-motor perceptual system involved in the perception of surface properties, and provide a novel method for its study using tangible display systems

Measurement of Forest Inventory Parameters with Apple iPad Pro and Integrated LiDAR Technology

The estimation of single tree and complete stand information is one of the central tasks of forest inventory. In recent years, automatic algorithms have been successfully developed for the detection and measurement of trees with laser scanning technology. Nevertheless, most of the forest inventories are nowadays carried out with manual tree measurements using traditional instruments. This is due to the high investment costs for modern laser scanner equipment and, in particular, the time-consuming and incomplete nature of data acquisition with stationary terrestrial laser scanners. Traditionally, forest inventory data are collected through manual surveys with calipers or tapes. Practically, this is both labor and time-consuming. In 2020, Apple implemented a Light Detection and Ranging (LiDAR) sensor in the new Apple iPad Pro (4th Gen) and iPhone Pro 12. Since then, access to LiDAR-generated 3D point clouds has become possible with consumer-level devices. In this study, an Apple iPad Pro was tested to produce 3D point clouds, and its performance was compared with a personal laser scanning (PLS) approach to estimate individual tree parameters in different forest types and structures. Reference data were obtained by traditional measurements on 21 circular forest inventory sample plots with a 7 m radius. The tree mapping with the iPad showed a detection rate of 97.3% compared to 99.5% with the PLS scans for trees with a lower diameter at a breast height (dbh) threshold of 10 cm. The root mean square error (RMSE) of the best dbh measurement out of five different dbh modeling approaches was 3.13 cm with the iPad and 1.59 cm with PLS. The data acquisition time with the iPad was approximately 7.51 min per sample plot; this is twice as long as that with PLS but 2.5 times shorter than that with traditional forest inventory equipment. In conclusion, the proposed forest inventory with the iPad is generally feasible and achieves accurate and precise stem counts and dbh measurements with efficient labor effort compared to traditional approaches. Along with future technological developments, it is expected that other consumer-level handheld devices with integrated laser scanners will also be developed beyond the iPad, which will serve as an accurate and cost-efficient alternative solution to the approved but relatively expensive TLS and PLS systems. Such a development would be mandatory to broadly establish digital technology and fully automated routines in forest inventory practice. Finally, high-level progress is generally expected for the broader scientific community in forest ecosystem monitoring, as the collection of highly precise 3D point cloud data is no longer hindered by financial burdens.

Project iPad: Investigating Tablet Integration in Learning and Libraries at Ryerson University

The year 2010 saw a major revolution in tablet technology with the introduction of the Apple iPad. Curious about the potential of this new technology for libraries, a group of librarians at Ryerson University in Toronto seized an opportunity to investigate the emerging role of the tablet in the daily academic lives of students.

Project iPad: Investigating Tablet Integration in Learning and Libraries at Ryerson University

The year 2010 saw a major revolution in tablet technology with the introduction of the Apple iPad. Curious about the potential of this new technology for libraries, a group of librarians at Ryerson University in Toronto seized an opportunity to investigate the emerging role of the tablet in the daily academic lives of students.

S3

With smart devices being an essential part of our everyday lives, unsupervised access to the mobile sensors' data can result in a multitude of side-channel attacks. In this paper, we study potential data leaks from Apple Pencil (2nd generation) supported by the Apple iPad Pro, the latest stylus pen which attaches to the iPad body magnetically for charging. We observe that the Pencil's body affects the magnetic readings sensed by the iPad's magnetometer when a user is using the Pencil. Therefore, we ask: Can we infer what a user is writing on the iPad screen with the Apple Pencil, given access to only the iPad's motion sensors' data? To answer this question, we present Side-channel attack on Stylus pencil through Sensors (S3), a system that identifies what a user is writing from motion sensor readings. We first use the sharp fluctuations in the motion sensors' data to determine when a user is writing on the iPad. We then introduce a high-dimensional particle filter to track the location and orientation of the Pencil during usage. Lastly, to guide particles, we build the Pencil's magnetic map serving as a bridge between the measured magnetic data and the Pencil location and orientation. We evaluate S3 with 10 subjects and demonstrate that we correctly identify 93.9%, 96%, 97.9%, and 93.33% of the letters, numbers, shapes, and words by only having access to the motion sensors' data.

Using Muse: Rapid Mobile Assessment of Brain Performance

The advent of mobile electroencephalography (mEEG) has created a means for large scale collection of neural data thus affording a deeper insight into cognitive phenomena such as cognitive fatigue. Cognitive fatigue – a neural state that is associated with an increased incidence of errorful performance – is responsible for accidents on a daily basis which at times can cost human lives. To gain better insight into the neural signature of cognitive fatigue in the present study we used mEEG to examine the relationship between perceived cognitive fatigue and human-event related brain potentials (ERPs) and electroencephalographic (EEG) oscillations in a sample of 1,000 people. As a secondary goal, we wanted to further demonstrate the capability of mEEG to accurately measure ERP and EEG data. To accomplish these goals, participants performed a standard visual oddball task on an Apple iPad while EEG data were recorded from a Muse EEG headband. Counter to traditional EEG studies, experimental setup and data collection was completed in less than seven minutes on average. An analysis of our EEG data revealed robust N200 and P300 ERP components and neural oscillations in the delta, theta, alpha, and beta bands. In line with previous findings we observed correlations between ERP components and EEG power and perceived cognitive fatigue. Further, we demonstrate here that a linear combination of ERP and EEG features is a significantly better predictor of perceived cognitive fatigue than any ERP or EEG feature on its own. In sum, our results provide validation of mEEG as a viable tool for research and provide further insight into the impact of cognitive fatigue on the human brain.