Advanced smartphone features entice users who want more from their devices, particularly in the health and entertainment sectors, but might these functions pose a security concern while making or receiving genuine phone calls? To answer that question, a group of academic researchers from Texas A&M University and four other universities developed harmful software, or malware.
EarSpy, the virus developed by the researchers, employed machine learning methods to extract a startling amount of caller information from ear speaker vibration data gathered by an Android smartphone’s own motion sensors—all without circumventing any safeguards or requiring user permissions.
“A standard cell phone attack taps the microphone and records the voices,” said Ahmed Tanvir Mahdad, a doctorate student in Texas A&M’s Department of Computer Science and Engineering. “We’re recording motion sensor data that isn’t directly related to speech and using it to detect caller information in a side-channel attack.”
Mahdad was the lead author of “EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers,” a paper that outlined the project’s findings and was released in December 2022 on the pre-print service arXiv.
Traditionally, the ear speakers on the top of cellphones are small and create modest sound pressure levels during talks. When the phone is put to the user’s ear, the vibrations improve clarity.
Because of their size and operation, the speakers are not regarded a good source of audible eavesdropping. However, some manufacturers are replacing these little speakers with larger ones in order to produce the stereo sounds required for films and streaming, without taking into account how much vibration data the larger ear speakers output. Because smartphones are equipped with motion sensors known as accelerometers that record vibration data while tracking user movements and locations, ear speaker vibrations can also be recorded and possibly hacked.
The researchers chose two current smartphones that were identical in style, ran Android, and had loud ear speakers. They only played recorded voices through ear speakers at a volume that was comfortable for the user’s hearing. The researchers then utilized EarSpy to examine the accelerometer data from the phones.
Thello discovered EarSpy had 91.6% accuracy in determining whether the speaker was a repeat caller and 98.6% accuracy in determining the speaker’s gender. The malware also detected spoken digits with 56% accuracy, which is five times better than a random guess.
“Say you’re speaking with a customer service representative from a health care provider or a bank, and they ask for your identification or credit card numbers,” Mahdad explained. “If the EarSpy malware was on your phone, the attacker could access your phone’s accelerometer data and pull it from your phone through an internet connection for processing so that they can extract this information.”
The study concentrated on Android cellphones since motion sensor data can be collected from them without the user’s explicit authorization.
Previous research revealed that extracting voice parameters from accelerometer data generated by tiny ear speakers on previous Android smartphones was problematic. The researchers used two newer phones with larger speakers that provided progressively more information; the program was able to distinguish 45-90% of the word areas from their accelerometer data for further study. Moving the accelerometer to a different location in the phone may lessen the amount of data gathered, but it would not halt the recordings entirely, according to the researchers.
Future studies on additional phones may be necessary, as the findings suggest that all smartphone manufacturers should be aware of the security dangers.
According to Mahdad, the hack could only occur if the attacker disguised malware in an application that the user downloaded.
“Any benign-looking app with malware can extract this information, but only if the user approves the app,” Mahdad explained. “Once installed, it could run in the background without notifying the user.”