Hackers Could Possibly Use Your Computer’s Microphone to “See” Your Screen


We all have heard about the dangers of webcam s and why we should always cover them when they are not in use, but now there is a new attack that can reliably know what’s on your computer’s screen by mere using its microphone.


In what seems like witchcraft, a group of researchers has figured out how to reveal the contents of your computer screen by just listening to “content-dependent acoustic leakage.” they have named the side-channel attack “Synesthesia” and it works are pretty nifty.

Here’s how it works

The side-channel attack leverages what is known as “coil whine,” which is the audio emissions from transformers and other electronic components that power the device’s LCD screen. However, due to how a computer renders a display, with signals being sent to each sub-pixel of a line with varying intensity levels for each sub-pixel, as the computer monitor goes through its refresh scans the power transmitted to each pixel fluctuates.

That fluctuation changes the sounds created by the power supply for the screen, which inadvertently leaks data about the image being refreshed, through the microphone.

If an attacker captures that audio and it is fed into a machine learning trained model, the model can correctly recreate what is on the screen. So just having audio alone won’t cut it anymore. The researchers also applied machine learning to three different types of attacks and demonstrated that a surprising amount of data could be reconstructed.

Examples of accuracy

For example, in one of the attack, they managed to reliably identify (96.5% accuracy) which of the Alexa top 10 websites was on a screen based on audio captured during a Google Hangouts call. Typed keystrokes were also able to be captured in another attack, with a 96.4% accuracy while a device was in portrait orientation. The last attack tried to deduce what text was shown on the remote screen, which again resulted in a scarily high level of accuracy.

“The per-character validation set accuracy containing approximately 10% of our 10,000 trace collection which ranges from 88% to 98%, except for the last character where this time the accuracy was 75%. So out of 100 recordings of test words, for two of them preprocessing returned an error. But for 56 of them, the most likely word on the list was the accurate one. For 72 of them, the correct word appeared in a list of top-five most probable words.”

While the researchers only used a single monitor type in their theory testing, they also demonstrated that a ‘cross-screen’ attack is possible by just calibrating a baseline for an unknown screen type. It’s kind of scary.

With more and more reliance on our mobile devices which sadly already come equipped with a pretty sensitive microphone, I am pretty sure this is not the last we will hear about such an attack. Mitigating it would require re-engineering the display technology we are currently using.

Well for most of us, the risk of this attack is relatively slim. For anyone that works with sensitive data, maybe you should not look at anything while you are making calls with Hangouts.

What do you think about the technology behind this? Though it’s quite scary, do let us know in the comments below.


Please enter your comment!
Please enter your name here