Seeing with Sounds Is Now Possible with Artificial Intelligence!

Meta brings Encodec, a solution that uses AI that can read audio rates noticeably.

AI and machine learning research are currently hot topics in almost every sector of business. Two recent inventions from the company's research laboratories were presented this month by engineers at Meta: an AI system that compresses audio files and an approach that can speed up AI performance for protein folding by 60x. MIT researchers also revealed that they are modeling how a listener might hear a sound from any place in a room to give machines a better sense of their surroundings. Google unveiled Lyra, a neural audio codec designed to compress speech with low bit rates last year. However, Meta asserts that its system is the first to support stereo audio in CD quality, making it suitable for business applications like voice calls.

Meta's Encodec compression solution uses AI to enable real-time compression and decompression of audio at rates ranging from 1.5 kbps to 12 kbps on a single CPU core. In comparison to MP3, Encodec can reach a compression rate of around 10x at 64 kbps without noticeably sacrificing quality. According to the researchers that developed Encodec, human listeners preferred the Encodec-processed audio quality over Lyra-processed audio, indicating that Encodec may someday be utilized to deliver higher-quality audio in circumstances when bandwidth is limited or expensive. The research Meta has done on protein folding has little immediate commercial application. However, it may provide the foundation for significant biological studies down the road. According to Meta, its AI system, ESMFold, predicted the structures of almost 600 million uncharacterized proteins from bacteria, viruses, and other microorganisms.

The Meta system isn't as precise. Only a third of the 600 million proteins it produced was of "excellent quality." However, because it predicts structures 60 times more quickly, it can scale up structure prediction to far bigger protein databases. To avoid drawing undue attention to Meta, the company's AI group also unveiled a system this month that uses mathematics to reason. According to corporate researchers, their "neural issue solver" learned to generalize to new, distinct types of problems from a dataset of successful mathematical proofs. It wasn't Meta who initially created such a system. OpenAI created its own, which it introduced in February and is called Lean. Separately, in the research of symmetry and knots, DeepMind has experimented with systems that can resolve difficult mathematical puzzles.

With regard to MIT's research, researchers there created a machine learning model that can depict how noises in a room will reverberate around the room. The system can learn a room's geometry from sound recordings by modeling the acoustics; this geometry may then be utilized to create visual representations of a room.

The technology, according to the researchers, might be used for robots that must negotiate challenging environments or for virtual and augmented reality applications. They intend to improve the technique in the future so that it can apply to new and larger settings, including complete buildings or even entire villages and cities. The rate at which a quadrupedal robot can learn to walk and perform other stunts is being sped up by two different teams at Berkeley's robotics department. In order to enable a robot to progress from a completely blank slate to robust walking on uncertain terrain in under 20 minutes in real-time, one team attempted to incorporate the best-of-breed work from multiple other advancements in reinforcement learning. "Perhaps surprisingly, we find that a quadrupedal robot can learn to walk from the beginning with deep RL in under 20 minutes, across a range of various locations and surface types, given a few judicious design considerations in terms of the task setup and algorithm implementation. The researchers emphasize that this does not necessitate new algorithmic elements or any other unforeseen innovation. Conversely, they choose and combine a few cutting-edge strategies to get extraordinary outcomes.

According to the researchers, it is difficult to implement the technique in the actual world because it is unclear whether there is enough data to train the forecasting system. However, they remain upbeat about the applications, some of which may involve foreseeing damage to bridges and other buildings.