In fact, Microsoft says the Azure Cognitive Services system can describe images as accurately as humans can. Image captioning allows developers to include automated descriptions of visual content. However, the AI that underpins the tech is unable to patch human descriptions in terms of accuracy. Microsoft’s new system claims to break through this barrier. If true, developers can leverage the system to deliver more accuracy to users through improved descriptions of images. For example, images that could be displayed by search results. It is worth noting that Microsoft has an important caveat. The company says that while the system can deliver human-like accuracy, it does not do it every time. Instead, the Azure Cognitive Services team says it is not perfect.
Usefulness
Saqib Shaikh, a software engineering manager with Microsoft’s AI team, says the system is still an important milestone. One example he points to is its ability to generate descriptions from photos from people with visual impairments. These image descriptions are known as ALT text and are found on websites and documents. However, some pages don’t manually ascribe ALT text. Microsoft’s AI system would automate the process. “Ideally, everyone would include alt text for all images in documents, on the web, in social media – as this enables people who are blind to access the content and participate in the conversation. But, alas, people don’t. So, there are several apps that use image captioning as way to fill in alt text when it’s missing.” Microsoft says the new AI is twice as accurate than the tool the company has been using previously. The company plans to integrate the new system into Microsoft Word and Outlook on Windows and Mac later this year.
