Idibon | Language Intelligence


So the decision to start Idibon came when I was traveling by bicycle across Alaska. I took some time after completing my PhD where I met my co-founder Tyler to decide how can I bring as many people into the information age as possible. And the way to do this was to build a large sustainable company bringing together the smartest minds of Silicon Valley. The amount of data that businesses are dealing with is doubling every two years, and the majority of that data is unstructured. What that means is that there’s an increasingly large amount of information captured in PDF documents, email, instant messaging, and other forms of unstructured communications that is not accessible to the majority of businesses out there right now. Organizations struggle with the sheer volume of information that’s going through. Often, they’ll have a roomful of analysts trying to pick out insights from amounts of data which are only getting larger and larger, when they can’t scale their analyst teams in the same way. The large organizations that we work for care what everybody is saying. Whether that someone is in San Francisco or in Seoul. So that these organizations can meet them there and provide the products and services that their consumers are already talking about. Text analytics is the process of understanding, managing, and taking action from text data. We use machine learning in order to automatically understand the insights from that data and process it at scale. What I really like about the human in the loop processing is that both the machine learning and the analyst get smarter faster. Then the data’s in the system and it can be constantly flowing into the system. And the first thing is to sort of get some annotations, what are the categories, define the categories you care about, and then annotate those. That’s what allows us to train machine learning models to help optimize the human in the loop, because what happens is that you’re able to take the smarts of the people and have the machines understand the patterns that are behind them, so the machine can, in turn, predict what’s going on. That frees the humans up to be able to do the stuff that they’re good at, which is think, rather than just sort of write rules or see every single document. Our biggest clients need 90-95 percent accuracy to fully understand their data and take the action, find those insights that are required. Off-the-shelf systems target 60 to 70 percent accuracy and won’t give the meaningful trends or find those new elusive insights . So you need the combination of human analysts and machine learning. And this is Idibon’s expertise. On a given day, only 5% of the world’s conversations are in English. And as more of the world comes online, what this means is that our digital communications are also going to be in any one of 7,000 different languages, with only a small percentage of those being in English. Idibon’s technology is language-independent. We can get to state-of-the-art accuracy in a new language in a matter of days. We’ve worked in over 50 languages to date, including emoji, with many more to come. What I enjoy most about coming to work today is our team. We have a team that combines years of experience here in Silicon Valley with years of experience in global development. Working in a large number of disaster response situations and in global epidemic tracking, it’s become very clear that putting a cell phone in the hands of everyone on the planet was actually the easy part; the much harder part is going to be understanding what they’re saying.

Leave a Reply

Your email address will not be published. Required fields are marked *