5 mins read

Artificial intelligence based on big data – Airbnb case

brown wooden center table
Photo by Skitterphoto on Pexels.com

In case you have wondered, much of what happens to us on social networks and the internet is programmed. Not programmed to be planned in advance but written in lines of code that a programmer has written and are based on the use of an algorithm. An algorithm is a process or collection of rules to be followed and that are calculated by a computer.

Nowadays most social networks and internet platforms use algorithms that decide for you. They execute a series of rules and offer you a result. We can say that these algorithms are artificial intelligence since the algorithm learns, understands, makes decisions and forms a certain idea of reality. And this leads to machine learning, which is machine learning.

For this learning, big data is usually used, an immense amount of data, which the machine processes and according to the result of its calculations makes decisions for you. In this article I am going to start a series of articles with different platforms that use artificial intelligence. There are going to be several articles and this is the first one where I talk about Airbnb.

Machine learning in Airbnb

Airbnb has a wide variety of machine learning issues ranging from models on traditional structured data to models based on unstructured data, such as user reviews, messages, and list images. The ability to create, iterate, and maintain healthy machine learning models is critical to Airbnb’ssuccess.

Many machine learning platforms cover data collection, feature engineering, training, deployment, production, and monitoring, but few, if any, do all of the above seamlessly.

Machine Learning Team

In the final quarter of 2016, a machine learning team was formed. Work was underway on several models that took an average of 8 to 12 weeks to build. Everything was built in Aerosolve,  Spark  and  Scala but there was no support for popular machine learning packages like  Tensorflow,  Pytorch,  SK-Learn,etc.

Before the formation of the Machine Learning team, I was very important in Airbnb for search by ranking, smart prices and search for fraud. After the implementation of this equipment, the scope of Machine Learning was amplified through business travel classifiers, classified lists, hosting availability, better photographs of the host, etc.

Machine learning infrastructure

The idea of this Machine Learning (ML) team was to reduce the complexity of the models using reusable solutions. In addition, everything is shared with the community and there are different solutions that can be employed. The workflow is simplified for effective solutions. These solutions come through libraries and a more efficient work environment.

It creates p0r both a platform that allows to transmit cross information shared throughout the company creating an adequate work environment to do the most appropriate in the shortest possible time.

Bighead, el machine learning

Bighead aims to bring together various internal and open source projects to remove incidental complexity from ML workflows. Bighead is based on Python and Spark and can be used in modular parts, as each ML problem presents unique challenges. By standardizing the path to production, training environments, and methods for collecting and transforming data in Spark, each model is reproducible and iterable.

Airbnb.io, a platform to consult projects

Airbnb has a platform for data scientists and engineers with several open source projects that allows the community to share, see, learn about how they work or how the world will work in the coming years. Some of these projects can be seen in Fig. 1.

Figure 1. Projects open source on Airbnb


  • https://twitter.com/thexxlman/status/1455140141970903043?s=21
  • https://databricks.com/session/bighead-airbnbs-end-to-end-machine-learning-platform
  • https://databricks.com/session/apache-spark-at-airbnb
  • https://scalac.io/blog/financial-intelligence-at-airbnb-with-scala-a-case-study/
  • https://www.tensorflow.org/?hl=es-419
  • https://scikit-learn.org/stable/
  • https://patents.google.com/patent/US11132499B2/en?q=airbnb&oq=airbnb
  • https://airbnb.io/

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Previous Story

Minería de datos a partir de tweets de políticos españoles

Next Story

Inteligencia artificial basada en big data – caso Airbnb

%d bloggers like this: