Evolution of Recommendation Models
🧬

Evolution of Recommendation Models

Tags
Data Science
Date Published
January 23, 2024

Introduction

Most of the our daily-used products, such as search engine, ecommence recommendation and online adverizing, are supported by recommendation models.

This blog is mainly about the evoluation of these techniques and the logics behind these evolution.

Recommendation Models

Problem Definition

Given specific context, find the relevance between user and item. Below is the function we want to get

Relevance=f(user,item,context)Relevance=f(user, item, context)

Main Applications

In terms of applications, there are mainly three categories

  1. Recommendation system: news, video, e-commence recommendation
  2. Search engine: Google search, search box in most products
  3. Online advertizing: click(conversion) prediction, real-time bidding

Evolution Graph

‣
Stage 1: Classic Recommendaton Model
‣
Stage 2: Deep Learning Recommendation Model

Models in Details

‣
Before 2005: Near-neighbor based Collaborative Filtering
‣
2006: Model-based Collaborative Filtering: MF (Maxtirx Factorization)
‣
2007: LR (Logistic Regression) & GBDT & Hybrid model
‣
2010-2013: FM (Factorization Machine)
‣
After 2016: Deep Learning/MLP
‣
Multi-task deep learning model

Evolutionary Trends

‣
Inclusion of more features
‣
Improved feature representation (particularly for categorical features)
‣
Enhanced feature interaction
‣
Balancing memorization and generalization
‣
Embracing multi-task learning

What’s not Included ?

There are still many topics that are no included in this blog:

  1. Embedding techniques including word2vector, graph embedding and so on
  2. Transformer in recommendation model
  3. Deep Reinforcement learning
  4. Causality inspired methods
  5. Application of LLM in recommendation
  6. …..
‣

Reference