FEA: Add FM2 in context-aware models

docs/source/index.rst

@@ -11,7 +11,7 @@ Introduction
 RecBole is a unified, comprehensive and efficient framework developed based on PyTorch.
 It aims to help the researchers to reproduce and develop recommendation models.
 
-In the lastest release, our library includes 86 recommendation algorithms `[Model List]`_, covering four major categories:
+In the lastest release, our library includes 87 recommendation algorithms `[Model List]`_, covering four major categories:
 
 - General Recommendation
 - Sequential Recommendation

docs/source/user_guide/model/context/fm2.rst

@@ -0,0 +1,86 @@
+FM2
+===========
+
+Introduction
+---------------------
+
+`[paper] <https://dl.acm.org/doi/10.1145/3442381.3449930>`_
+
+**Title:** FM^2: Field-matrixed Factorization Machines for Recommender Systems
+
+**Authors:** Yang Sun, Junwei Pan, Alex Zhang, Aaron Flores
+
+**Abstract:** Click-through rate (CTR) prediction plays a critical role in recom-
+mender systems and online advertising. The data used in these
+applications are multi-field categorical data, where each feature
+belongs to one field. Field information is proved to be important
+and there are several works considering fields in their models. In
+this paper, we proposed a novel approach to model the field in-
+formation effectively and efficiently. The proposed approach is
+a direct improvement of FwFM, and is named as Field-matrixed
+Factorization Machines (FmFM, or 𝐹 𝑀 2 ). We also proposed a new
+explanation of FM and FwFM within the FmFM framework, and
+compared it with the FFM. Besides pruning the cross terms, our
+model supports field-specific variable dimensions of embedding
+vectors, which acts as a soft pruning. We also proposed an efficient
+way to minimize the dimension while keeping the model perfor-
+mance. The FmFM model can also be optimized further by caching
+the intermediate vectors, and it only takes thousands floating-point
+operations (FLOPs) to make a prediction. Our experiment results
+show that it can out-perform the FFM, which is more complex. The
+FmFM model’s performance is also comparable to DNN models
+which require much more FLOPs in runtime.
+
+.. image:: ../../../asset/fm2.png
+    :width: 500
+    :align: center
+
+Running with RecBole
+-------------------------
+
+**Model Hyper-Parameters:**
+
+- ``embedding_size (int)`` : The embedding size of users, items and entities. Defaults to ``10``.
+
+**A Running Example:**
+
+Write the following code to a python file, such as `run.py`
+
+.. code:: python
+
+   from recbole.quick_start import run_recbole
+
+   run_recbole(model='FM2', dataset='ml-100k')
+
+And then:
+
+.. code:: bash
+
+   python run.py
+
+Tuning Hyper Parameters
+-------------------------
+
+If you want to use ``HyperTuning`` to tune hyper parameters of this model, you can copy the following settings and name it as ``hyper.test``.
+
+.. code:: bash
+
+    learning_rate choice [0.01,0.005,0.001,0.0005,0.0001]
+
+Note that we just provide these hyper parameter ranges for reference only, and we can not guarantee that they are the optimal range of this model.
+
+Then, with the source code of RecBole (you can download it from GitHub), you can run the ``run_hyper.py`` to tuning:
+
+.. code:: bash
+
+	python run_hyper.py --model=[model_name] --dataset=[dataset_name] --config_files=[config_files_path] --params_file=hyper.test
+
+For more details about Parameter Tuning, refer to :doc:`../../../user_guide/usage/parameter_tuning`.
+
+
+If you want to change parameters, dataset or evaluation settings, take a look at
+
+- :doc:`../../../user_guide/config_settings`
+- :doc:`../../../user_guide/data_intro`
+- :doc:`../../../user_guide/train_eval_intro`
+- :doc:`../../../user_guide/usage`

docs/source/user_guide/model_intro.rst

@@ -1,6 +1,6 @@
 Model Introduction
 =====================
-We implement 86 recommendation models covering general recommendation, sequential recommendation,
+We implement 87 recommendation models covering general recommendation, sequential recommendation,
 context-aware recommendation and knowledge-based recommendation. A brief introduction to these models are as follows:
 
 
@@ -74,6 +74,7 @@ are also support for these models. And evaluation is always conducted in the way
    model/context/lightgbm
    model/context/kd_dagfm
    model/context/fignn
+   model/context/fm2
 
 
 Sequential Recommendation

recbole/model/context_aware_recommender/__init__.py

@@ -6,6 +6,7 @@
 from recbole.model.context_aware_recommender.dssm import DSSM
 from recbole.model.context_aware_recommender.ffm import FFM
 from recbole.model.context_aware_recommender.fm import FM
+from recbole.model.context_aware_recommender.fm2 import FM2
 from recbole.model.context_aware_recommender.fnn import FNN
 from recbole.model.context_aware_recommender.fwfm import FwFM
 from recbole.model.context_aware_recommender.lr import LR
@@ -15,4 +16,4 @@
 from recbole.model.context_aware_recommender.xdeepfm import xDeepFM
 from recbole.model.context_aware_recommender.fignn import FiGNN
 from recbole.model.context_aware_recommender.kd_dagfm import KD_DAGFM
-from recbole.model.context_aware_recommender.eulernet import EulerNet
+from recbole.model.context_aware_recommender.eulernet import EulerNet

recbole/model/context_aware_recommender/fm2.py

@@ -0,0 +1,78 @@
+# -*- coding: utf-8 -*-
+# @Time   : 2023/4/9
+# @Author : Yilin Liu
+# @Email  : liu_elaine2022@outlook.com
+# @File   : fm2.py
+
+r"""
+FM2
+#####################################################
+Reference:
+    Sun Y, Pan J, Zhang A, et al. “FM2: Field-matrixed Factorization Machines for Recommender Systems.” in WWW 2021.
+"""
+
+import torch
+import torch.nn as nn
+from torch.nn.init import xavier_normal_, constant_
+from recbole.model.abstract_recommender import ContextRecommender
+
+
+class FM2(ContextRecommender):
+    """
+        Field-matrixed Factorization Machines (FmFM, or 𝐹M^2 ), propose a novel approach FmFM to model the
+interactions of field pairs as a matrix.
+    """
+
+    def __init__(self, config, dataset):
+        super(FM2, self).__init__(config, dataset)
+
+        self.embedding_dim = config["embedding_size"]
+        self.num_fields = self.num_feature_field
+        self.interact_dim = int(self.num_fields * (self.num_fields - 1) / 2)
+
+        self.sigmoid = nn.Sigmoid()
+        self.loss = nn.BCEWithLogitsLoss()
+
+        self.weight = torch.randn(
+            self.interact_dim, self.embedding_dim, self.embedding_dim, requires_grad=True, device=self.device
+        )
+
+        # parameters initialization
+        self.apply(self._init_weights)
+
+        # used to mark cross items
+        self.triu_index = torch.triu(torch.ones(self.num_fields, self.num_fields), 1).nonzero().to(self.device)
+
+    def _init_weights(self, module):
+        if isinstance(module, nn.Embedding):
+            xavier_normal_(module.weight.data)
+        elif isinstance(module, nn.Linear):
+            xavier_normal_(module.weight.data)
+            if module.bias is not None:
+                constant_(module.bias.data, 0)
+
+    def fm2_matrix(self, infeature):
+        """
+            FmFM interaction terms calculation
+        """
+        left_emb = torch.index_select(infeature, 1, self.triu_index[:, 0])  # [batch_size, interact_dim, embed_dim]
+        right_emb = torch.index_select(infeature, 1, self.triu_index[:, 1])
+        left_emb = torch.matmul(left_emb.unsqueeze(2), self.weight).squeeze(2)
+        fm2_output = (left_emb * right_emb).sum(dim=-1).sum(dim=-1, keepdim=True)  # [batch_size, 1]
+
+        return fm2_output
+
+    def forward(self, interaction):
+        fm2_all_embeddings = self.concat_embed_input_fields(interaction)  # [batch_size, num_field, embed_dim]
+
+        output = self.first_order_linear(interaction) + self.fm2_matrix(fm2_all_embeddings)
+
+        return output.squeeze(-1)
+
+    def calculate_loss(self, interaction):
+        label = interaction[self.LABEL]
+        output = self.forward(interaction)
+        return self.loss(output, label)
+
+    def predict(self, interaction):
+        return self.sigmoid(self.forward(interaction))

recbole/properties/model/FM2.yaml

@@ -0,0 +1 @@
+embedding_size: 10               # (int) The embedding size of features.

tests/model/test_model_auto.py

@@ -408,6 +408,14 @@ def test_kd_dagfm(self):
         }
         quick_test(config_dict)
 
+
+    def test_fm2(self):
+        config_dict = {
+            "model": "FM2",
+            "threshold": {"rating": 4},
+        }
+        quick_test(config_dict)
+
     def test_eulernet(self):
         config_dict = {
             "model": "EulerNet",