The sequence of user-item interactions can be effectively incorporated in the generation of personalized explanations in recommender systems. By modeling user behavior history sequentially, it is possible to enhance the quality and personalization of explanations provided alongside recommendations, without affecting recommendation quality.

In a study with Alejandro Ariza-Casabona, Maria Salamó, and Gianni Fenu, published in the proceedings of ACM RecSys ’23, we present SEQUER (SEQuence-Aware Explainable Recommendation), a novel approach aiming to leverage user-item interaction sequences for generating personalized explanations alongside maintaining recommendation performance.

The SEQUER Model

SEQUER stands on the shoulders of the PETER model, incorporating the sequence of user-item interactions to enhance the explanation generation step. This model addresses a significant gap in existing explainable recommender systems, which often overlook the user behavior sequence crucial for modeling user interest drifts for high-quality recommendations.

The architecture of our model is presented in the following figure.

Sequential Input Representation

• Input. The model’s input includes user ID, a sequence of item IDs, optional item feature (e.g., words), and textual tokens from explanations.
• Encoding and Positional Encoding. The tokens are encoded using embedding codebooks and a whitespace tokenizer, followed by positional encoding.
• Limitation on Input Size. To manage computational complexity, the maximum sequence length for items and the size of the explanation text are fixed.

Transformer and Attention Masking

• Use of a Transformer architecture. SEQUER employs a Transformer-like structure, which is beneficial for tasks like sequential recommendation and natural language generation due to its attention mechanism.
• Attention Masking. The model explores various mask-creation alternatives to optimize the use of sequential information in generating explanations. This includes causal, bidirectional, and sequence-adapted masks.

Optimization Objectives

• Multiple Task Optimization. SEQUER is optimized for several tasks: rating prediction, context prediction, explanation generation, and next-item prediction, which function as task regularizers to each other.
• Task-Specific Details:
  • Rating Prediction. Uses contextualized item embeddings to predict item ratings.
  • Context Prediction. Aims to predict words belonging to the current context, enhancing the personalization of explanations.
  • Explanation Generation. Focuses on modeling the conditional probability distribution of each token based on previous ones, using a negative log-likelihood loss.
  • Next-Item Prediction. Incorporates user behavior history as input, using contextualized embeddings to predict the next item.

Experimental Validation

The effectiveness of SEQUER was tested on four real-world datasets: Amazon Beauty, Amazon Sports, Amazon Toys, and Yelp. The experiments evaluated the model against state-of-the-art baselines across various metrics, including rating prediction accuracy, explanation diversity, and text quality.

Results

The key results of SEQUER can be summarized as follows:

1. Explainability Improvement. SEQUER outperformed other baseline models in terms of explainability, indicating its ability to generate more personalized and diverse explanations.
2. Diverse Feature Generation. The model demonstrated superior performance in generating diverse features closely matching user keywords, which underlines its capability for detailed user personalization.
3. Text Quality Metrics. In the Amazon datasets, SEQUER showed better results in text quality metrics compared to PETER, especially when the model did not have access to item feature information.
4. Rating Prediction Performance. The results indicated that SEQUER’s performance in rating prediction was comparable to other models, despite its complex multitask optimization involving next-item prediction and explanation generation.
5. Effectiveness of Sequential Modeling. The incorporation of user behavior sequences in the recommendation process proved effective in improving the quality of explanations.
6. Attention Masking Strategies: A fully causal masking strategy was found to be the most effective for next-item prediction and language generation tasks within the SEQUER model.
7. Impact of Recommendation Tasks as Regularizers. The study revealed that the best performance in text quality metrics was achieved when both next-item prediction and rating prediction tasks were included in the multitask setup.

Conclusions and Future Directions

The SEQUER model presents a significant advancement in explainable recommendation systems, effectively utilizing user behavior sequences for personalization. Future research could explore multi-modal item representations and continuous prompt learning, further enhancing the model’s capabilities.