Recommendation System

Contributors

• Brian Hutchinson, Assistant Professor, Computer Science Department, WWU
• Aaron Tuor, Graduate Student, Computer Science Department, WWU
• Katy McClintic, Undergraduate, Computer Science Department, WWU

Experiments

• • Research Guide

Experiments

Tips and Tricks

Permissions

• The script is hutch_research/bin/group_permissions.sh.

  It takes a single argument: the directory whose contents you want to (recursively) fix permissions on.

• Or for an individual file or folder just:

  chmod g+wx /home/hutch_research/restOfFilepath

  • Tutorial on chmod

Wiki tips

• Basic Wiki editing

General Data Processing

• Hutch Wiki Data Processing Page

Matlab Cheat Sheet

Notes and Posters

• Posters
  • Matrix Factorization and Cold Start: WWU Graduate Student Conference 2015
  • Matrix Factorization and Cold Start: WWU Scholar's Week 2015
• Brian's gradient notes for Joint Factorization Model
  • Part 1
  • Part 2
  • Part 3
• Data Notes
• Matrix Factorization Notes
• Similarity Notes
• Correspondence

Background and Basics

Video Lectures

• Linear Algebra MIT Open Courseware (Gilbert Strang)
• Mining Massive Data Sets Coursera (Jure Leskovec, J. McCauley. Stanford)
• Machine Learning Coursera (Andrew Ng. Stanford)
• Recommender Systems Coursera (Joseph Konstan, Michael Ekstrand. University of Minnesota)

Notes

• Linear Algebra Review (Carnegie Mellon)
• Hadamard Product Properties
• Backpropagation (Makin 2006)

Books

• Mining of Massive Datasets (2012. Leskovec, et. al.)
  • Resource for data mining, distance measures, intro to recommendations systems

• Collaborative Filtering Recommendation Systems (Konstan, et. al. 2011)
  • Good introductory survey of the basics and the state of the art in the field circa 2011

• Recommender Systems Handbook (Ricci, et. al. 2011)
  • A more advanced overview of the field of recommender systems

• Graph Mining: Laws, tools, and case studies (Chakrabarti and Faloutsos 2012)
  • Few chapters on tensors (chpts 14 \& 15)
  • Free online checkout via WWU

• Matrix Cookbook
  • Lots of valuable derived formulas for finding gradients of matrix equations

• Non-Linear Optimization

Papers

• Tensor decompositions and applications (Kolda and Bader 2009)
  • The basics about tensors

• A guide to recurrent neural networks and backpropagation (Boden 2001)

• Accurate Methods for the Statistics of Surprise and Coincidence. (Dunning 1993)
  • In case you want to know more about why log-likelihood makes sense

• Latent Dirichlet Allocation (Ng, et. al. 2002)
  • Famous paper on Latent Dirichlet Allocation

Wikis

• ACM RecSys Wiki
  • compiled by the ACM Conference Series on Recommendation Systems
  • has links to many resources from datasets, articles, books, projects, software and other tools

Tutorials

• Matrix Factorization Tutorial in Python
  • Great beginning tutorial on matrix factorization as it applies to Recommendation Systems

Web Pages

• Top 10 movie recommendation engines
• Winning the Netflix Prize: A summary
• Funk SVD Explanation
• Why Netflix Never Implemented The Algorithm That Won
• Deconstructing Recommender Systems
• Amazon Reviewers
  • Top Reviewers: Tons of Free Stuff
  • Amazon's top reviewers

Conferences and Jourals

• Recsys Page of Relevant Conferences and Journals

• TKDE Transactions on Knowledge and Data Engineering

• SIGIR 2016 (Special Interest Group on Information Retrieval)

• Recsys 2016

Public Datasets

• Netflix Challenge Training Set (UW version)

• RecSys DataSets

• IMBD Movie Meta Data

• Movie Lens Data source (already downloaded)

• Amazon Reviews Snap Data(Stanford)
  • The dynamics of viral marketing (Leskovec, et. al. 2007)
    • First citation of Amazon Reviews Dataset
  • Hidden Factors and Hidden Topics: Understanding Rating Dimensions with Review Text. (Leskovec and McAuley 2013)
    • Later citation from more developed dataset
  • Max Span Tree Graphs of Cross-Domain reviewers
    • graphs 1-3
    • graph 4

• Recommendation and Ratings Public Data Sets For Machine Learning

• Hetrec 2011 datasets

Open Source Software

• Apache Mahout
  • Tool for building machine learning applications
  • Resource for this tool: Mahout in Action pdf Book
  • JavaDocs
  • Apache Mahout tutorial (series)
  • Mahout Item Recommender Tutorial using Java and Eclipse

• RiVal
  • Recommender evaluation toolkit
  • javadocs

• MyMediaLite: Recommender System Library
  • Written in C#, for the .NET platform; runs on every architecture supported by Mono: Linux, Windows, Mac OS X.

• LensKit
  • Recommender Systems toolkit implemented in java

• Recommender 101
  • Framework written in Java to carry out offline experiments for Recommender Systems.

• Local Collective Embeddings
  • This repository contains the MATLAB implementation of the Local Collective Embeddings model for cold-start recommendations.

• Matlab scripts for latent semantic analysis and related methods

• pmtk3: Matlab probabilistic modelling toolkit
  • Matlab implementations of neural networks and other machine learning tools

• Matrix Factorization Jungle
  • Downloadable implementations of many advanced matrix factorization algorithms.

• CNTK Computational Network Toolkit
  • Computational networks (CNs) generalize models that can be described as a series of computational steps such as DNN, CNN, RNN, LSTM, and maximum entropy models.
  • CNTK is a research code and ongoing project. There will be bugs in places.

• Poblano
  • Poblano is a Matlab toolbox of large-scale algorithms for unconstrained nonlinear optimization problems. The algorithms in Poblano require only first-order derivative information (e.g., gradients for scalar-valued objective functions), and therefore can scale to very large problems. The driving application for Poblano development has been tensor decompositions in data analysis applications (bibliometric analysis, social network analysis, chemometrics, etc.).

Past Updates

• Winter 2015
  • AT 1/14/2015
  • AT 2/04/2015
  • AT 2/11/2015
  • AT 2/18/2015
  • AT 3/05/2015

• Spring 2015
  • AT 4/02/2015
  • AT & KM 4/23/2015
  • KM 4/30/2015
  • AT & KM 5/07/2015
  • AT & KM 5/14/2015
  • AT & KM 5/21/2015
  • AT & KM 5/28/2015

CourseRank

• CourseRank: A Closed-Community Social System Through the Magnifying Glass (Kourtrika, et. al. 2009)

• RecSys 2014 Keynote by Hector Garcia-Molina: The Future of Recommender Systems

• CourseRank to be taken down at the end of the month (Nov 2014)

Literature

Extended Literature

• Recommendation Systems Literature Page

Non-Linearity

• ⇒ Probabilistic Matrix Factorization (Salakhutdinov 2008)

• ? Restricted Boltzmann Machines for Collaborative Filtering (Salakhutdinov 2007)

MF Optimization Techniques

• ? Matrix Factorization and Neighbor based Algorithms for the Netflix Prize Problem (Takacs 2008)

• ? Generalized Hebbian Algorithm for Incremental Singular Value Decomposition in Natural Language Processing

• Algorithms and applications for approximate nonnegative matrix factorization

• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization (Recht 2010)

• Convex Non-Negative Matrix Factorization in the wild (Thurau 2009)

• ? Convex Collective Matrix Factorization (Bouchard 2013)

• ? Nonnegative Matrix and Tensor Factorizations (Amari 2009)

• Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent (Gemulla. 2011)

• Modern Optimization Modelling Techniques (Commenetti 2012)

• Fast maximum margin matrix factorization for collaborative prediction (Rennie, 2005)

• ? Projected Gradient Methods for Non-negative Matrix Factorization (Lin, 2007)

• ⇒ Algorithms for Non-negative Matrix Factorization (Seung, Lee. 2001)

• ? Scalable Collaborative Filtering Approaches for Large Recommender Systems (Takacs 2009)

• A Fast Distributed Stochastic Gradient Descent Algorithm for Matrix Factorization (Li 2014)

• On the Convergence of Multiplicative Update Algorithms for Non-negative Matrix Factorization (Lin 3007)

• ? Maximum-Margin Matrix Factorization (Rennie 2005)

Combining MovieLens and Imdb data

• Extraction and Integration of MovieLens and IMDb Data (Peralta 2007)

• Description of the IMDb and MovieLens datasets (Silvestri and Pagh 2013)

• Matching of MovieLens and IMDb Movie Titles (Peralta 2007)

Tensors

• ? Generalized Coupled Tensor Factorization (Yilmaz 2011)

• ⇒ Tensor Factorization via Matrix Factorization

• Matrix and Tensor Factorization approaches applied to recommender systems: A survey (Ricci, et. al. 2012)

• Multiverse Recommendation: N-dimensional tensor factorization for context aware collaborative filtering (Karatzoglou et. al. 2010)

• DFactTo: Distributed Factorization of Tensors (Choi, et. al. 2014)

• Personalized Recommendation via Cross-Domain Triadic Factorization (Hu et. al. 2014)

• Tensor-based Item Recommendation using Probabilistic Ranking in Social Tagging Systems (Ifada and Nayak 2014)

• ? All-at-once Optimization for Coupled Matrix and Tensor Factorizations (Kolda, et. al. 2011)

• ? A Three-Way Model for Collective Learning on Multi-Relational Data (Nickel, et. al. 2011)

• Tensor Factorization for Multi-Relational Learning (Nickel and Tresp 2013)

Matrix Factorization

• Initializations for the Nonnegative Matrix Factorization (Meyer, et. al. 2006)

• ⇒ Advances in collaborative filtering (Koren and Bell 2010)

• Factorization meets the neighborhood (Koren 2008)

• Matrix factorization techniques for recommender systems (Bell, et. al. 2009)

• Non-negative Matrix Factorization on Manifold (Cai, et. al. 2008)

Neural Networks

• LEARNING ACTIVATION FUNCTIONS TO IMPROVE DEEP NEURAL NETWORKS (Hoffman, et. al. 2015)

• The Collaborative Filtering Recommendation Algorithm Based on BP Neural Networks (Chen 2009)

• Collaborative Filtering for Recommendation Using Neural Networks (Kim, et. al. 2005)

• CONTEXT DEPENDENT RECURRENT NEURAL NETWORK LANGUAGE MODEL (Mikolov and Zweig 2012)

• A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems

• Distributed Representations of Sentences and Documents (Mikolov 2014)

• Deep Sentence Embedding Using Long Short-Term Memory Networks (He 2015)

Joint Factorization

• ⇒ A Unified view of Matrix Factorization models (Singh and Gordon 2008)

• ? Relational learning via collective matrix factorization (Singh and Gordon 2008)

• ? Item Cold-Start Recommendations: Learning Local Collective Embeddings (Saveski and Mantrach 2014)

Research Methodology

• Rethinking the recommender research ecosystem: reproducibility, openness, and lenskit. (Konstan, et. al. 2011)

Review text & Product description analysis

• ? Hidden Factors and Hidden Topics: Understanding Rating Dimensions with Review Text. (Leskovec and McAuley 2013)

• Explicit Factor Models for Explainable Recommendation based on Phrase-level Sentiment Analysis (Zhang, et. al. 2014)

• Ratings meet reviews, a combined approach to recommend (Ling, et. al. 2014)

• Solving the Cold Start Problem Using Product Related Contents (Hu 2013)

• fLDA: Matrix Factorization through Latent Dirichlet Allocation (Chen, et. al. 2010)

User Cold Start

• Representation Learning for cold-start recommendation (Contardo, et. al. 2015)