Developing A Contextual Combinational Approach for Predictive Analysis of Users Mobile Phone Trajectory Data in LBSNs
Research Areas : Data Mining
Fatemeh Ghanaati
^{
1
}
Gholamhossein Ekbatanifard
^{
2
}
Kamrad Khoshhal Roudposhti
^{
3
}
Keywords: LBSN, Trajectory data, Contextual Information, GRU,
Abstract :
Today, smartphones, due to their ubiquity, have become indispensable in human daily life. Progress in the technology of mobile phones has recently resulted in the emergence of several popular services such as locationbased social networks (LBSNs) and predicting the next Point of Interest (POI), which is an important task in these services. The gathered trajectory data in LBSNs include various contextual information such as geographical and temporal contextual information (GTCI) that play a crucial role in the next POI recommendations. Various methods, including collaborating filtering (CF) and recurrent neural networks, incorporated the contextual information of the user’ trajectory data to predict the next POIs. CF methods do not consider the effect of sequential data on modeling, while the next POI prediction problem is inherently a time sequence problem. Although recurrent models have been proposed for sequential data modeling, they have limitations such as similarly considering the effect of contextual information. Nonetheless, they have a separate impact as well. In the current study, a geographical temporal contextual informationextended attention gated recurrent unit (GTCIEAGRU) architecture was proposed to separately consider the influence of geographical and temporal contextual information on the next POI recommendations. In this research, the GRU model was developed using three separate attention gates to consider the contextual information of the user trajectory data in the recurrent layer GTCIEAGRU architecture, including timestamp, geographical, and temporal contextual attention gates. Inspired by the assumption of the matrix factorization method in CF approaches, a ranked list of POI recommendations was provided for each user. Moreover, a comprehensive evaluation was conducted by utilizing largescale realworld datasets based on three LBSNs, including Gowalla, Brightkite, and Foursquare. The results revealed that the performance of GTCIEAGRU was higher than that of competitive baseline methods in terms of Acc@10, on average, by 42.11% in three datasets.
[1] L. Huang, Y. Ma, Sh. Wang, Y. Liu, “An Attentionbased Spatiotemporal LSTM Network for Next POI Recommendation”, Journal of IEEE Transactions on Services Computing, vol. 12, 2019, pp. 113.
[2] J. Manotumruksa, C. Macdonald, I. Ounis, “A Contextual Attention Recurrent Architecture for Context Aware Venue Recommendation”, in 18th ACM SIGIR Conference on Research and Development in Information Retrieval, 2018, pp. 555564.
[3] K. Kala, M. Nandhini,” Context Category Specific sequence aware Point of Interest Recommender System with Multi Gated Recurrent Unit”, Journal of Ambient Intelligence and Humanized Computing, 2019, https://doi.org/10.1007/s1265201901583w.
[4] C. Liu, J. Liu, J. Wang, S. Xu, H. Han, Y. Chen, “An AttentionBased Spatiotemporal Gated Recurrent Unit Network for PointofInterest Recommendation”, International Journal of GeoInformation, vol. 8, No. 8, 2019, pp.355373.
[5] S. Wang, Z. Bao, J. Culpepper, G. Cong, “A Survey on Trajectory Data Management, Analytics, and Learning”, ACM Computing Surveys, vol. 54, No.3, 2020, pp. 133.
[6] J. Feng Y. Li, C. Zhang, F. Sun, F. Meng, A. Guo, D. Jin, “DeepMove: Predicting Human Mobility with Attentional Recurrent Networks”, in 18th ACM IW3C2 Conference on International World Wide Web, 2018, pp. 14591468.
[7] Q. Guo, Z .Sun, J. Zhang, Y. Theng, “An Attentional Recurrent Neural Network for Personalized Next Location Recommendation”, in 34th ACM AAAI Conference on Artificial Intelligence, 2020, pp. 8390.
[8] D.Yao,C. Zhang, J. Huang, J. Bi, “SERM: A Recurrent Model for Next Location Prediction in Semantic Trajectories”, in 17th ACM CIKM Conference on Information and Knowledge Management, 2017 ,pp. 24112414.
[9] L. Zhang, Z. Sun, J. Zhang, H. Kloeden, F. Klanner, “Modeling hierarchical category transition for next POI recommendation with uncertain checkins”, Journal of Information Sciences, Elsevier, vol.515, 2019, pp. 169190.
[10] L. Chang, W. Chen, J. Huang, Ch. Bin, W. Wang, “Exploiting multiattention network with contextual influence for pointofinterest recommendation”, Journal of Applied Intelligence, vol. 51, 2021, pp. 1904–1917.
[11] L. Huang, Y. Ma, Y. Liu, K. He, “DANSNR: A Deep Attentive Network for SocialAware Next PointofInterest Recommendation”, Journal of ACM Transactions on Internet Technology, Vol.21, No.2, 2020, pp. 1–27.
[12] G. Christoforidis, P. Kefalas, A. Papadopoulos, Y. Manolopoulos, “RELINE: PointofInterest Recommendations using Multiple Network Embeddings”, Journal of Knowledge and Information Systems, Vol. 63, No.4, 2019, pp. 791817.
[13] J. Manotumruksa, C. Macdonald, I. Ounis, “A Deep Recurrent Collaborative Filtering Framework for Venue Recommendation”, in 17th ACM CIKM Conference on Information and Knowledge Management, 2017, pp. 14291438.
[14] D. Yang, D Zhang, V. Zheng, Z. Yu, “Modeling User Activity Preference by Leveraging User Spatial Temporal Characteristics in LBSNs”, Journal of IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol.45, No.1, 2014, pp. 129 – 142.
[15] M. Quadrana, P. Cremonesi, D. Jannach, “SequenceAware Recommender Systems”, Journal of ACM Computing Surveys, Vol.51, No.4, 201, pp. 1–36.
[16] Q. Cui, Y. Tang, S. Wu, L. Wang, “Distance2Pre: Personalized Spatial Preference for Next PointofInterest Prediction”, in PAKDD Conference on Knowledge Discovery and Data Mining, 2019, pp. 289301.
[17] Q. Gao, F. Zhou, G. Trajcevski, K. Zhang, T. Zhong, F. Zhang, “Predicting Human Mobility via Variational Attention”, in IW3C2 Conference on International World Wide Web Conference Committee, 2019, pp. 2750–2756.
[18] A.Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin, “Attention Is All You Need”, in 31th NIPS Conference on Neural Information Processing System, 2017, pp. 59986008.
[19] Y. Chen, C. Long, G.Cong, C. Li, “Contextaware Deep Model for Joint Mobility and Time Prediction”,in 13th ACM WSDM Conference on Web Search and Data Mining, 2020, pp. 106114.
[20] S. Rendle, C. Freudenthaler, Z. Gantner, L. Thieme, “BPR: Bayesian Personalized Ranking from Implicit Feedback”, in 25th ACM UAI Conference on Uncertainty in Artificial Intelligence, 2009, pp. 452–461.
[21] E.Cho, S. Myers, J. Leskovec, “Friendship and Mobility: User Movement in LocationBased Social Networks”, in 17th ACM KDD Conference on Knowledge Discovery and Data Mining, 2011, pp. 1082–1090.
[22] P. Zhao, H. Zhu, Y. Liu, J. Xu, F. Zhuang, V. Sheng, X. Zhou, “Where to Go Next: A SpatioTemporal Gated Network for Next POI Recommendation”,. in 33th AAAI Conference on Artificial Intelligence, 2019, pp. 58775884.
[23] A M. Islam, M. M. Mohammad, S. S. Das, M. E. Ali, “A Survey on Deep Learning Based PointOfInterest (POI) Recommendations”, 2020, arXiv:2011.10187v1.
[24] C. Zheng, D. Tao, “AttentionBased Dynamic Preference Model for Next PointofInterest Recommendation”. in 15th Springer WASA Conference on Wireless Algorithms, Systems, and Applications, 2020, pp. 768–780.
[25] D. K. Bokde, Sh. Girase, D. Mukhopadhyay, “Role of Matrix Factorization Model in Collaborative Filtering Algorithm: A Survey”, International Journal of Advance Foundation and Research in Computer, vol.1, 2014, pp. 111118.
[26] M. Gan, L. Gao, “Discovering MemoryBased Preferences for POI Recommendation in LocationBased Social Networks”, International Journal of GeoInformation (IJGI), Vol.8, No.6, 2019, pp. 279294.
[27] X. Meng, J. Fang, “A Diverse and Personalized POI Recommendation Approach by Integrating GeoSocial Embedding Relations”, Journal of IEEE Access, Vol.8, 2020, pp. 226309 226323.
[28] Q. Yuan, G. Cong, Z. Ma, A. Sun, N. Thalmann, “Timeaware Pointofinterest Recommendation”, in 36th ACM SIGIR Conference on Research and development in Information Retrieval, 2013, pp. 363–372.
[29] P. Wang, H. Wang, H. Zhang, F. Lu, S. Wu, “A Hybrid Markov and LSTM Model for Indoor Location Prediction”, Journal of IEEE Access, Vol.7, 2019, pp. 185928 – 185940.
[30] J. Li, G. Liu, C. Yan, C. Jiang, “LORI: A LearningtoRankBased Integration Method of Location Recommendation”, IEEE Transactions on Computational Social Systems, Vol.6, No.3, 2019, pp. 430 – 440.
[31] L.Yao, Q. Z. Sheng, Y. Qin, X. Wang, A. Shemshadi, Q. He, “Contextaware PointofInterest Recommendation Using Tensor Factorization with Social Regularization”, in 38th.ACM SIGIR Conference on Research and Development in Information Retrieval, 2015, pp. 1007–1010.
[32] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, T. Chua, “Neural collaborative filtering”, in 26th ACM IW3C2Conference on World Wide Web Conference Committee, 2017, pp. 173182.
[33] Q. Liu, S.Wu, L. Wang, T. Tan, “Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts”, in 30th ACM AAAI Conference, 2016 , pp. 194–200.
[34] S. Kumar, M.I. Nezhurina, “An ensemble classification approach for prediction of user’s next location based on Twitter data”, Journal of Ambient Intelligence and Humanized Computing, Vol.10, No. 11, 2018, pp. 45034513.
[35] Q. Liu, S. Wu, D. Wang, Z. Li, L. Wang, “ContextAware Sequential Recommendation”, in ICDM Conference on Data Mining, IEEE, 2016, pp. 10531058.
[36] D. Bokde, S. Girase, D. Mukhopadhya, “Matrix Factorization Model in Collaborative Filtering Algorithms: A Survey”, Procedia Computer Science, Vol.49, 2015, pp. 136146.
[37] D. Lian, Y. Wu, Y. Ge, X. Xie, E. Chen, “GeographyAware Sequential Location Recommendation”, in 26th ACM ICGKDD Conference on Knowledge Discovery and Data Mining, 2020, pp. 2009–2019.
[38] K. Yang, J. Zhu, “Next POI Recommendation via Graph Embedding Representation from HDeepwalk on Hybrid Network”, Journal of IEEE Access, Vol 7, 2019, pp. 171105 – 171113.
[39] D. P. Kingma, J.L. Ba, “A Method for Stochastic Optimization”, in International Conference for Learning Representations, 2015, arXiv:1412.6980v.
http://jist.acecr.org ISSN 23221437 / EISSN:23452773 
Journal of Information Systems and Telecommunication

Developing A Contextual Combinational Approach for Predictive Analysis of Users Mobile Phone Trajectory Data in LBSNs 
Fatemeh Ghanaati1, Gholamhossein Ekbatanifard2*, Kamrad Khoshhal Roudposhti2

1. Department of Computer Engineering, Rasht Branch, Islamic Azad University, Rasht, Iran 2. Department of Computer Engineering, Lahijan branch, Islamic Azad University, Lahijan, Iran 
Received: 09 Dec 2021/ Revised: 12 Feb 2022/ Accepted: 22 March 2022 

Abstract
Today, smartphones, due to their ubiquity, have become indispensable in human daily life. Progress in the technology of mobile phones has recently resulted in the emergence of several popular services such as locationbased social networks (LBSNs) and predicting the next Point of Interest (POI), which is an important task in these services. The gathered trajectory data in LBSNs include various contextual information such as geographical and temporal contextual information (GTCI) that play a crucial role in the next POI recommendations. Various methods, including collaborating filtering (CF) and recurrent neural networks, incorporated the contextual information of the user’ trajectory data to predict the next POIs. CF methods do not consider the effect of sequential data on modeling, while the next POI prediction problem is inherently a time sequence problem. Although recurrent models have been proposed for sequential data modeling, they have limitations such as similarly considering the effect of contextual information. Nonetheless, they have a separate impact as well. In the current study, a geographical temporal contextual informationextended attention gated recurrent unit (GTCIEAGRU) architecture was proposed to separately consider the influence of geographical and temporal contextual information on the next POI recommendations. In this research, the GRU model was developed using three separate attention gates to consider the contextual information of the user trajectory data in the recurrent layer GTCIEAGRU architecture, including timestamp, geographical, and temporal contextual attention gates. Inspired by the assumption of the matrix factorization method in CF approaches, a ranked list of POI recommendations was provided for each user. Moreover, a comprehensive evaluation was conducted by utilizing largescale realworld datasets based on three LBSNs, including Gowalla, Brightkite, and Foursquare. The results revealed that the performance of GTCIEAGRU was higher than that of competitive baseline methods in terms of Acc@10, on average, by 42.11% in three datasets.
Keywords: LBSN; Trajectory Data; Contextual Information; GRU.
1 Introduction
Nowadays, people widely use locationbased social networks (LBSNs) and enjoy locationbased services (LBSs) using their mobile devices for sharing their locations with others by making checkins at locations or points of interests (POIs) that they have visited, including shops, museums, and restaurants [1]. The massive record of users’ checkin data provides a chance to conduct research on people’s mobility behaviors, in particular, for POI recommendation systems [2,3]. In addition, governments can use predictions about people’s future destinations and develop better transportation and scheduling strategies for alleviating traffic jams and handling crowd congestions [5,6,7,8]. Some geographical and temporal information exists in a user’s historical checkin sequence [4,9], having different effects on recommending the next POI. In this study, it was attempted to separately consider this contextual information to better train the proposed model. Human mobility is extremely complex and diverse; therefore, many previous studies were unable to simply determine the offering of the next POI recommendation [4,6]. Matrix factorization (MF) and other collaborative filtering (CF) techniques have widespread use for recommending a list of personally ranked POIs to the users [2]. Typically, approaches to MF include contextual information about the user. This helps provide valuable recommendations to users who lack enough historical checkins and is generally referred to as the coldstart problem. However, the employment of collaboration filtering (CF)based methods complicates the processing of sequence data and capturing of dynamic user’s preferences [2,6,11]. As a result, the ongoing challenges lie in the manner of integrating the information of different features to accurately model users’ complex behavioral preferences and then recommending reliable POIs [13].
Recurrent neural networks (RNNs) have recently been successfully applied to sequential recommender systems [1,4,8,15]. Thus, longterm dependencies can be captured by the hidden states of recurrent methods [4,16]. Many types of recurrentbased approaches have considered geographical and temporal factors to enhance the performance of POI recommendation algorithms [2,4,11,12,15]. Nonetheless, the present RNNbased POI recommendation methods face the alleviation of the coldstart problem [11]. In this regard, one of the excellent choices is to incorporate RNNbased POI recommendation methods with the MF method to enjoy the benefits of each one [2]. The user’s historical checkin behaviors do not significantly pose any problems in predicting the next behavior; hence, it is necessary to take only the important information into serious consideration [1,11]. Therefore, the attention mechanism (AM) has been proposed to deal with this challenge. The AM can enhance the capability of the neural network in capturing longterm dependencies and boost the ability to interpret neural networks [18]. In this study, the idea of the AM was used to address the most important contextual information.
11 Motivations
This study focused on the next POI recommendation through modeling checkin sequences and considering geographical and temporal contextual influences separately and proposed a novel geographical temporal contextual information extended attention gated recurrent unit (GTCIEAGRU) for the next POI recommendation. Among the recurrent models, the GRU model is highly simple and does not include many parameters in contrast to the longshort term memory (LSTM) model. In addition, this model can ignore the earlier unit hidden state, which is impossible with the traditional RNN [4,6]. Thus, a GRU network was developed to model checkin sequences while paying attention to geographical distances and time intervals between two successive checkins [19]. It is noteworthy that any piece of contextual information needs individual consideration during modeling since the effects of contextual information on user behavior are different [2,3]. Further, the GRU network was upgraded by inspiration from the AM to consider more important contextual information.
Furthermore, factorization approaches were employed, and the preference score was computed by the dot product. Following the prediction scores, it is possible to recommend topk POIs to a user, and there is a high chance that the user will go there if the score is higher. The Bayesian personalized ranking (BPR) framework [20] learned the parameters of GTCIEAGRU. In the last stage, three general datasets were utilized to conduct extensive experiments. Five uptodate POI recommendation methods were compared with Brightkite, Gowalla, and Foursquare to evaluate the model.
12 Main Contributions
1 The proposed architecture is presented by combining the development of the GRU model with the MF method, which aims to apply the strengths of the models and reduce the challenges of each of these methods. According to the MF method, in the CF approach, places visited on social networks by a user on social networks can affect the next POI of other users on those networks. However, CFbased approaches are weak in modeling sequential data and do not consider the effect of sequential data on modeling, while the next POI prediction problem is inherently a time sequence problem. Although recurrent models have been proposed for sequential data modeling, they have limitations. The traditional RNN model cannot integrate the corresponding checkin contextual information into the modeling. Newer recurrent models also consider the effect of temporal and spatial contextual information similarly, while they have a separate effect.
Therefore, there is a need to develop these models. In the recurrent layer of the proposed architecture, a development of the GRU model is presented using three attention gates that consider the contextual information separately and in terms of their importance.
2 Within the recurrent layer of the proposed architecture, the flexibility of the GRU model is employed, and the GRU model was expanded following the attentionbased approach. Moreover, three additional attention gates were proposed, including timestamp contextual attention gate (Gts), geographical contextual attention gate (Gge), and temporal contextual attention gate (Gte). The Gts controls the influence of timestamp earlier visited locations, whereas Gge and Gte control the effect of the hidden state of the earlier recurrent unit based on geographical distances and time intervals between two successive checkins, respectively. This innovation makes it possible to extend the model to another context.
3 In this research, user contextual information is classified into two categories of absolute and transitional content information. The first category includes checkin timestamp and geographical coordinates and the second one consists of the time interval and geographical distance between two successive consecutive checkins. Our proposed architecture considers two types of absolute and transitional contextual information separately. This category focuses on developing a model to consider more contextual information in the future.
4. Some comprehensive experiments were conducted on three largescale realworld datasets, namely, Brightkite, Gowalla [21], and Foursquare [14] that are widely used in related studies to predict the user POI in LBSNs. The aim was to show the effectiveness of the proposed GTCIEAGRU architecture for the next POI recommendation.
13 Problem Statement
Human mobility prediction is important for a wide spectrum of LBSN applications, and the next POI recommendation is one of the usages of predicting people’s mobility [1]. In some LSBNs, users share their location by registering checkins. The checkins gathered in LBSNs contain geographical and temporal contextual information (TCI), and each piece of information has a separate effect on predicting the user’s next location [3, 8]. In previous studies, some restrictions were applied for dividing sequence into different checkin trajectories such as using the time interval of less than six hours [1]. Nonetheless, applying restrictions for the time interval and geographical distance, when considering registered checkins in data preprocessing, is not a proper approach for the mentioned purpose. The AM can address the mentioned issue. Instead of using multiple assumptions to consider the time interval or geographical distance constraints between two checkins, it can be addressed by automatic weighting given to the model inputs inspired by the AM.
According to evidence [2], CFbased approaches have weaknesses in sequential data modeling and fail to consider the effect of sequential data, while the problem of the next location prediction is inherently a matter of time sequence (Challenge 1). Traditional recurrent models are unable to consider contextual information, but this information is highly important in determining the next POI (Challenge 2). Meanwhile, some earlier studies, based on recurrent models, consider the effect of temporal and geographical contextual information (GCI) to be the same, while they have a different effect (Challenge 3). Furthermore, according to [11], some proposed architectures, which are a combination of recurrent models and AM, are highly complex (Challenge 4). In this work, the GTCIEAGRU model was proposed to address the abovementioned challenges.
14 Organizations
The remaining parts of this research are as follows: The related methods are briefly reviewed in Section 2. Sections 3 and 4 describe some preliminaries to the study and the details of the GTCIEAGRU network, respectively. In Section 5, an illustration of the experiments is presented, followed by providing the results of the proposed method. Finally, Section 6 summarizes conclusions and an outline for future works.
2 Related Works
This section classifies related studies under three approaches generally used for the next POI recommendations, including CF, RNN, and AM. Table 1 provides a summary of related works with their challenges considered in our research.
Table 1. summarize of related works
Model Name  Model Approach  Method summery  challenges 

[28] Unified method  CF based  Believing that time plays an important role in POI recommendations and defining a new problem, namely, the timeaware POI recommendation to recommend POIs for a given user at a specified time in a day  Focusing on temporal contextual information and paying less attention to geographic contextual information 
[30] LORI  CF based  Applying a confidence coefficient for each user in the integration process and designing a learningtorank based algorithm to train confidence coefficients  Not taking into consideration time interval and geographical distance 
[33] STRNN  RNN based  Extending RNN and using a transition matrix for capturing the temporal cyclic effect and geographical influence  Vanishing gradient problem in long sequence due to the use of the traditional RNN 
[22] STGN  RNN based  Modifying the basic LSTM model slightly by introducing gates and cells to capture short and longterm preferences  Considering the same effect for temporal and geographical contextual information 
[8] SERM  RNN based  Jointly learning the embedding of multiple factors (user, location, time, and keywords) and the transition parameters of an RNN in a unified framework  Not taking into account the geographical distance in the training of this model 
[35] CARNN  RNN based  Employing adaptive contextspecific input matrices and adaptive contextspecific transition matrices  Using a traditional RNN model and restrictions on paying attention to the contextual information, low performance 
[1] ATSTLSTM  AM and RNN based  Developing an attentionbased spatiotemporal LSTM network to focus on the relevant historical checkin records in a checkin sequence selectively using the spatiotemporal contextual information  Encountering with high complexity of implementation and a lack of attention to the scarcity 
[6] Deep Move  AM and RNN based  Capturing complex dependencies and multilevel periodicity nature of humans using embedding, GRU, and AM  Not taking into account the time interval between two checks to model the behavioral pattern of user checkins 
[11] DANSNR  AM based  Makes use of the selfAM. By leveraging multihead selfattention, the DANSNR can model longrange dependencies between any two historical checkins efficiently and weigh their contributions to the next destination adaptively  Using only the attention mechanism and had low performance rather than applying recurrent neural networks for modeling the sequential influence and social influence 
3 Preliminaries
The research problem is formulated, and the applied preliminaries in this study are presented in the following section.
31 Notations and Definitions
Table 2 presents some primary notations used in this study.
Definition 1 (Checkin): A checkin is an action that a user takes under a geographical and temporal context. In addition, it is a registration of a location in the LBSN that contains geographical and temporal information. When a user u checks in a location l (including latitude and longitude) with venueId v at the timestamp t, the checkin record can be modeled as a quadruple: cu, v, t < u, l, v, t >.
Definition 2 (Checkin sequence): A user’s checkin sequence or Su is a set of all user checkins.
Definition 3 (Trajectory): Given a user u, a trajectory t is a sequence of chronologically ordered checkin associated with u. For example tru: < u, l1, v1, t1 >,…,< u, li, vi, ti >,…,< u, lk, vk, tk >, where tru is the trajectory of a user u before time tk. Here, a trajectory set Tr (u) is used to denote all the trajectories of user u.
Definition 4 (POI): In LBSNs, a POI is a spatial item related to a geographical location and known as a venue, including a hotel or an office. In this research, POI is represented by v, and the set of POIs is demonstrated as V={v1, v2,...}. Each POI v has a unique identifier and geographical coordinate, consisting of geographical latitude and geographical longitude.
Definition 5 (the next POI recommendation): Given all users’ trajectories, the aim of the next POI recommendation is to predict the most likely location 𝑣𝑘 that a user 𝑢 will visit at a certain time point 𝑡𝑁+1.
Definition 6 (POI recommendations): Given a set of users’ checkin sequences Su and a set of POIs V, the POI recommendation task is to recommend topk POIs that are preferable for user u.
Table 2: Notations and descriptions used in this study
32 MF in CF Based Approach
CFbased methods aim to discover similarities in the user’s previous behavior and make predictions to the user based on a similar preference with other users [25]. There are various modelbased CF algorithms, but MF is the most commonly applied in recommender systems [2]. MF seems to be the most accurate approach for lowering the problem from high levels of scarcity in the recommender systems database. Generally, MF models map both users and items to a joint latent factor space of dimensionality d in such a way that useritem interactions are modeled as inner products in that space. In the next POI recommendation, the item is the same POI or venue that a user has selected at the time of the checkin.
Accordingly, each venue v is related to a vector qv∈ Rd, and each user u is associated with a vector pu ∈ Rd. For a given venue v, the elements of qv measure the extent to which the venue possesses those factors, positive or negative. For a given user u, the elements of pu measure the extent of interest the user has in venues that are high on the corresponding factors, positive or negative. The resulting dot product (qvTpu) captures the interaction between user u and venue vthe user u’s overall interest in the venue’s characteristics. This approximates the user’s rating of venue v, which is denoted by ruv, leading to the following estimate [36]:
The objective is to minimize the prediction error or the loss function in Eq. (2) where K is the set of (u,v) pairs of known ratings [25, 36].
(2)
Different approaches exist [13,36] for the extension of MF using RNN models to capture the user’s dynamic preferences from the sequence of user’s checkins. Specifically, with respect to the sequence of a user’s checkins, the output of an RNN model can be effective in representing a user’s dynamic preferences and modifying MFbased approaches.
33 GRU in RNN Based Approach
The next POI recommendation is immediately faced with the challenge of learning personalized user preferences for POIs and the sequential correlations jointly and efficiently between the checkins [1]. To solve this problem, the RNN takes a sequence of inputs and learns the sequential pattern of the input sequence using hidden states [2,3,8]. The problem that the RNN faces is the exploding and vanishing gradients; therefore, it cannot capture longterm preferences [2,11]. The problems can be solved by longshort term memory (LSTM), which employs a gate mechanism and can capture longterm preferences [1,23].
The information flow among consecutive LSTM cells is controlled through input, forget, and output gates. LSTM resolves the problems of the RNN, but it has three gates thus the training of an LSTMbased model is slower and requires a large amount of training data. GRU [4,6,23,40] has updated and reset gates in the network, dealing with the update degree of each hidden state. In fact, it determines which information should pass to the next state [2,3]. Fig. 1 displays the block diagram of basic GRU. As shown, GRU uses only two gates (i.e., reset and update gates). The GRUbased model can be trained faster and perform better compared to LSTM when there are less training data. GRU calculates hidden state hτ at time τ from the output of update gate zτ, reset gate rτ, current input xτ, and previous hidden state hτ−1. τ and hτ are computed from the reset gate as follows:
zτ = (Wz xτ + Uz hτ−1 + bz) (3)
rτ = (Wr xτ + Ur hτ−1 + br) (4)
τ = tanh (W xτ+U (rτ hτ−1) + bh) (5)
hτ =(1 zτ) hτ−1+ zτ τ (6)
where ⊙ is a basic multiplication operation, and W and U represent weight matrices for training the network.
Fig. 1. An illustration of the GRU cell including two gates.[3]
34 Attention Mechanism
The AM was proposed based on the selective AM in the human visual system [1,18]. It should be noted that humans are prone to giving higher attention to key parts of the input, helping in breaking down a complex input into simpler parts that can easily be processed accordingly. Rather than paying attention to all available information, selective attention mainly focuses on the most relevant information in a system. Accordingly, learning to pay attention to the specific components of the input data resulted in different attention models in deep learning [4,11].
The present study proposed a novel model that is applied to this mechanism for the next location prediction. The key idea in the AM is that inputs are mapped to query, key, and value vectors. The outputs are calculated by taking the weighted sum of the value vectors where weights are determined by a function of query and key values [11]. Specifically, the attention function presents a query and a group of keyvalue pairs to a context vector, which is a weighted sum of all values. The queries, keys, and values are merged as 𝑄, 𝐾, and 𝑉𝑣𝑎𝑙 matrices, respectively [1,18]. For the output of the attention function, an alignment function or the compatibility function, which measures the quality of the match between the input query matches and the corresponding key, calculates the weight assigned to each value. Eq. (7) is used for the computation of the matrix of outputs where (𝑄, 𝐾) refers to the attention function [18]:
Attention (Q, K, Vval) = Softmax (f(Q,K))Vval (7)
Additive attention and dotproduct (multiplicative) attention are two of the most commonly used attention functions and are defined as follows [1]:
fadd(Q,k)=tanh(wQ+WK k) (8)
fmul(Q,k)= Q kT (9)
In theory, these two functions are similar in computation complexity. Additive attention and dotproduct attention use a feedforward neural network with a single hidden layer for the calculation and optimized matrix multiplication operation, respectively [1]. The present project, which is inspired by previous studies [2,3], employed a feedforward neural network to calculate the alignment function to develop the GRU model.
4 Proposed GTCIEAGRU Model Description
The GTCIEAGRU architecture consists of input, embedding, recurrent, and output layers. Fig. 2 presents a schematic of our purpose architecture. The details of these layers and the learning procedure for the parameters are provided as follows:
41 Input Layer
The input layer contains model inputs that include absolute context and relative or transition context. In the proposed model, the absolute context is user id, timestamp, geographical coordinates (including latitude and longitude), and venue id. Further, the relative context (also called the transition context) is the time interval (Δt) and geographical distance (Δg) between two successive checkins.
The geographical distance and time intervals are calculated in the input layer. For a given user u, venue vn, and time tτ , the geographical distance (Δgτ) and time interval (Δtτ) between the POIs at current time tτ and previous time tτ −1, as well as the given venue vn and venue vn1 previously visited at time are computed as: Δtτ = tτ − tτ −1 and Δgτ = dist (lat v1, lng v1, lat v2, lng v2), respectively, where dist () is the Haversine and its function is as Eq.(10)^{1}. It should be noted that the Haversine distance is the angular distance between two points on the surface of a sphere. The former coordinate of each point is taken as the latitude and the latter one is the longitude given in radians. The data dimension must be two^{2}.
D(x,y)=2arcsin
[] (10)
Fig. 2. GTCI EAGRU architecture with input, embedding, recurrent, and output layers
Note. GTCI EAGRU: Geographical temporal contextual information Extended attention gated recurrent unit; LBSN: Locationbased social network. In the recurrent layer, the GRU model is extended with three additional attention gates, and a ranked list of the next POI recommendations is provided in the output layer.
42 Embedding Layer
This layer is for embedding inputs from the checkin sequence before it goes to the recurrent layer. In this layer, embedding or latent factors are generated from the inputs. In addition, the latent factors of the user, namely, POI (or venue) and time are generated as ui ∈ U, vτi ∈ V and time ∅tτ ∈ T, respectively. Note that θe = {U, V, T} denotes the set of the parameters of the embedding layer. Next, the latent factors of venue vτj, the latent factors of the given time tτ, and the contextual transition features (Δgτ and Δtτ) are passed to the recurrent layer for training using GTCIEAGRU.
43 Recurrent Layer
In this layer, the GRU model was developed with three attention gates. Following Manotumruksa et al. [2] and Kala et al. [3], this study presented timestamp attention gate (Gts), geographical attention gate (Gge), and temporal attention gate (Gte). The input of Gts is the checkin time (i.e., the time that the checkin is registered by a user in LBSN and includes the year, month, day, hour, minute, and second). This gate is used to specify a more important timestamp in the sequence of historical checkins of a user. However, the input of Gte is the time interval (Δt) between two successive checkins used to specify more important time intervals in the sequence of historical checkins of a user. The input of Gge is the geographical distance (Δg) between two successive checkins applied to specify a more important geographical distance in the sequence of historical checkins of a user. The output of this layer is the hidden state of the recurrent unit at time step τ, hτ, and is defined as Eq. (11):
hτ = f (vjτ,tτ, Δtτ, Δgτ ; θr ) (11)
As mentioned earlier, the purposed model treats the absolute and relevant (or) transition contextual information separately. It is noteworthy that this contextual information has a different effect on the user’s dynamic preference and requires independent consideration. The following part describes the extension of the traditional GRU for the integration of absolute and relevant contextual information.
Generally, in the GRU model, given the user’s sequence of checkins Su and dynamic preference at time step τ, the hidden state (hτ) is estimated by the update and reset gates, which are defined as:
z τ =(Wz vjτ+ Uz hτ −1 + bz) (12)
r τ =(Wr vjτ+ Urhτ −1 + br) (13)
τ = tanh(Wh vjτ+Uh (rτ hτ−1) + bh) (14)
hτ =(1 zτ) hτ−1+ zτ τ (15)
where vτj represents the latent factor of venue j that user i visited at time step .
() and tanh() are the sigmoid and hyperbolic tangent functions, respectively. Furthermore, U is a recurrent connection weight matrix that captures sequential signals between every two adjacent hidden states hτ and hτ −1 using , which shows the elementwise product. Moreover, W and b are the transition matrix between the latent factors of venues and the corresponding bias, respectively. It should be noted that θr = {W, U, b} denotes the set of the parameters of the recurrent layer. Overall, W is the transition matrix between the latent factors of venues and b indicates the corresponding bias. Additionally, U is a recurrent connection weight matrix that captures sequential signals between every two adjacent hidden states. All the recurrent layer parameters (i.e., Wz, Uz, and bz) are the set of the parameters of the update gate. Wr, Ur, and br, as well as Wh, Uh, and bh are the set of parameters of the reset gate and candidate hidden state, respectively. Similarly, WGts, UGts, and bGts are the set of the parameters of our proposed Gts. Finally, WGge, UGge, and bGge, as well as WGte, UGte, and bGte are the set of the parameters of our proposed Gge and Gte, respectively.
At current step , the correlation between the latent factor of absolute contexts ϕt τ and the hidden state from the earlier step h τ1 is calculated by Eq. (16):
Gts= (WGts h τ1 + WGts ϕt τ + bGts) (16)
To effectively model the users’ sequential order of checkins, the relevant contextual information needs to be examined separately. To address this issue, the current study proposed Gge and Gte to individually incorporate the geographical distance (Δgτ) and time interval (Δtτ) between two checkins as Eqs. (17) and (18):
Gge= (WGge h τ1+ WGge Δgτ + bge (17)
Gte= (WGte h τ1+ WGte Δtτ + bte) (18)
With the proposed gates for GTCIEAGRU architecture, the equations of the traditional GRU are updated as Eqs. (19), (20), and (21):
z τ =(Wz vτj+ Uz hτ −1 + Wz((Gts ϕt τ ) +
(Gge Δgτ ) + (Gte Δtτ ))+bz) (19)
r τ =(Wr vτj+ Urhτ −1 + Wr((Gts ϕt τ ) +
(Gge Δgτ ) + (Gte Δtτ )) +br) (20)
τ = tanh(W vτj +U (rτ hτ−1) + Wr((Gts ϕt τ ) + (Gge Δgτ ) + (Gte Δtτ )) + bh) (21)
In the following section, the hidden sate hτ will be updated and as previously mentioned, it will be the output of the recurrent unit at time step τ.
44 Output Layer
In the next POI recommendations based on the MF approach, recommendations are mainly derived from a dot product of the latent factors of users U ∈ RU×d and venues V ∈ RV×d where d is the number of latent dimensions (i.e. i,j = ui vjT ) and ui and vj denote the latent factors of user i and venue j, respectively [2,36]. In the output layer, the preference of user u on venue v at timestamp t is estimated using Eq. (22):
u,v,t=uu hτ T (22)
According to previous works, the pairwise loss function outperformed the classification loss function in learning patterns from sequential data and was more efficient for the network training of the recurrentbased recommendation [2,3,13,20]. Therefore, following Manotumruksa et al. [2,13], the pairwise BPR [20] can be applied to estimate the embedding and recurrent layer parameters and the probability distribution over all venues given the hidden state hτ.
45 Network training
This study employed datasets consisting of a set of sampled triplets each containing one user and a pair of POIs in which one POI is positive (known as visited) while the other one is negative (known as unvisited). As mentioned earlier, this study applied the pairwise BPR to learn the embedding and recurrent layer parameters ( = {e,r}). Based on an underlying assumption, stating that a user prefers the observed POI to all unobserved ones, BPR considers the relative order of the predictions for the pairs of POIs [1,4]. At each sequential position k in the BPR framework, the goal of GTCIEAGRU is to maximize the following probability [1,4,20]:
P(u, t, v > ) =g( ou,t,v – ou,t, v’) (23)
𝑣 and 𝑣′ stand for a positive (visited) POI and a negative (unvisited) POI, respectively, and 𝑔(∙) represents a nonlinear function defined by Eq. (24) as [1, 20]:
(24)
The objective function of the network for the next POI recommendation can be solved by integrating the loss function and a regularization term as follows [20]:
J=+  2 (25)
where 𝜆 is used to specify the power of regularization and is the parameter set. The dimension of the latent factors d and hidden layers hτ of GTCIEAGRU architecture d = 10 across three datasets can be set based on methods by Manotumruksa et al. [2] and Kala et al. [3], and all embedding and recurrent layers’ parameters can be randomly be initiated with a Gaussian distribution. Initially, the learning rate and the batch size are set to 0.001 and 256, respectively. An Adam optimizer was employed to optimize the model parameters. The output of the GTCIEAGRU model is a set of scores for POIs, similar to their likelihood of being the next POI in each sequence. A summary of the learning algorithm of GTCIEAGRU is provided as follows:
Notations  Descriptions 

u, l, v, & t  User, location (including latitude and longitude), venue or POI, and timestamp 
cu, v, & t  A checkin recorded by user u in POI v and timestamp t 
lat v & lng v  Latitude and longitude of POI v (i.e., geographical coordinates of POI v) 
Δt & Δg  Time interval and geographical distance between two successive checkins 
Su  A set of all checkins generated by user u 
Us, V, & T  Sets of users, POIs, and timestamp 
vτu  POI visited by user u at time step 
gτu & tτu  Vector representations of geographical and temporal intervals 
tru  A sequence of chronologically ordered checkins related to u 
Tr (u)  All trajectories from user u 
ϕu  The latent factor of user u 
ϕv  The latent factor of POI v 
ϕt  The latent factor of timestamp t 
d  The number of latent dimensions 
v+  A set of positive POIs (visited venues) for each user uUs 
v  A set of positive negative POIs (unvisited venues) for each user uUs 
 Sigmoid function 
5 Experimental Result and Analysis
This section presents the experimental setup and empirical results of this study. Empirical experiments are conducted on three public datasets in LBSNs for validating the efficiency of the proposed method. To address the challenges made in Section 24, the experiments are designed for the following research questions:
RQ1: How can the basic GRU architecture be extended to separately consider the absolute and relative (or transition) contextual information associated with the sequence of checkins?
RQ2: Is it important to model absolute and relative (or transition) contextual information separately?
RQ3: Does GTCIEAGRU that leverages multiple types of contextual information improve prediction accuracy by applying additional attention gates? Or, does it outperform the previous methods?
51 Datasets and Experimental Settings
The experiments were conducted for evaluating three publicly LBSN datasets (i.e., BrightKite^{3}, Gowalla^{4}, and Foursquare^{5} datasets). Following Manotumruksa et al. [2] some deletions were made to lessen data sparsity and cold start problems. Users with less than 10 checkins and POIs with less than 10 were eliminated from the three datasets. Table 3 presents an overview of the statistics of the three datasets. In this study, a checkin record is a quadruple composed of a user, the corresponding checkin timestamp, the geographical coordinates of the checkin, and a location Id or POI. The checkin records in these three datasets were regarded as user sequences. The density calculation formula for three datasets is as follows [38]:
(26)
Table 3. Statistics of the three datasets
Algorithm 1: Training of GTCIEAGRU  
Input: Set of users 𝑈s and set of historical checkin sequences Su  
Output: GTCIEAGRU model { 𝛩 }  
//construct training instances  
1.  Initialize D=Usu Du = ∅ Du is a set of checkin trajectory samples combined with negative POIs of 𝑢 
2.  For each user 𝑢 𝑈s do 
3.  For each checkin sequence Su = {st1u , st2u ,…, stnu } do 
4.  Get the set of negative samples 
5.  For each checkin activity in Su do 
6.  Compute the embedded vector vτu 
7.  Compute the geographical contexts vector gτu 
8.  Compute the temporal contexts vector tτu 
9.  End for 
10.  Add a training instance ({vτu , gτu , tτu },{}) into Du 
11.  End for 
12.  End for 
//train the model  
13.  Initialize the parameter set 𝛩 
14.  While (exceed(maximum number of iterations)==FALSE) do 
15.  For each user 𝑢 in 𝑈 do 
16.  Randomly select a batch of instances 𝐷𝑏𝑢 from 𝐷𝑢 
17.  Find 𝛩 minimizing the objective (23) with 𝐷𝑏𝑢 
18.  End for 
19.  End While 
20.  Return the set of parameter 𝛩 
Dataset  #Users  #Checkins  #POIs  Density 
Brightkite  915  676721  7527  0.0982 
Gowalla  1047  614340  5011  0.1170 
Foursqure  615  108195  19245  0.0091 
A leaveoneout evaluation method was adopted to evaluate the efficiency of the proposed GTCIEAGRU architecture based on earlier works [2], [3]. Each user’s most recent checkin was taken as the base, and 100 POIs, which had not been visited before, were randomly selected for this purpose. They were the testing set, and the other remaining checkins were considered as the training set. The task of the GTCIEAGRU was to rank those 100 venues for each user as their preferred contexts (i.e., timestamp, time interval, and geographical distance), aiming at ranking highest the recent, ground truth checkin. Following Manotumruksa et al. [2] and Kala et al. [3], the researchers set the dimension of the latent factors d and hidden layers hτ of the proposed GTCIEAGRU architecture: d = 10. As mentioned before, Gaussian distribution [32] was employed for the random initialization of the recurrent layer’s parameters, and Adam Optimizer [39] was utilized for optimizing the parameters because it had a faster convergence compared to the stochastic gradient descent optimization, which automatically adjusts the learning rate for each iteration. In addition, the batch size and the dropout rate were set to 256 and 0.2, respectively, to prevent overfitting.
52 Comparison
The following five uptodate methods were compared to validate the efficiency of the GTCIEAGRU in the next POI recommendation task. Table 4 summarizes these methods into different aspects. Based on data, they are categorized into MF, RNN, and AMbased approaches. The compared models are also classified according to the use of GCI and TCI.
A brief description of these models is given below:
STGN: Spatiotemporal gated network was proposed by Zhao et al. [22], and improved the LSTM network, in which STGs are introduced for capturing the Spatiotemporal relationships between successive checkins. By introducing new gates and cells to capture short and longterm preferences, STGN modified the basic LSTM model.
ARNN: An attentional RNN was proposed by Guo et al. [7] to jointly model the transition regularities and sequential regularity of similar locations (neighbors). Using embedding, knowledge graph, LSTM, and AM, the ARNN captured sequential, spatial, temporal, and semantic influences.
GeoSAN: By introducing a new loss function, Lian et al. [37] resolved the sparsity issue. GeoSAN represents the hierarchical gridding of each GPS point with a selfattention based geography encoder for better use of geographical information.
DRCF: To benefit from the traditional RNN to model the sequential order of users’ checkins, Manotumruksa et al. [13] extended NeuMF. DRCF has two components each having its recurrent layer.
CARA: By employing embedding, GRU, and two gating mechanisms, Manotumruksa et al. [2] captured various types of the impact of different contextual information.
Following earlier works [68,22], the current study used prediction accuracy (Acc@k, k = 10) for evaluating the performance of the abovementioned methods and checking if the groundtruth location can be found in the topk recommendation list. Generally, the Accuracy@ is defined by Eq. (27) as follows [29]:
(27)
Table 4. Summary of all the baseline methods used in this study
Note. MF; Matrix factorization; RNN: Recurrent neural network; AM: Attention mechanism; GCI: Geographical contextual information; TCI: Temporal contextual information; STGN: Spatiotemporal gated network; ARNN: Attentional Recurrent Neural Network; GeoSAN: Geographyaware sequential recommender based on the SelfAttention Network; DRCF: Deep Recurrent Collaborative Filtering; CARA: Contextual attention recurrent architecture ; GTCIEAGRU: Geographical temporal contextual informationextended attention gated recurrent unit.
53 Results and Discussion
Table 5 compares the recommendation results of six methods on the three datasets. The numbers in bold in each column represent the best performance.
Table 5. Comparison of different methods in recommendation performance
Methods  Approaches and Contextual Information  
MF  RNN  AM  GCI  TCI  
STGN  × 
 × 

 
ARNN  × 



 
GeoSAN  ×  × 


 
DRCF 

 ×  ×  ×  
CARA 




 
GTCIEAGRU 





Methods  Acc@10  
Brightkite  Gowalla  Foursquare  
STGN  0.2020  0.5231  0.3017 
ARNN    0.2336  0.4285 
GeoSAN  0.6425  0.6028  0.4867 
DRCF  0.7363    0.8805 
CARA  0.7385    0.8851 
GTCIEAGRU  0.9751  0.9606  0.8901 
Note.: STGN: Spatiotemporal gated network; ARNN: Attentional Recurrent Neural Network; GeoSAN: Geographyaware sequential recommender based on the SelfAttention Network; DRCF: Deep Recurrent Collaborative Filtering; CARA: Contextual attention recurrent architecture; GTCIEAGRU: Geographical temporal contextual informationextended attention gated recurrent unit.
The comparison of the experimental results of the models demonstrated that the use of AM alone (i.e., the GeoSAN model) has not increased prediction accuracy. Moreover, the experimental results of other previous studies (e.g., DANSNR) revealed lower evaluation metrics values. Although the STGN model separately considered the GCI and TCI, it did not use the attenuation mechanism approach. It applied the LSTM model and was less prediction accurate compared to models that employed the GRU such as CARA. Although the ARNN model applied the LSTM model, it had a higher accuracy prediction in comparison with the STGN model due to the use of the attenuation mechanism. Similar to the STGN, it had less prediction accuracy compared to models that considered the GRU model.
The GeoSAN model only uses the AM for location recommendation, and despite considering geographical and TCI, it is less prediction accurate than DRCF and CARA models. The DRCF model pays attention to the sequence of previously visited venues while not taking into consideration the contextual information related to the checkins. Thus, its prediction accuracy is lower than that of the hybrid models. However, it should be stated that the performance of these hybrid approaches was not worse than that of RNN and LSTM. Thus, it is worth modeling geographical and spatial contextual information for the task of the next POI recommendations. It means that it is insufficient to have a good network architecture, but more geographical and spatial contextual information of human checkin behaviors should be taken into account to obtain excellent results [1]. This is the reason for the outperformance of CARA over DRCF.
The accuracy prediction in the CARA model is higher compared to other models due to the separate use of TCI and GCI and a combination of the RNN, attention, and factoring approaches. Inspired by the idea behind this model, the researchers introduced a new initiative to employ three gates in the GRU model to address GCI and TCI to better predict the accuracy of the next POI recommendation. As mentioned in previous sections, the proposed model uses three separate attention gates, namely, Gts, Gge, and Gte, which consider the timestamp, geographical distance, and time interval between successive checkins, respectively, and the output of each of them separately affects the values of the reset and update gates of the GRU model. As depicted in Fig. 3, the experiment results of the proposed models indicate that it has achieved this goal, and the accuracy prediction has been improved in the proposed GTCIEAGRU architecture.
Fig. 3. Comparison of GTCIEAGRU with baseline methods in terms of Accuracy@10 on three datasets
To answer RQ1 for the development of the GRU model, it should be mentioned that three gates were introduced and implemented as a feedforward network. The output of these gates affects the values of the GRU reset and update gates, and they are responsible for controlling the geographical and temporal information of the user’s trajectory data. To answer RQ2 and RQ3, these results were obtained (Table 6) by comparing the accuracy prediction of the GTCIEAGRU model with uptodate architectures.
Table 6. Percentage of Improvement of GTCIEAGRU
Methods  Percentage of Improvement  
Brightkite (%)  Gowalla (%)  Foursquare (%)  
STGN  79.28  45.54  66.10  
ARNN    75.68  51.86  
GeoSAN  34.11  37.25  45.32  
DRCF  24.49    01.08  
CARA  24.26    00.56  
Improvement  40.54  52.82  32.98  
On Average in three Dataset 

6 Conclusions
In recent years, the next POI recommendation is of great importance for a wide spectrum of LBSN applications. The influences of contextual information (e.g., spatial and temporal context information) are crucial for analyzing individual behaviors for personalized POI recommendations. Hence, many studies have considered this contextual information to improve the performance of POI recommendation algorithms such as the CF and RNN. There are still many challenges regarding how to integrate contextual information to accurately model users’ complex behavioral preferences and recommend reliable POIs to users.
The current study proposed a novel GTCIEAGRU for the next POI recommendation by addressing the challenges concerning previous studies. Our proposed architecture was presented with the development of the GRU model, in which the contextual information of the user trajectory data is considered separately. Moreover, the development of the model inspired by the AM makes contextual information more important in modeling sequential user data. POIs were scored to provide recommendations to a user from her/his historical checkins. The simple development of this model for considering more contextual information is one of the other features of the proposed model.
By comparing the experimental results of baseline methods, an increase in the accuracy of prediction indicates the importance of considering contextual information separately. The proposed GTCIEAGRU architecture with three additional contextual attention gates worked well for the next POI recommendation.
In this study, the comprehensive experiments conducted on three large scale datasets from the Brightkite, Gowalla, and Foursquare demonstrated a significant improvement in the GTCIEAGRU architecture for the next POI recommendations compared with various uptodate recurrent architectures and many different recent factorization approaches.
To enhance the quality of recommendations for the next POI, the GTCIEAGRU architecture could be enriched by adding the impact of each user’s social relationships with other users on LBSNs. Furthermore, it can be possible to include more contextual information (e.g., visual and text information) related to users’ checkins or the weather condition of the checkin registration location as well.
Acknowledgment
This manuscript is prepared based on PhD. thesis of the first author at Rasht Branch, Islamic Azad University, Rasht, Iran.
References
[1] L. Huang, Y. Ma, Sh. Wang, Y. Liu, “An Attentionbased Spatiotemporal LSTM Network for Next POI Recommendation”, Journal of IEEE Transactions on Services Computing, vol. 12, 2019, pp. 113.
[2] J. Manotumruksa, C. Macdonald, I. Ounis, “A Contextual Attention Recurrent Architecture for Context Aware Venue Recommendation”, in 18th ACM SIGIR Conference on Research and Development in Information Retrieval, 2018, pp. 555564.
[3] K. Kala, M. Nandhini,” Context Category Specific sequence aware Point of Interest Recommender System with Multi Gated Recurrent Unit”, Journal of Ambient Intelligence and Humanized Computing, 2019, https://doi.org/10.1007/s1265201901583w.
[4] C. Liu, J. Liu, J. Wang, S. Xu, H. Han, Y. Chen, “An AttentionBased Spatiotemporal Gated Recurrent Unit Network for PointofInterest Recommendation”, International Journal of GeoInformation, vol. 8, No. 8, 2019, pp.355373.
[5] S. Wang, Z. Bao, J. Culpepper, G. Cong, “A Survey on Trajectory Data Management, Analytics, and Learning”, ACM Computing Surveys, vol. 54, No.3, 2020, pp. 133.
[6] J. Feng Y. Li, C. Zhang, F. Sun, F. Meng, A. Guo, D. Jin, “DeepMove: Predicting Human Mobility with Attentional Recurrent Networks”, in 18th ACM IW3C2 Conference on International World Wide Web, 2018, pp. 14591468.
[7] Q. Guo, Z .Sun, J. Zhang, Y. Theng, “An Attentional Recurrent Neural Network for Personalized Next Location Recommendation”, in 34th ACM AAAI Conference on Artificial Intelligence, 2020, pp. 8390.
[8] D.Yao,C. Zhang, J. Huang, J. Bi, “SERM: A Recurrent Model for Next Location Prediction in Semantic Trajectories”, in 17th ACM CIKM Conference on Information and Knowledge Management, 2017 ,pp. 24112414.
[9] L. Zhang, Z. Sun, J. Zhang, H. Kloeden, F. Klanner, “Modeling hierarchical category transition for next POI recommendation with uncertain checkins”, Journal of Information Sciences, Elsevier, vol.515, 2019, pp. 169190.
[10] L. Chang, W. Chen, J. Huang, Ch. Bin, W. Wang, “Exploiting multiattention network with contextual influence for pointofinterest recommendation”, Journal of Applied Intelligence, vol. 51, 2021, pp. 1904–1917.
[11] L. Huang, Y. Ma, Y. Liu, K. He, “DANSNR: A Deep Attentive Network for SocialAware Next PointofInterest Recommendation”, Journal of ACM Transactions on Internet Technology, Vol.21, No.2, 2020, pp. 1–27.
[12] G. Christoforidis, P. Kefalas, A. Papadopoulos, Y. Manolopoulos, “RELINE: PointofInterest Recommendations using Multiple Network Embeddings”, Journal of Knowledge and Information Systems, Vol. 63, No.4, 2019, pp. 791817.
[13] J. Manotumruksa, C. Macdonald, I. Ounis, “A Deep Recurrent Collaborative Filtering Framework for Venue Recommendation”, in 17th ACM CIKM Conference on Information and Knowledge Management, 2017, pp. 14291438.
[14] D. Yang, D Zhang, V. Zheng, Z. Yu, “Modeling User Activity Preference by Leveraging User Spatial Temporal Characteristics in LBSNs”, Journal of IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol.45, No.1, 2014, pp. 129 – 142.
[15] M. Quadrana, P. Cremonesi, D. Jannach, “SequenceAware Recommender Systems”, Journal of ACM Computing Surveys, Vol.51, No.4, 201, pp. 1–36.
[16] Q. Cui, Y. Tang, S. Wu, L. Wang, “Distance2Pre: Personalized Spatial Preference for Next PointofInterest Prediction”, in PAKDD Conference on Knowledge Discovery and Data Mining, 2019, pp. 289301.
[17] Q. Gao, F. Zhou, G. Trajcevski, K. Zhang, T. Zhong, F. Zhang, “Predicting Human Mobility via Variational Attention”, in IW3C2 Conference on International World Wide Web Conference Committee, 2019, pp. 2750–2756.
[18] A.Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin, “Attention Is All You Need”, in 31th NIPS Conference on Neural Information Processing System, 2017, pp. 59986008.
[19] Y. Chen, C. Long, G.Cong, C. Li, “Contextaware Deep Model for Joint Mobility and Time Prediction”,in 13th ACM WSDM Conference on Web Search and Data Mining, 2020, pp. 106114.
[20] S. Rendle, C. Freudenthaler, Z. Gantner, L. Thieme, “BPR: Bayesian Personalized Ranking from Implicit Feedback”, in 25th ACM UAI Conference on Uncertainty in Artificial Intelligence, 2009, pp. 452–461.
[21] E.Cho, S. Myers, J. Leskovec, “Friendship and Mobility: User Movement in LocationBased Social Networks”, in 17th ACM KDD Conference on Knowledge Discovery and Data Mining, 2011, pp. 1082–1090.
[22] P. Zhao, H. Zhu, Y. Liu, J. Xu, F. Zhuang, V. Sheng, X. Zhou, “Where to Go Next: A SpatioTemporal Gated Network for Next POI Recommendation”,. in 33th AAAI Conference on Artificial Intelligence, 2019, pp. 58775884.
[23] A M. Islam, M. M. Mohammad, S. S. Das, M. E. Ali, “A Survey on Deep Learning Based PointOfInterest (POI) Recommendations”, 2020, arXiv:2011.10187v1.
[24] C. Zheng, D. Tao, “AttentionBased Dynamic Preference Model for Next PointofInterest Recommendation”. in 15th Springer WASA Conference on Wireless Algorithms, Systems, and Applications, 2020, pp. 768–780.
[25] D. K. Bokde, Sh. Girase, D. Mukhopadhyay, “Role of Matrix Factorization Model in Collaborative Filtering Algorithm: A Survey”, International Journal of Advance Foundation and Research in Computer, vol.1, 2014, pp. 111118.
[26] M. Gan, L. Gao, “Discovering MemoryBased Preferences for POI Recommendation in LocationBased Social Networks”, International Journal of GeoInformation (IJGI), Vol.8, No.6, 2019, pp. 279294.
[27] X. Meng, J. Fang, “A Diverse and Personalized POI Recommendation Approach by Integrating GeoSocial Embedding Relations”, Journal of IEEE Access, Vol.8, 2020, pp. 226309 226323.
[28] Q. Yuan, G. Cong, Z. Ma, A. Sun, N. Thalmann, “Timeaware Pointofinterest Recommendation”, in 36th ACM SIGIR Conference on Research and development in Information Retrieval, 2013, pp. 363–372.
[29] P. Wang, H. Wang, H. Zhang, F. Lu, S. Wu, “A Hybrid Markov and LSTM Model for Indoor Location Prediction”, Journal of IEEE Access, Vol.7, 2019, pp. 185928 – 185940.
[30] J. Li, G. Liu, C. Yan, C. Jiang, “LORI: A LearningtoRankBased Integration Method of Location Recommendation”, IEEE Transactions on Computational Social Systems, Vol.6, No.3, 2019, pp. 430 – 440.
[31] L.Yao, Q. Z. Sheng, Y. Qin, X. Wang, A. Shemshadi, Q. He, “Contextaware PointofInterest Recommendation Using Tensor Factorization with Social Regularization”, in 38th.ACM SIGIR Conference on Research and Development in Information Retrieval, 2015, pp. 1007–1010.
[32] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, T. Chua, “Neural collaborative filtering”, in 26th ACM IW3C2Conference on World Wide Web Conference Committee, 2017, pp. 173182.
[33] Q. Liu, S.Wu, L. Wang, T. Tan, “Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts”, in 30th ACM AAAI Conference, 2016 , pp. 194–200.
[34] S. Kumar, M.I. Nezhurina, “An ensemble classification approach for prediction of user’s next location based on Twitter data”, Journal of Ambient Intelligence and Humanized Computing, Vol.10, No. 11, 2018, pp. 45034513.
[35] Q. Liu, S. Wu, D. Wang, Z. Li, L. Wang, “ContextAware Sequential Recommendation”, in ICDM Conference on Data Mining, IEEE, 2016, pp. 10531058.
[36] D. Bokde, S. Girase, D. Mukhopadhya, “Matrix Factorization Model in Collaborative Filtering Algorithms: A Survey”, Procedia Computer Science, Vol.49, 2015, pp. 136146.
[37] D. Lian, Y. Wu, Y. Ge, X. Xie, E. Chen, “GeographyAware Sequential Location Recommendation”, in 26th ACM ICGKDD Conference on Knowledge Discovery and Data Mining, 2020, pp. 2009–2019.
[38] K. Yang, J. Zhu, “Next POI Recommendation via Graph Embedding Representation from HDeepwalk on Hybrid Network”, Journal of IEEE Access, Vol 7, 2019, pp. 171105 – 171113.
[39] D. P. Kingma, J.L. Ba, “A Method for Stochastic Optimization”, in International Conference for Learning Representations, 2015, arXiv:1412.6980v.
* Gholamhossein Ekbatanifard
ekbatanifard@liau.ac.ir
[1] https://scikitlearn.org.
[2] In general, the Eq. (10) is used to calculate the Haversine distance between samples in X and Y (x1 and x2 are latitude and longitude of X and y1 and y2 are latitude and longitude of Y, respectively).
[3] https://snap.stanford.edu/data/locbrightkite.html
[4] https://snap.stanford.edu/data/locgowalla.html
[5] https://sites.google.com/site/yangdingqi/home/foursquaredataset


Extracting Credit Rules from Imbalanced Data: The Case of an Iranian Export Development Bank
Print Date : 20150324 

