AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition

May combine two claims.

Adding more convolutional layers tends to improve model performance, but the benefit levels off after a certain number.

“It can be found that increasing the number of convolutional layers tends to improve the performance of the model, but it approaches to be stable after a certain number.”

May combine two claims.

The best performance is achieved with sliding window widths of 20 for Heterogeneous, 15 for Skoda, and 20 for PAMAP2.

“We get the best performance when using 20, 15, and 20 width sliding window for Heterogeneous, Skoda and PAMAP2 accordingly.”

May combine two claims.

AttnSense attains the highest F1 scores among all compared algorithms and improves on state-of-the-art methods.

“the best among all algorithms, which achieves performance improvement compared to the state-of-the-art.”

• Also stated in: § 6 Conclusion · boosted

  “As demonstrated in the experiments, the proposed method outperformed the state-of-the-art methods.”

May combine two claims.

The optimal convolutional layer counts are 3 for Heterogeneous, 2 for Skoda, and 4 for PAMAP2.

“The best numbers of convolutional layers for Heterogeneous, Skoda and PAMAP2 dataset are 3, 2, and 4 accordingly.”

May combine two claims.

Existing activity recognition methods either depend on domain knowledge or do not model the spatial and temporal dependencies of sensor signals.

“The existing activity recognition approaches either rely on domain knowledge or fail to address the spatial-temporal dependencies of the sensing signals.”

• cites Bengio, 2013Not checked yet

  Not checked in the free run.

• cites Yang et al., 2015Not checked yet

  Not checked in the free run.

May combine two claims.

AttnSense combines attention with CNN and GRU to capture spatial and temporal dependencies of sensing signals, giving advantages in prioritized sensor selection and improving comprehensibility.

“AttnSense introduce the framework of combining attention mechanism with a convolutional neural network (CNN) and a Gated Recurrent Units (GRU) network to capture the dependencies of sensing signals in both spatial and temporal domains, which shows advantages in prioritized sensor selection and improves the comprehensibility.”

• AssumptionRests on an assumption: Input sequences usually contain irrelevant parts in the spatial and temporal dimensions.

Probably not a substantial claim: may not pass S3 (load-bearing).

Deep neural networks, particularly CNNs and RNNs, have been applied to HAR to extract features automatically without human domain knowledge.

“With the rapid development of deep learning [LeCun et al., 2015], more and more researchers tried to apply Deep Neural Networks (DNN), especially Convolutional Neural networks (CNN) and Recurrent Neural Networks (RNNs), to HAR systems to achieve automatic feature extraction without human domain knowledge [Wang et al., 2018].”

• cites Wang et al., 2018Not checked yet

  Not checked in the free run.

May combine two claims.

In CNN-RNN HAR, modalities originate in separate domains; combining them while ignoring their distinctions can reduce model capability and worsen activity classification.

“Firstly, different sensor modalities come from different domains, and merge them without considering their difference may limit the model's ability.”

• Also stated in: § 1 Introduction · plain

  “Treating them the same without distinction may degrade the performance of activity classification.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Visualized analysis of attention weights improves the model's comprehensibility.

“Visualized analysis of attention weights is provided to improve the model's comprehensibility.”

May combine two claims.

Shallow supervised machine learning for HAR depends heavily on handcrafted features, which are constrained by human domain knowledge and capture only shallow features.

“Such shallow supervised machine learning algorithms heavily relied on handcrafted features, which were usually limited by human domain knowledge [Bengio, 2013] and could only capture shallow features [Yang et al., 2015].”

• cites Bengio, 2013Not checked yet

  Not checked in the free run.

• cites Yang et al., 2015Not checked yet

  Not checked in the free run.

May combine two claims.

The attention mechanism in AttnSense, through attention layers integrated with CNN and GRU subnets, is able to model spatial and temporal dependencies in multimodal sensor signals and to emphasize more informative modalities and timesteps during classification.

“By integrating attention layers with both a CNN subnet and a GRU subnet to recognize multimodal sensing data, the proposed attention mechanism can capture spatial and temporal dependencies of the multimodal sensor signal, which amplifies the more important and informative modalities and timesteps during classification.”

• AssumptionRests on an assumption: Input sequences usually contain irrelevant parts in the spatial and temporal dimensions.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A CNN-RNN model has been successfully used in HAR and performed better than simple CNN and RNN models.

“Recently, a novel deep learning model CNN-RNN had been successfully employed in HAR and outperformed the simple CNN and RNN model [Yao et al., 2017] [Ordóñez andRoggen, 2016].”

• cites Yao et al., 2017Not checked yet

  Not checked in the free run.

• cites Ordóñez andRoggen, 2016Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

A hybrid CNN-RNN model has shown promising activity recognition performance.

“Moreover, a hybrid CNN-RNN model that combined CNN and RNN had shown promising results in activity recognition performance [Ordóñez andRoggen, 2016] [Yao et al., 2017].”

• cites Ordóñez andRoggen, 2016Not checked yet

  Not checked in the free run.

• cites Yao et al., 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Convolutional neural networks have demonstrated promise for detecting salient patterns in sensing signals used for human activity recognition.

“Convolutional neural networks had shown great potential in identifying the various salient patterns of sensing signals for HAR [Yang et al., 2015].”

• cites Yang et al., 2015Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

DeepSense is the leading model on the Heterogeneous dataset.

“DeepSense [Yao et al., 2017]: This model is the stateof-the-art model in Heterogeneous dataset”

• cites Yao et al., 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

CNNs applied to HAR capture salient spatial features and outperform shallow machine learning approaches.

“Deep Convolutional Neural Networks (CNN) was also applied to human activity recognition, several works took different sensor modalities as an image and fed it to a 2D CNN for feature extraction, which had been shown to capture salient features in the spatial dimension and outperforms shallow machine learning approaches [Hammerla et al., 2016][Yang et al., 2015].”

• cites Hammerla et al., 2016Not checked yet

  Not checked in the free run.

• cites Yang et al., 2015Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

RNNs with LSTM capture long-term information in time series and have been applied successfully in HAR.

“In the meantime, the RNN with long short-term memory (LSTM) was proposed and successfully applied in HAR [Hammerla et al., 2016][Guan andPlötz, 2017], which can capture the long-term information in time series.”

• cites Hammerla et al., 2016Not checked yet

  Not checked in the free run.

• cites Guan andPlötz, 2017Not checked yet

  Not checked in the free run.

May combine two claims.

Gated recurrent units can capture long-range dependencies in sequences and overcome the vanishing gradient issue that affects standard recurrent neural networks.

“Similar to LSTM [Greff et al., 2015], GRU can model long-term dependencies in a sequence and solves the vanishing gradient problem of conventional RNN.”

• cites chu, 2014Not checked yet

  Not checked in the free run.

• cites Greff et al., 2015Not checked yet

  Not checked in the free run.

May combine two claims.

A standard GRU produces a hidden state at every timestep but relies only on the final timestep's hidden state as the single representation of the entire input sequence.

“Standard GRU generates hidden state at each timestep [chu, 2014], but only use hidden state at the last timestep as a single representation for the whole input sequence”

• cites chu, 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Deep learning approaches provide promising results in HAR.

“Deep learning approaches applied the deep neural network (DNN) framework to perform automatic feature extraction and classification, which provided promising results in HAR domain [Yang et al., 2015].”

• cites Yang et al., 2015Not checked yet

  Not checked in the free run.

May combine two claims.

AttnSense can capture the important part of the sequence to improve performance and comprehensibility.

“our proposed model can capture the important part of the sequence to increase performance and comprehensibility.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study used four-fold cross-validation on Skoda and leave-one-subject-out validation on Heterogeneous and PAMAP2 to select the best model configuration.

“Moreover, we perform 4-fold cross-validation for Skoda dataset and leave-one-subject-out validation for Heterogeneous and PAMAP2 dataset to obtain the best model configuration.”

Probably not a substantial claim: may not pass S4 (based).

The data preprocessing technique helps AttnSense capture noise patterns and frequency features in multidimensional sensor signals.

“The data preprocessing technique also help our model to capture the noise pattern and frequency features in the multidimensional sensing signals.”

Classified as probably not a substantial claim. May combine two claims. The quote only partly states this claim.

The experiments were repeated twenty times, and the averaged weighted F1 score was used as the final measure of model performance.

“Furthermore, we ran 20 repetitions of the experiments and report averaged F w as the final measure of a model's performance.”

May combine two claims.

The Heterogeneous dataset's use of twelve different smartphones and smartwatches makes the activity recognition task more complex and allows the model's robustness to be tested.

“An important fact of the dataset is that users perform these activities with 12 different smartphones and smartwatches, which increases the complexity of the task and can test the model's robustness.”