A Comparative Analysis of a Multi-Headed Self-Attention Mechanism-Based Transformer Model for Bus Travel Time Prediction Using Heterogeneous Datasets Across Multiple Bus Routes

May combine two claims.

The Transformer model showed fewer high-error regions compared to GRU, LSTM, and Historical Average models, which exhibited more widespread red areas.

“The Transformer model showed fewer high-error regions compared to GRU, LSTM, and the Historical Average model, all of which exhibit more widespread red areas.”

May combine two claims.

Using five hours of historical data to predict the next hour, the proposed transformer achieved the lowest minimum and mean MAPE values (4.32% and 8.29%).

“Using five hours of historical data to predict the next hour, the proposed transformer achieved the lowest minimum and mean MAPE values (4.32% and 8.29%) and performed consistently well during both peak and off-peak periods.”

May combine two claims.

The transformer remained competitive in computation time (7.42 seconds) while offering higher accuracy than XGBoost.

“While XGBoost delivered the fastest computation time (6.28 seconds), the transformer remained competitive (7.42 seconds) while offering higher accuracy.”

May combine two claims.

The LSTM model's final test loss stabilized at 0.0016, and its MAPE values ranged between 17% and 18%.

“The final test loss stabilized at 0.0016, while MAPE values ranged between 17-18%.”

The quote only partly states this claim.

The Univariate Transformer Model had the highest prediction accuracy across all six routes at the ninth hour.

“The outcomes showed that the Univariate Transformer Model had the highest prediction accuracy across all six of the routes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Early stopping terminated Transformer training after 13 epochs once improvements in validation loss fell below the minimum threshold (min_delta = 0.0005) for 10 consecutive epochs.

“Although the model was initially set for 100 epochs, early stopping terminated training after 13 epochs once improvements in validation loss fell below the minimum threshold (min_delta = 0.0005) for 10 consecutive epochs.”

May combine two claims.

The GRU model's training and validation loss curves demonstrated smooth convergence, with the final test loss stabilizing at 0.0098.

“Figure 9 showed the GRU network structure and the training and validation loss curves, which demonstrated smooth convergence with the final test loss stabilizing at 0.0098.”

May combine two claims.

The Transformer achieved the lowest minimum, maximum, and median MAPE values among the models.

“In contrast to the other models, the Transformer achieved the lowest minimum, maximum, and median MAPE values, as illustrated in the error bar plots in Figure 14.”

May combine two claims.

The MAPE values for the three models were approximately 7.15% for the Transformer, 10.61% for the GRU, and 10.99% for the LSTM, indicating that the Transformer consistently produced the lowest prediction errors.

“The MAPE values for the three models were approximately 7.15% for the Transformer, 10.61% for the GRU, and 10.99% for the LSTM, indicating that the Transformer consistently produced the lowest prediction errors.”

May combine two claims.

The Transformer model achieved the lowest minimum MAPE of 4.32% and the best mean MAPE of 8.29%, substantially better than the GRU, LSTM, and Historical Average models, and only slightly higher than the XGBoost model.

“The Transformer model achieved the lowest minimum MAPE of 4.32% and the best mean MAPE of 8.29%, which was substantially better than the GRU, LSTM, and Historical Average models, and only slightly higher than the XGBoost model.”

May combine two claims.

The standard deviation of the Transformer (2.19%) was larger than that of XGBoost (1.82%) but lower than those of the GRU, LSTM, and Historical Average models.

“While the standard deviation of the Transformer (2.19%) was larger than that of XGBoost (1.82%), it was still lower than those of the GRU, LSTM, and Historical Average models, confirming that the Transformer balanced accuracy with stability.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In most cases, mean bus travel times ranged from 20 to 75 seconds, although higher values were occasionally observed.

“In most cases, mean travel times ranged from 20 to 75 seconds, although higher values were occasionally observed.”

May combine two claims.

Although the Transformer remained the top-performing model overall, XGBoost displayed competitive performance in some cases, occasionally approaching the Transformer's accuracy.

“Although the Transformer remained the top-performing model overall, XGBoost displayed competitive performance in some cases, occasionally approaching the Transformer's accuracy.”

May combine two claims.

XGBoost demonstrated stronger performance than GRU, LSTM, and Historical Average but still lagged behind the Transformer in several locations.

“XGBoost demonstrated stronger performance than GRU, LSTM, and Historical Average but still lagged behind the Transformer in several locations.”

May combine two claims.

The Univariate Transformer achieved a minimum MAPE of 4.32% and a mean MAPE of 8.29% for one-hour-ahead bus travel time prediction.

“Results demonstrated that the Univariate Transformer achieved superior predictive accuracy, with minimum and mean MAPE values of 4.32% and 8.29%, respectively, outperforming all other models.”

May combine two claims. The quote only partly states this claim.

The Transformer's computation time (7.42 s) was slightly longer than GRU and Historical Average, higher than XGBoost (6.28 s), but faster than LSTM (8.67 s).

“The Transformer model achieved a computation time of 7.42 s, which was only slightly longer than GRU and Historical Average, confirming that despite its higher architectural complexity, the Transformer remains computationally efficient and practical for real-time applications.”

• Also stated in: § Figure 17: Model Prediction Computation Times · plain

  “The model achieved an average prediction time of 7.42 seconds, slightly higher than GRU (6.99 seconds) and XGBoost (6.28 seconds), but faster than LSTM (8.67 seconds).”

May combine two claims.

The proposed framework demonstrates scalability, efficiency, and potential for real-time deployment in complex urban transit networks.

“demonstrating its scalability, efficiency, and potential for real-time deployment in complex urban transit networks.”

• AssumptionRests on an assumption: The one-week data from St. Louis, Missouri, is representative of large, complex urban transit networks.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

With increasing computational power and the availability of large-scale Intelligent Transportation System data, deep learning has emerged as a powerful alternative to traditional machine learning.

“With increasing computational power and the availability of large-scale Intelligent Transportation System (ITS) data, deep learning has emerged as a powerful alternative to traditional machine learning.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The attention mechanism has become a powerful component in forecasting models and has recently been integrated into deep learning architectures such as LSTM and GRU for bus travel time prediction.

“The attention mechanism has become a powerful component in forecasting models and has recently been integrated into deep learning architectures such as LSTM and GRU for bus travel time prediction.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Unlike RNNs, which process inputs sequentially, the Transformer simultaneously models relationships across all elements of the input sequence, allowing it to capture long-range dependencies more effectively and at scale.

“Unlike Recurrent Neural Network (RNNs), which process inputs sequentially, the Transformer simultaneously models relationships across all elements of the input sequence, allowing it to capture long-range dependencies more effectively and at scale.”

Classified as probably not a substantial claim.

A key advantage of incorporating an attention layer is its ability to leverage historical traffic data along with contextual factors, such as weather, time of day, and dwell time that influence bus travel times.

“A key advantage of incorporating an attention layer is its ability to leverage historical traffic data along with contextual factors, such as weather, time of day, and dwell time that influence bus travel times.”

May combine two claims.

Most prior studies focus on small networks or single routes, restricting generalizability to large, multi-route, multi-station systems where long-range dependencies are critical.

“Most prior studies focus on small networks or single routes, restricting generalizability to large, multi-route, multi-station systems where long-range dependencies are critical.”

May combine two claims.

The results demonstrate the transformer's suitability and scalability for real-time bus travel time prediction in large, complex urban transit networks.

“These results demonstrated the transformer's suitability and scalability for real-time bus travel time prediction in large, complex urban transit networks.”

• AssumptionRests on an assumption: The one-week data from St. Louis, Missouri, is representative of large, complex urban transit networks.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Existing bus travel time prediction models are often constrained to small networks, struggle to capture long-range dependencies, and rarely integrate real-time GTFS, static transit, and vehicle probe data while accounting for spatiotemporal and external factors.

“Existing bus travel time prediction models are often constrained to small networks, struggle to capture long-range dependencies, and rarely integrate real-time GTFS, static transit, and vehicle probe data while accounting for spatiotemporal and external factors.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Conventional modeling techniques are unable to capture these interactions, and more sophisticated machine learning and deep learning algorithms that incorporate real-time data and can capture intricate non-linear correlations between these factors are needed to provide accurate predictions.

“Conventional modeling techniques are unable to capture these interactions, and more sophisticated machine learning and deep learning algorithms that incorporate real-time data and can capture intricate non-linear correlations between these factors are needed to provide accurate predictions.”

Probably not a substantial claim: may not pass S4 (based).

Predicting bus travel times accurately is challenging because of the complex interactions between various nonlinear factors such as traffic conditions, incidents, weather conditions, dwell times, passenger load, passenger boarding/alighting time, and number of signalized intersections.

“However, predicting bus travel times accurately is challenging because of the complex interactions between various nonlinear factors such as traffic conditions, incidents, weather conditions, dwell times, passenger load, passenger boarding/alighting time, number of signalized intersections, etc.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

To the best of the authors' knowledge, this is the first application of a Transformer model for bus travel time prediction in St. Louis.

“To the best of our knowledge, this is the first application of a Transformer model for bus travel time prediction in St. Louis, demonstrating its scalability, efficiency, and potential for real-time deployment in complex urban transit networks.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The chosen Transformer hyperparameters allowed the model to capture temporal dependencies effectively while maintaining robustness and generalization.

“These hyperparameters allowed the model to capture temporal dependencies effectively while maintaining robustness and generalization.”

The quote only partly states this claim.

The LSTM's final test loss and MAPE values confirmed its stability as a comparative deep learning baseline.

“These results confirmed the stability of the LSTM as a comparative deep learning baseline.”

May combine two claims.

Although the Transformer was not the fastest model, its computation time remained practical for real-time applications and was justified by substantial gains in predictive reliability.

“This highlighted an important trade-off: although the Transformer was not the fastest model, its computation time remained practical for real-time applications and was justified by the substantial gains in predictive reliability.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The slightly higher variability in the Transformer predictions compared to XGBoost can be attributed to the large number of trainable parameters and the sensitivity of deep learning models to hyperparameters.

“The slightly higher variability in the Transformer predictions compared to XGBoost can be attributed to the large number of trainable parameters and the sensitivity of deep learning models to hyperparameter”

Classified as probably not a substantial claim. May combine two claims.

Unlike conventional recurrent neural networks, which process input sequentially and often struggle with vanishing gradients, the Transformer leverages a self-attention mechanism that allows the model to focus on relevant portions of the input sequence simultaneously.

“Unlike conventional recurrent neural networks, which process input sequentially and often struggle with vanishing gradients, the Transformer leverages a self-attention mechanism that allows the model to focus on relevant portions of the input sequence simultaneously.”

• cites Vaswani et al. 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The same input parameters were used for training XGBoost as for the GRU and LSTM models, ensuring comparability in predicting average bus travel times across different hours.

“The same input parameters were used for training XGBoost as those applied to the GRU and LSTM models, ensuring comparability in predicting average bus travel times across different hours.”

Classified as probably not a substantial claim.

The Transformer model is a neural network framework designed to capture long-range dependencies in sequential data.

“The Transformer model, first introduced by Vaswani et al., is a neural network framework designed to capture longrange dependencies in sequential data (Vaswani et al. 2017).”

• cites Vaswani et al. 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The quote only partly states this claim.

Early stopping ensured that the best Transformer model was preserved without unnecessary overfitting.

“This ensured that the best model was preserved without unnecessary overfitting.”

May combine two claims. The quote only partly states this claim.

The Transformer's superior capture of sharp peaks and complex temporal variations highlighted its ability to handle complex patterns without reliance on additional external features.

“This highlighted the model's ability to effectively handle complex patterns more without reliance on additional external features.”

May combine two claims.

These outcomes reinforced the Transformer's capability to capture complex temporal dependencies even under varying traffic dynamics, while also highlighting the complementary strengths of XGBoost as a lightweight alternative.

“These outcomes reinforced the Transformer's capability to capture complex temporal dependencies even under varying traffic dynamics, while also highlighting the complementary strengths of XGBoost as a lightweight alternative.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The single-step multi-station forecasting approach ensured that the Transformer could capture temporal variations throughout the day and provide accurate, station-specific bus travel time forecasts.

“This approach ensured that the Transformer could capture temporal variations throughout the day and provide accurate, station-specific bus travel time forecasts.”

May combine two claims.

Despite its higher architectural complexity, the Transformer remains computationally efficient and practical for real-time applications.

“The Transformer model achieved a computation time of 7.42 s, which was only slightly longer than GRU and Historical Average, confirming that despite its higher architectural complexity, the Transformer remains computationally efficient and practical for real-time applications.”

• Also stated in: § Figure 17: Model Prediction Computation Times · plain

  “although the Transformer was not the fastest model, its computation time remained practical for real-time applications and was justified by the substantial gains in predictive reliability.”

May combine two claims.

The Transformer's architectural strength enabled it to deliver robust predictions across diverse conditions, while its runtime requirements remained manageable.

“Its architectural strength enabled it to deliver robust predictions across diverse conditions, while its runtime requirements remained manageable.”

May combine two claims.

The Transformer offered clear advantages in accuracy, scalability, and generalization.

“Although XGBoost produced the fastest computation time (6.28 seconds), the Transformer remained competitive (7.42 seconds) and offered clear advantages in accuracy, scalability, and generalization.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The exceptions where XGBoost outperformed the Transformer on Routes 18128 and 18145 may be attributed to mild overfitting of the Transformer on specific data segments or to more optimal XGBoost hyperparameters for those conditions.

“These exceptions may be attributed to mild overfitting of the Transformer on specific data segments or to the possibility that the chosen hyperparameters for XGBoost were more optimal for those particular conditions.”

The quote only partly states this claim.

The smooth convergence and final test loss of 0.0098 confirmed the robustness of the GRU model.

“This confirmed the robustness of the GRU model.”

May combine two claims. The quote only partly states this claim.

The Transformer's lowest minimum, maximum, and median MAPE values highlighted the stability of its predictions and its ability to capture sharp variations in travel times more effectively than the other deep learning and baseline approaches.

“This visualization highlighted the stability of the Transformer predictions and its ability to capture sharp variations in travel times more effectively than the other deep learning and baseline approaches.”