The main objective of this work is to model and exploit available contextual information to provide a hypothesis on suspicious vehicle maneuvers. This paper presents an innovative anomaly detection scheme, which utilizes L1 tracking to perform L2/L3 data fusion, i.e., situation/threat refinement and assessment. The proposed concept involves a context-aided tracker called the Con-Tracker, a multiple-model adaptive estimator, and an L2/L3 hypothesis generator. The purpose of the Con-Tracker is to incorporate the contextual information into a traditional Kalman filter-based tracker in such a way that it provides a repeller or attractor characteristic to a specific region of interest. Any behavior of the vehicle that is inconsistent with the repeller or attractor characteristic of the current vehicle location would be classified as suspicious. Such inconsistent vehicle behavior would be directly indicated by a high measurement residual, which then may be used to estimate the process noise covariance associated with the context-aware model using a multiple-model adaptive estimator. Based on the rate of change of the estimated process noise covariance values, an L2/L3 hypothesis generator red-flags the target vehicle. Simulation results indicate that the proposed concept involving context-aided tracking enhances the reliability of anomaly detection.