Search Results (1,658)

While deep neural networks (DNNs) have been widely and successfully used for time series classification (TSC) over the past decade, their vulnerability to adversarial attacks has received little attention. Most existing attack methods focus on white-box setups, which are unrealistic as attackers typically only have access to the model’s probability outputs. Defensive methods also have limitations, relying primarily on adversarial retraining which degrades classification accuracy and requires excessive training time. On top of that, we propose two new approaches in this paper: (1) A simulated annealing-based random search attack that finds adversarial examples without gradient estimation, searching only on the $l_{\infty }$-norm hypersphere of allowable perturbations. (2) A post-processing defense technique that periodically reverses the trend of corresponding loss values while maintaining the overall trend, using only the classifier’s confidence scores as input. Experiments applying these methods to InceptionNet models trained on the UCR dataset benchmarks demonstrate the effectiveness of the attack, achieving up to 100% success rates. The defense method provided protection against up to 91.24% of attacks while preserving prediction quality. Overall, this work addresses important gaps in adversarial TSC by introducing novel black-box attack and lightweight defense techniques. Full article

Geostationary (GEO) satellites are employed in optical frequencies for a variety of satellite services providing wide coverage and connectivity. Multi-beam GEO high-throughput satellites offer Gbps broadband rates and, jointly with low-Earth-orbit mega-constellations, are anticipated to enable a large-scale free-space optical (FSO) network. In this paper, a power allocation methodology based on deep reinforcement learning (DRL) is proposed for optical satellite systems disregarding any channel statistics knowledge requirements. An all-FSO, multi-aperture GEO-to-ground system is considered and an ergodic capacity optimization problem for the downlink is formulated with transmitted power constraints. A power allocation algorithm was developed, aided by a deep neural network (DNN) which is fed channel state information (CSI) observations and trained in a parameterized on-policy manner through a stochastic policy gradient approach. The proposed method does not require the channels’ transition models or fading distributions. To validate and test the proposed allocation scheme, experimental measurements from the European Space Agency’s ARTEMIS optical satellite campaign were utilized. It is demonstrated that the predicted average capacity greatly exceeds other baseline heuristic algorithms while strongly converging to the supervised, unparameterized approach. The predicted average channel powers differ only by 0.1 W from the reference ones, while the baselines differ significantly more, about 0.1–0.5 W. Full article

Holographic memory offers high-capacity optical storage with rapid data readout and long-term durability. Recently, read data pages have been classified using digital deep neural networks (DNNs). This approach is highly accurate, but the prediction time hinders the data readout throughput. This study presents a diffractive DNN (D2NN)-based classifier for holographic memory. D2NNs have so far attracted a great deal of attention for object identification and image transformation at the speed of light. A D2NN, consisting of trainable diffractive layers and devoid of electronic devices, facilitates high-speed data readout. Furthermore, we numerically investigated the classification performance of a D2NN-based classifier. The classification accuracy of the D2NN was 99.7% on 4-bit symbols, exceeding that of the hard decision method. Full article

Inspired by the biological nervous system, deep neural networks (DNNs) are able to achieve remarkable performance in various tasks. However, they struggle to handle label noise, which can poison the memorization effects of DNNs. Co-teaching-based methods are popular in learning with noisy labels. These methods cross-train two DNNs based on the small-loss criterion and employ a strategy using either “disagreement” or “consistency” to obtain the divergence of the two networks. However, these methods are sample-inefficient for generalization in noisy scenarios. In this paper, we propose CoDC, a novel Co-teaching-basedmethod for accurate learning with label noise via both Disagreement and Consistency strategies. Specifically, CoDC maintains disagreement at the feature level and consistency at the prediction level using a balanced loss function. Additionally, a weighted cross-entropy loss is proposed based on information derived from the historical training process. Moreover, the valuable knowledge involved in “large-loss” samples is further developed and utilized by assigning pseudo-labels. Comprehensive experiments were conducted on both synthetic and real-world noise and under various noise types. CoDC achieved 72.81% accuracy on the Clothing1M dataset and 76.96% (Top1) accuracy on the WebVision1.0 dataset. These superior results demonstrate the effectiveness and robustness of learning with noisy labels. Full article

Commercial buildings in hot and humid tropical climates rely significantly on cooling systems to maintain optimal occupant comfort. A well-accurate day-ahead load profile prediction plays a pivotal role in planning the energy requirements of cooling systems. Despite the pressing need for effective day-ahead cooling load predictions, current methodologies have not fully harnessed the potential of advanced deep-learning techniques. This paper aims to address this gap by investigating the application of innovative deep-learning models in day-ahead hourly cooling load prediction for commercial buildings in tropical climates. A range of multi-output deep learning techniques, including Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory networks (LSTMs), are employed to enhance prediction accuracy. Furthermore, these individual deep learning techniques are synergistically integrated to create hybrid models, such as CNN-LSTM and Sequence-to-Sequence models. Experiments are conducted to choose the time horizons from the past that can serve as input to the models. In addition, the influence of various categories of input parameters on prediction performance has been assessed. Historical cooling load, calendar features, and outdoor weather parameters are found in decreasing order of influence on prediction accuracy. This research focuses on buildings located in Singapore and presents a comprehensive case study to validate the proposed models and methodologies. The sequence-to-sequence model provided better performance than all the other models. It offered a CV-RMSE of 7.4%, 10%, and 6% for SIT@Dover, SIT@NYP, and the simulated datasets, which were 2.3%, 3%, and 1% less, respectively, than the base Deep Neural Network model. Full article

In this paper, we explore the problem of direction-of-arrival (DOA) estimation for a non-uniform linear array (NULA) under strong noise. The compressed sensing (CS)-based methods are widely used in NULA DOA estimations. However, these methods commonly rely on the tuning of parameters, which are hard to fine-tune. Additionally, these methods lack robustness under strong noise. To address these issues, this paper proposes a novel DOA estimation approach using a deep neural network (DNN) for a NULA in a low SNR. The proposed network is designed based on the denoising convolutional neural network (DnCNN) and the alternating direction method of multipliers (ADMM), which is dubbed as LDnADMM-Net. First, we construct an unfolded DNN architecture that mimics the behavior of the iterative processing of an ADMM. In this way, the parameters of an ADMM can be transformed into the network weights, and thus we can adaptively optimize these parameters through network training. Then, we employ the DnCNN to develop a denoising module (DnM) and integrate it into the unfolded DNN. Using this DnM, we can enhance the anti-noise ability of the proposed network and obtain a robust DOA estimation in a low SNR. The simulation and experimental results show that the proposed LDnADMM-Net can obtain high-accuracy and super-resolution DOA estimations for a NULA with strong robustness in a low signal-to-noise ratio (SNR). Full article

Remote sensing-based models usually have difficulty in generating spatio-temporally continuous terrestrial evapotranspiration (ET) due to cloud cover and model failures. To overcome this problem, machine learning methods have been widely used to reconstruct ET. Therefore, studies comparing and evaluating the accuracy and effectiveness of reconstruction among different machine learning methods at the basin scale are necessary. In this study, four popular machine learning methods, including deep forest (DF), deep neural network (DNN), random forest (RF) and extreme gradient boosting (XGB), were used to reconstruct the ET product, addressing gaps resulting from cloud cover and model failure. The ET reconstructed by the four methods was evaluated and compared for Heihe River Basin. The results showed that the four methods performed well for Heihe River Basin, but the RF method was particularly robust. It not only performed well compared with ground measurements (R = 0.73) but also demonstrated the ability to fully reconstruct gaps generated by the TSEB model across the entire basin. Validation based on ground measurements showed that the DNN and XGB models performed well (R > 0.70). However, some gaps still existed in the desert after reconstruction using the DNN and XGB models, especially for the XGB model. The DF model filled these gaps throughout the basin, but this model had lower consistency compared with ground measurements (R = 0.66) and yielded many low values. The results of this study suggest that machine learning methods have considerable potential in the reconstruction of ET at the basin scale. Full article

In the quest for sustainable energy solutions, predicting electricity prices for renewable energy sources plays a pivotal role in efficient resource allocation and decision making. This article presents a novel approach to forecasting electricity prices for renewable energy sources using deep learning models, leveraging historical data from the power system operator (PSE). The proposed methodology encompasses data collection, preprocessing, feature engineering, model selection, training, and evaluation. By harnessing the power of recurrent neural networks (RNNs) and other advanced deep learning architectures, the model captures intricate temporal relationships, weather patterns, and demand fluctuations that impact renewable energy prices. The study demonstrates the applicability of this approach through empirical analysis, showcasing its potential to enhance energy market predictions and aid in the transition to more sustainable energy systems. The outcomes underscore the importance of accurate renewable energy price predictions in fostering informed decision making and facilitating the integration of renewable sources into the energy landscape. As governments worldwide prioritize renewable energy adoption, this research contributes to the arsenal of tools driving the evolution towards a cleaner and more resilient energy future. Full article

Deep Neural Network (DNN)-based point cloud semantic segmentation has presented significant breakthrough using large-scale labeled aerial laser point cloud datasets. However, annotating such large-scaled point clouds is time-consuming. Self-Supervised Learning (SSL) is a promising approach to this problem by pre-training a DNN model utilizing unlabeled samples followed by a fine-tuned downstream task involving very limited labels. The traditional contrastive learning for point clouds selects the hardest negative samples by solely relying on the distance between the embedded features derived from the learning process, potentially evolving some negative samples from the same classes to reduce the contrastive learning effectiveness. This work proposes a hard-negative sample-aware self-supervised contrastive learning algorithm to pre-train the model for semantic segmentation. We designed a k-means clustering-based Absolute Positive And Negative samples (AbsPAN) strategy to filter the possible false-negative samples. Experiments on two typical ALS benchmark datasets demonstrate that the proposed method is more appealing than supervised training schemes without pre-training. Especially when the labels are severely inadequate (10% of the ISPRS training set), the results obtained by the proposed HAVANA method still exceed 94% of the supervised paradigm performance with full training set. Full article

Overconfidence in deep neural networks (DNN) reduces the model’s generalization performance and increases its risk. The deep ensemble method improves model robustness and generalization of the model by combining prediction results from multiple DNNs. However, training multiple DNNs for model averaging is a time-consuming and resource-intensive process. Moreover, combining multiple base learners (also called inducers) is hard to master, and any wrong choice may result in lower prediction accuracy than from a single inducer. We propose an approximation method for deep ensembles that can obtain ensembles of multiple DNNs without any additional costs. Specifically, multiple local optimal parameters generated during the training phase are sampled and saved by using an intelligent strategy. We use cycle learning rates starting at 75% of the training process and save the weights associated with the minimum learning rate in every iteration. Saved sets of the multiple model parameters are used as weights for a new model to perform forward propagation during the testing phase. Experiments on benchmarks of two different modalities, static images and dynamic videos, show that our method not only reduces the calibration error of the model but also improves the accuracy of the model. Full article

The proliferation of uncategorized information on the Internet has intensified the need for effective recommender systems. Recommender systems have evolved from content-based filtering to collaborative filtering and, most recently, to deep learning-based and hybrid models. However, they often face challenges such as high computational costs, reduced reliability, and the Cold Start problem. We introduce a persona-based user modeling approach for real-time movie recommendations. Our system employs Non-negative Matrix Factorization (NMF) and Deep Learning algorithms to manage complex and sparse data types and to mitigate the Cold Start issue. Experimental results, based on criteria involving 50 topics and 35 personas, indicate a significant performance gain. Specifically, with 500 users, the precision@K for NMF was 86.01%, and for the Deep Neural Network (DNN), it was 92.67%. Tested with 900 users, the precision@K for NMF increased to 97.04%, and for DNN, it was 95.55%. These results represent an approximate 10% and 5% improvement in performance, respectively. The system not only delivers fast and accurate recommendations but also reduces computational overhead by updating the model only when user personas change. The generated user personas can be adapted for other recommendation services or large-scale data mining. Full article

This study presents a model-based deep reinforcement learning (MB-DRL) controller for the fluidized bed biomass gasification (FBG) process. The MB-DRL controller integrates a deep neural network (DNN) model and a reinforcement learning-based optimizer. The DNN model is trained with operational data from a pilot-scale FBG plant to approximate FBG process dynamics. The reinforcement learning-based optimizer employs a specially designed reward function, determining optimal control policies for FBG. Moreover, the controller includes an online learning component, ensuring periodic updates to the DNN model training. The performance of the controller is evaluated by testing its control accuracy for regulating synthetic gas composition, flow rate, and CO concentration in the FBG. The evaluation also includes a comparison with a model predictive controller. The results demonstrate the superior control performance of MB-DRL, surpassing MPC by over 15% in regulating synthetic gas composition and flow rate, with similar effectiveness observed in synthetic gas temperature control. Additionally, this study also includes systematic investigations into factors like DNN layer count and learning update intervals to provide insights for the practical implementation of the controller. The results, presenting a 50% reduction in control error with the addition of a single layer to the DNN model, highlight the significance of optimizing MB-DRL for effective implementation. Full article

Autism spectrum disorder (ASD) is a global concern, with a prevalence rate of approximately 1 in 36 children according to estimates from the Centers for Disease Control and Prevention (CDC). Diagnosing ASD poses challenges due to the absence of a definitive medical test. Instead, doctors rely on a comprehensive evaluation of a child’s developmental background and behavior to reach a diagnosis. Although ASD can occasionally be identified in children aged 18 months or younger, a reliable diagnosis by an experienced professional is typically made by the age of two. Early detection of ASD is crucial for timely interventions and improved outcomes. In recent years, the field of early diagnosis of ASD has been greatly impacted by the emergence of deep learning models, which have brought about a revolution by greatly improving the accuracy and efficiency of ASD detection. The objective of this review paper is to examine the recent progress in early ASD detection through the utilization of multimodal deep learning techniques. The analysis revealed that integrating multiple modalities, including neuroimaging, genetics, and behavioral data, is key to achieving higher accuracy in early ASD detection. It is also evident that, while neuroimaging data holds promise and has the potential to contribute to higher accuracy in ASD detection, it is most effective when combined with other modalities. Deep learning models, with their ability to analyze complex patterns and extract meaningful features from large datasets, offer great promise in addressing the challenge of early ASD detection. Among various models used, CNN, DNN, GCN, and hybrid models have exhibited encouraging outcomes in the early detection of ASD. The review highlights the significance of developing accurate and easily accessible tools that utilize artificial intelligence (AI) to aid healthcare professionals, parents, and caregivers in early ASD symptom recognition. These tools would enable timely interventions, ensuring that necessary actions are taken during the initial stages. Full article

The deformation of tunnel support structures during tunnel construction is influenced by geological factors, geometrical factors, support factors, and construction factors. Accurate prediction of tunnel support structure deformation is crucial for engineering safety and optimizing support parameters. Traditional methods for tunnel deformation prediction have often relied on numerical simulations and model experiments, which may not always meet the time-sensitive requirements. In this study, we propose a fusion deep neural network (FDNN) model that combines multiple algorithms with a complementary tunnel information encoding method. The FDNN model utilizes Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks to extract features related to tunnel structural deformation. FDNN model is used to predict deformations in the Capital Ring Expressway, and the predictions align well with monitoring results. To demonstrate the superiority of the proposed model, we use four different performance evaluation metrics to analyze the predictive performance of FDNN, DNN, XGBoost, Decision Tree Regression (DTR), and Random Forest Regression (RFR) methods. The results indicate that FDNN exhibits high precision and robustness. To assess the impact of different data types on the predictive results, we use tunnel geometry data as the base and combine geological, support, and construction data. The analysis reveals that models trained on datasets comprising all four data types perform the best. Geological parameters have the most significant impact on the predictive performance of all models. The findings of this research guide predicting tunnel construction parameters, particularly in the dynamic design of support parameters. Full article

With the increasing availability of computational power, contemporary machine learning has undergone a paradigm shift, placing a heightened emphasis on deep learning methodologies. The pervasive automation of various processes necessitates a critical re-evaluation of contemporary network implementations, specifically concerning security protocols and the imperative need for swift, precise responses to system failures. This article introduces a meticulously crafted solution designed explicitly for 6G software-defined networks (SDNs). The approach employs deep neural networks for anomaly detection within network traffic, contributing to a more robust security framework. Furthermore, the paper delves into the realm of network monitoring automation by harnessing dynamic telemetry, providing a specialized and forward-looking strategy to tackle the distinctive challenges inherent in SDN environments. In essence, our proposed solution aims to elevate the security and responsiveness of 6G mobile networks. By addressing the intricate challenges posed by next-generation network architectures, it seeks to fortify these networks against emerging threats and dynamically adapt to the evolving landscape of next-generation technology. Full article