IEEE SMARTCOMP 2023

Quantum Key Distribution (QKD) enables secure communications via the exchange of cryptographic keys exploiting the properties of quantum mechanics. Nowadays the related technology is mature enough for production systems, thus field deployments of QKD networks are expected to appear in the near future, starting from local/metropolitan settings, where edge computing is already a thriving reality. In this paper, we investigate the interplay between the resource allocation of resources in the QKD network and edge nodes, which creates unique research challenges. After modeling mathematically the problem, we propose practical online policies for admitting edge application requests, which also select the edge node for processing and the path in the QKD network. Our simulation results provide initial insights into this emerging topic and lead the way to upcoming studies on the subject.

With the growing consumption of online visual contents, there is an urgent need for video translation in order to reach a wider audience from around the world. However, the materials after direct translation and dubbing are unable to create a natural audio-visual experience since the translated speech and lip movement are often out of sync. To improve viewing experience, an accurate automatic lip-movement synchronization generation system is necessary. To improve the accuracy and visual quality of speech to lip generation, this research proposes two techniques: Embedding Attention Mechanisms in Convolution Layers and Deploying SSIM as Loss Function in Visual Quality Discriminator. The proposed system as well as several other ones are tested on three audio-visual datasets. The results show that our proposed methods achieve superior performance over the state-of-the-art speech to lip synthesis on not only the accuracy but also the visual quality of audio-lip synchronization generation.

With the increased popularity of Mobile Crowd Sensing (MCS), large volumes of sensing data can be collected by leveraging the sensing capabilities of the mobile devices carried by crowds already roaming in a mobility field. In participatory MCS, tasks are allocated to participants via some allocation mechanism, which are challenging in terms of their evaluation due to the lack of general-purpose, modular, and extendable simulators. Thus, forcing researchers to either launch their own testbeds or develop single-purpose simulators. In this paper, we present our design and implementation of an extendable simulator, namely TACSim, for the evaluation of task allocation mechanisms in a participatory MCS setting over realistic urban environments. TACSim is designed to accommodate realistic urban road networks, as well as spatio-temporal traces of sensing tasks and participant mobility. Moreover, it includes various built-in autonomous task allocation mechanisms, which can be extended by researchers to accommodate their own algorithms with minimal effort. We discuss the components and architecture of the simulator, and present a use-case of integrating existing autonomous task allocation mechanisms that further exemplifies the usability and extendability of the simulator.

Accurate estimation of the battery state of charge (SOC) of unmanned aerial vehicles (UAV) along a mission is an essential in-flight monitoring task to guarantee the survivability of the system. Physics-based models of the battery have been developed in the past with successful applications. However, in general, these models do not account for the effect of the mission profile and environmental conditions on power consumption. Recently, data-driven methods have been leveraged given their ease-of-use and scalability. Yet, most benchmarking experiments have been conducted on simulated battery datasets. In this work, we compare different data-driven models for battery SOC estimation of a hexacopter UAVs by using real flight data. We analyze the importance of numerous flight variables under different environmental conditions to determine which factors impact battery consumption over the course of the flight. We demonstrate that additional flight variables are necessary to create an accurate SOC estimation model through data-driven methods.

Although recent performance improvements to Secure Multiparty Computation (SMPC) made it a practical solution for complex applications such as privacy-preserving machine learning (ML), other characteristics such as robustness are also critical for its practical viability. For instance, since ML training under SMPC may take longer times (e.g., hours or days in many cases), any interruption of the computation will require restarting the process, which results in more delays and waste of computing resources. While one can maintain exchanged SMPC messages in a separate database, their integrity and authenticity should be guaranteed to be able to re-use them later. Therefore, in this paper, we propose ReplayMPC, an efficient failure recovery mechanism for SMPC based on blockchain technology that enables resuming and re-synchronizing SMPC parties after any type of communication or system failures. Our approach allows SMPC parties to save computation state snapshots they use as restoration points during the recovery and then reproduce the last computation rounds by retrieving information from immutable messages stored on a blockchain. Our experiment results on Algorand blockchain show that recovery is much faster than starting the whole process from scratch, saving time, computation, and networking resources.

IoT sensors, especially video cameras, are ubiquitously deployed around the world to perform a variety of computer vision tasks in several verticals including retail, health- care, safety and security, transportation, manufacturing, etc. To amortize their high deployment effort and cost, it is desirable to perform multiple video analytics tasks, which we refer to as Analytical Units (AUs), off the video feed coming out of every camera. As AUs typically use deep learning-based AI/ML models, their performance depend on the quality of the input video, and recent work has shown that dynamically adjusting the camera setting exposed by popular network cameras can help improve the quality of the video feed and hence the AU accuracy, in a single AU setting. In this paper, we first show that in a multi-AU setting, changing the camera setting has disproportionate impact on different AUs performance. In particular, the optimal setting for one AU may severely degrade the performance for another AU, and further the impact on different AUs varies as the environmental condition changes. We then present Elixir, a system to enhance the video stream quality for multiple analytics on a video stream. Elixir leverages Multi-Objective Reinforcement Learning (MORL), where the RL agent caters to the objectives from different AUs and adjusts the camera setting to simultaneously enhance the performance of all AUs. To define the multiple objectives in MORL, we develop new AU-specific quality estimator values for each individual AU. We evaluate Elixir through real-world experiments on a testbed with three cameras deployed next to each other (overlooking a large enterprise parking lot) running Elixir and two baseline approaches, respectively. Elixir correctly detects 7.1% (22,068) and 5.0% (15,731) more cars, 94% (551) and 72% (478) more faces, and 670.4% (4975) and 158.6% (3507) more persons than the default-setting and time-sharing approaches, respectively. It also detects 115 license plates, far more than the time-sharing approach (7) and the default setting (0).

Road damages such as potholes may occur on roads due to aging, which may affect the traffic. Periodic inspections of road damages are difficult due to the high cost of road surveys, and we tend to overlook road damages which is therefore considered to be a problem in a long-term. The development of a system that automatically detects potholes and other road damages from dash cam images can allow inexpensive roadside inspections, and can overall improve the problem of the long-term oversight of road damages. Last year, we conducted a demonstration experiment in Edogawa City, Tokyo, using an existing image-based road damage detection method. From that experiment, we confirmed that the detection of potholes on actual roads often causes false detections due to the presence of shadows and manholes. In this study, we propose a method to reduce false positives in pothole detection which was considered to be a problem through the demonstration experiment, and evaluate its performance. Since we believe that the evaluation based on a pothole-only dataset is not valid, we reconstruct a dataset for evaluation by adding shadow and manhole images for validation. Our method consists of two main components: data expansion by image generation and ensemble of classification mechanisms for object detection models. As a result of the validation on the reconstructed pothole dataset, the average precision (AP), which is a measure to evaluate false positives, was improved by 0.172 compared to the existing method. In addition, the reduction of AR (average Recall), which is a trade-off with AP, was suppressed to 0.07. Since our method is not dependent on the domain of potholes, it is expected to be an effective pipeline in tasks and situations where false positives are more problematic than false negatives due to the high incidence of false positives.

The use of supervised learning for Human Activity Recognition (HAR) on mobile devices leads to strong classification performances. Such an approach, however, requires large amounts of labeled data, both for the initial training of the models and for their customization on specific clients (whose data often differ greatly from the training data). This is actually impractical to obtain due to the costs, intrusiveness, and time-consuming nature of data annotation. Moreover, even with the help of a significant amount of labeled data, model deployment on heterogeneous clients faces difficulties in generalizing well on unseen data. Other domains, like Computer Vision or Natural Language Processing, have proposed the notion of pre-trained models, leveraging large corpora, to reduce the need for annotated data and better manage heterogeneity. This promising approach has not been implemented in the HAR domain so far because of the lack of public datasets of sufficient size. In this paper, we propose a novel strategy to combine publicly available datasets with the goal of learning a generalized HAR model that can be fine-tuned using a limited amount of labeled data on an unseen target domain. Our experimental evaluation, which includes experimenting with different state-of-the-art neural network architectures, shows that combining public datasets can significantly reduce the number of labeled samples required to achieve satisfactory performance on an unseen target domain.

To access all the potential value present in IoT, the IoT devices need to be interoperable. Some works in the literature target this issue, but it is not yet entirely solved, mainly because the proposed solutions are not standard based at the semantic level. This paper presents the detailed implementation of our standard-based software framework targeting IoT interoperability, named NextGenGW. In this IoT gateway-based software framework, we propose the first integration of IETF SDF with the MQTT protocol. We define an evaluation baseline for validating IoT gateway performance while focusing on interoperability. Our evaluation results shows the NextGenGW suitability for slow processes, as well as demanding use cases where it needs to be deployed in a device with reduced resources, considering its scalability both in terms of connected IoT end nodes and number of requests per time interval.

This paper introduces a novel weight-dropout approach to train a neural network controller in real-time closed-loop control and to accelerate the embedded implementation for a solar inverter. The essence of the approach is to drop small-magnitude weights of neural network controllers during training with the goal of minimizing the required numbers of connections and guaranteeing the convergence of the neural network controllers. In order not to affect the convergence of neural network controllers, only non-diagonal elements of the neural network weight matrices were dropped. The dropout approach was incorporated into Levenberg-Marquardt and Forward Accumulation Through Time algorithms to train the neural network controller for trajectory tracking more efficiently. The Field Programmable Gate Array (FPGA) implementation on the Intel Cyclone V board shows significant improvement in terms of computation and resource requirements using the sparse weight matrices after dropout, which makes the neural network controller more suitable in an embedded environment.

Millimeter-wave (MMW) radar is becoming an essential sensing technology in smart environments due to its light and weather-independent sensing capability. Such capabilities have been widely explored and integrated with intelligent vehicle systems, often deployed in industry-grade MMW radars. However, industry-grade MMW radars are often expensive and difficult to attain for deployable community-purpose smart environment applications. On the other hand, commercially available MMW radars pose hidden underpinning challenges that are yet to be well investigated for tasks such as recognizing objects, and activities, real-time person tracking, object localization, etc. Such tasks are frequently accompanied by image and video data, which are relatively easy for an individual to obtain, interpret, and annotate. However, image and video data are light and weather-dependent, vulnerable to the occlusion effect, and inherently raise privacy concerns for individuals. It is crucial to investigate the performance of an alternative sensing mechanism where commercially available MMW radars can be a viable alternative to eradicate the dependencies and preserve privacy issues. Before championing MMW radar, several questions need to be answered regarding MMW radar's practical feasibility and performance under different operating environments. To answer the concerns, we have collected a dataset using commercially available MMW radar, Automotive mmWave Radar (AWR2944) from Texas Instruments, and reported the optimum experimental settings for object recognition performance using several deep learning algorithms in this study. Moreover, our robust data collection procedure allows us to systematically study and identify potential challenges in the object recognition task under a cross-ambience scenario. We have explored the potential approaches to overcome the underlying challenges and reported extensive experimental results.

Public transit is a vital mode of transportation in urban areas, and its efficiency is crucial for the daily commute of millions of people. To improve the reliability and predictability of transit systems, researchers have developed separate single-task learning models to predict the occupancy and delay of buses at the stop or route level. However, these models provide a narrow view of delay and occupancy at each stop and do not account for the correlation between the two. We propose a novel approach that leverages broader generalizable patterns governing delay and occupancy for improved prediction. We introduce a multitask learning toolchain that takes into account General Transit Feed Specification feeds, Automatic Passenger Counter data, and contextual information temporal and spatial information. The toolchain predicts transit delay and occupancy at the stop level, improving the accuracy of the predictions of these two features of a trip given sparse and noisy data. We also show that our toolchain can adapt to fewer samples of new transit data once it has been trained on previous routes/trips as compared to state-of-the-art methods. Finally, we use actual data from Chattanooga, Tennessee, to validate our approach. We compare our approach against the state-of-the-art methods and we show that treating occupancy and delay as related problems improves the accuracy of the predictions. We show that our approach improves delay prediction significantly by as much as 6% in F1 scores while producing equivalent or better results for occupancy.

Wearable Internet of Things (IoT) devices with inertial sensors can enable personalized and fine-grained Human Activity Recognition (HAR). While activity classification on the Extreme Edge (EE) can reduce latency and maximize user privacy, it must tackle the unique challenges posed by the constrained environment. Indeed, Deep Learning (DL) techniques may not be applicable, and data processing can become burdensome due to the lack of input systems. In this paper, we address those issues by proposing, implementing, and validating an EE-aware HAR system. Our system incorporates a feature selection mechanism to reduce the data dimensionality in input, and an unsupervised feature separation and classification technique based on Self-Organizing Maps (SOMs). We developed the system on an M5Stack IoT prototype board and implemented a new SOM library for the Arduino SDK. Experimental results on two HAR datasets show that our proposed solution is able to overcome other unsupervised approaches and achieve performance close to state-of-art DL techniques while generating a model small enough to fit the limited memory capabilities of EE devices.

Camouflaged object detection is a challenging task in computer vision that involves identifying objects that are intentionally or unintentionally hidden in their surrounding environment. Vision Transformer mechanisms play a critical role in improving the performance of deep learning models by focusing on the most relevant features that help object detection under camouflaged conditions. In this paper, we utilized a vision transformer (VT) in two phases, a) By integrating VT with the deep learning architecture for efficient monocular depth map generation for camouflaged objects and b) By embedding VT in the architecture of the multiclass object detection model with multimodal feature input (RGB with RGB-D) that increases the visual cues and provides more representational information to the model for performance enhancement. Additionally, we performed an ablation study to understand the role of the vision transformer in camouflaged object detection and incorporated GRAD-CAM on top of the model to visualize the performance improvement achieved by embedding the VT in the model architecture. We deployed the model in the resource-constrained edge devices for real-time object detection to realistically test the performance of the trained model.

Remote Photoplethysmography (rPPG) systems offer contactless, low-cost, and ubiquitous heart rate (HR) monitoring by leveraging the skin-tissue blood volumetric variation-induced reflection. However, collecting large-scale time-synchronized rPPG data is costly and impedes the development of generalized end-to-end deep learning (DL) rPPG models to perform under diverse scenarios. We formulate the rPPG estimation as a generative task of recovering time-series PPG from facial videos and propose SrPPG, a novel semi-supervised adversarial learning framework using heterogeneous, asynchronous, and noisy rPPG data. More specifically, we develop a novel encoder-decoder architecture, where rPPG features are learned from video in a self-supervised manner (encoder) to reconstruct the time-series PPG (decoder/generator) with physics-inspired novel temporal consistency regularization. The generated PPG is scrutinized against the real rPPG signals by a frequency-class conditioned discriminator, forming a generative adversarial network. Thus, SrPPG generates samples without point-wise supervision, alleviating the need for time-synchronized data collection. We experiment and validate SrPPG by amassing three public datasets in heterogeneous settings. SrPPG outperforms both supervised and self-supervised state-of-the-art methods in HR estimation across all datasets without any time-synchronous rPPG data. We also perform extensive experiments to study the optimal generative setting (architecture, joint optimization) and provide insight into the SrPPG behavior.

The annotation load for a new dataset has been greatly decreased using domain adaptation based semantic segmentation, which iteratively constructs pseudo labels on unlabeled target data and retrains the network. However, realistic segmentation datasets are often imbalanced, with pseudo-labels tending to favor certain head classes while neglecting other tail classes. This can lead to an inaccurate and noisy mask. To address this issue, we propose a novel hard sample mining strategy for an active domain adaptation based semantic segmentation network, with the aim of automatically selecting a small subset of labeled target data to fine-tune the network. By calculating class-wise entropy, we are able to rank the difficulty level of different samples. We use a fusion of focal loss and regional mutual information loss instead of cross-entropy loss for the domain adaptation based semantic segmentation network. Our entire framework has been implemented in real-time using the Robotics Operating System (ROS) with a server PC and a small Unmanned Ground Vehicle (UGV) known as the ROSbot2.0 Pro. This implementation allows ROSbot2.0 Pro to access any type of data at any time, enabling it to perform a variety of tasks with ease. Our approach has been thoroughly evaluated through a series of extensive experiments, which demonstrate its superior performance compared to existing state-of-the-art methods. Remarkably, by using just 20% of hard samples for fine-tuning, our network has achieved a level of performance that is comparable (≈ 88%) to that of a fully supervised approach, with mIOU scores of 60.51% in the In-house dataset.

On-device training is becoming a novel way to deliver intelligence into low cost hardware e.g., Micro Controller Units (MCUs) for the realization of low power tailored applications. However, the training of deep learning models on embedded systems is a very challenging process mainly due to their low amount of memory, available energy, and computing power which significantly limit the complexity of the tasks that can be executed, thus making impossible the use of traditional training algorithms such as backpropagation (BP). During these years techniques such as weights compression and quantization have emerged as solutions, but they only address the inference phase. Forward-Forward (FF) is a novel training algorithm that has been recently proposed as a possible alternative to BP when the available resources are limited. This is achieved by training the layers of a neural network separately, thus reducing the amount of required energy and memory. In this paper, we propose a variation of the original FF which tackles the training process with a multivariate Ridge regression approach and adopts the Mean Squared Error (MSE) as loss function. Such an approach does not use BP and does not need to compute gradients, thus saving memory and computing resources to enable the on-device training directly on MCUs of the STM32 family. Experimental results conducted on the Fashion-MNIST dataset demonstrate the effectiveness of the proposed approach in terms of memory and accuracy.

Existing massive deployments of IoT devices in support of smart computing applications across a range of domains must leverage critical features of 5G, such as network slicing, to receive differentiated and reliable services. However, the voluminous, dynamic, and heterogeneous nature of IoT traffic imposes complexities on the problems of network flow classification, network traffic analysis, and accurate quantification of the network requirements, thereby making the provisioning of 5G network slices across the application mix a challenging problem. To address these needs, we propose a novel network traffic classification approach that consists of a pipeline that combines Principal Component Analysis (PCA), with KMeans clustering and Hellinger distance. PCA is applied as the first step to efficiently reduce the dimensionality of features while preserving as much of the original information as possible. This significantly reduces the runtime of KMeans, which is applied as the second step. KMeans, being an unsupervised approach, eliminates the need to label data which can be cumbersome, error-prone, and time-consuming. In the third step, a Hellinger distance-based recursive KMeans algorithm is applied to merge similar clusters toward identifying the optimal number. This makes the final clustering results compact and intuitively interpretable within the context of the problem, while addressing the limitations of traditional KMeans algorithm, such as sensitivity to initialization and the requirement of manual specification of the number of clusters. Evaluation of our approach on a real-world IoT dataset demonstrates that the pipeline can compactly represent the dataset as three clusters. The service properties of these clusters can be easily inferred and directly mapped to different types of slices in the 5G network.

Breathing exercises are gaining attention in managing anxiety and stress in daily life. Diaphragmatic breathing, in particular, fosters tranquility for both body and mind. Existing use cases, such as meditation, yoga, and medical devices for guided breathing, often require expert guidance, complex instruments, cumbersome devices, and sticky electrodes. To address these challenges, we present Nisshash, an IoT-based smart T-shirt offering a personalized solution for regulated breathing exercises. Nisshash is embedded with three-channel e-textile respiration sensors and a tailored analog front-end (AFE) board to simultaneously monitor respiration rate (RR) and heart rate (HR). In this work, we seamlessly integrate soft textile sensors into a T-shirt and develop a detachable and Wi-Fi-enabled (2.4GHz) bio-instrumentation board, creating a pervasive wireless system (WPS) for guided breathing exercises (GBE). The system features an intuitive graphical user interface (GUI) and a seamless IoT-based control and computing system (CCS). It offers realtime instructions for inhaling and exhaling at various breathing speeds, including slow, normal, and fast breathing. Functions such as filtering, peak detections for respiration, and heart rate analysis are computed conjointly at the sender and receiver ends. We utilized the Pan-Tompkins and custom algorithms to calculate HR and RR from the filtered time-series signals. We conducted a study with 10 healthy adult participants who wore the Tshirt and performed guided breathing exercises. The average respiration event (inhale-exhale) detection accuracy was ≈ 98%. We validated the recorded HR against the 3-lead PC-80B ECG monitoring device, achieving an accuracy of ≈ 99%. The RRHR correlation analysis showed an R square value of 0.987. Collectively, these results demonstrate Nisshash's potential as a personal guided breathing exercise solution.

In smart computing, the labels of training samples for a specific task are not always abundant. However, the labels of samples in a relevant but different dataset are available. As a result, researchers have relied on unsupervised domain adaptation to leverage the labels in a dataset (the source domain) to perform better classification in a different, unlabeled dataset (target domain). Existing non-generative adversarial solutions for UDA aim at achieving domain confusion through adversarial training. The ideal scenario is that perfect domain confusion is achieved, but this is not guaranteed to be true. To further enforce domain confusion on top of the adversarial training, we propose a novel UDA algorithm, E-ADDA, which uses both a novel variation of the Mahalanobis distance loss and an out-of-distribution detection subroutine. The Mahalanobis distance loss minimizes the distribution-wise distance between the encoded target samples and the distribution of the source domain, thus enforcing additional domain confusion on top of adversarial training. Then, the OOD subroutine further eliminates samples on which the domain confusion is unsuccessful. We have performed extensive and comprehensive evaluations of E-ADDA in the acoustic and computer vision modalities. In the acoustic modality, E-ADDA outperforms several state-of-the-art UDA algorithms by up to 29.8%, measured in the f1 score. In the computer vision modality, the evaluation results suggest that we achieve new state-of-the-art performance on popular UDA benchmarks such as Office-31 and Office-Home, outperforming the second best-performing algorithms by up to 17.9%.

Variable speed limit (VSL) control has emerged as a promising traffic management strategy for enhancing safety and mobility. In this study, we introduce a multi-agent reinforcement learning framework for implementing a large-scale VSL system to address recurring congestion in transportation corridors. The VSL control problem is modeled as a Markov game, using only data widely available on freeways. By employing parameter sharing among all VSL agents, the proposed algorithm can efficiently scale to cover extensive corridors. The agents are trained using a reward structure that incorporates adaptability, safety, mobility, and penalty terms; enabling agents to learn a coordinated policy that effectively reduces spatial speed variations while minimizing the impact on mobility. Our findings reveal that the proposed algorithm leads to a significant reduction in speed variation, which holds the potential to reduce incidents. Furthermore, the proposed approach performs satisfactorily under varying traffic demand and compliance rates.

We present Application in a Box (AnB) product concept aimed at simplifying the deployment and operation of remote 5G applications. AnB comes pre-configured with all necessary hardware and software components, including sensors like cameras, hardware and software components for a local 5G wireless network, and 5G-ready apps. Enterprises can easily download additional apps from an App Store. Setting up a 5G infrastructure and running applications on it is a significant challenge, but AnB is designed to make it fast, convenient, and easy, even for those without extensive knowledge of software, computers, wireless networks, or AI-based analytics. With AnB, customers only need to open the box, set up the sensors, turn on the 5G networking and edge computing devices, and start running their applications. Our system software automatically deploys and optimizes the pipeline of microservices in the application on a tiered computing infrastructure that includes device, edge, and cloud computing. Dynamic resource management, placement of critical tasks for low-latency response, and dynamic network bandwidth allocation for efficient 5G network usage are all automatically orchestrated. AnB offers cost savings, simplified setup and management, and increased reliability and security. We've implemented several real-world applications, such as collision prediction at busy traffic light intersections and remote construction site monitoring using video analytics. With AnB, deployment and optimization effort can be reduced from several months to just a few minutes. This is the first-of-its-kind approach to easing deployment effort and automating self-optimization of the application during system operation.