Bar-Ilan Faculty of Engineering

In recent years, it has become popular to tackle image restoration tasks with a single pretrained diffusion model (DM) and data-fidelity guidance, instead of training a dedicated deep neural network per task. However, such "zero-shot" restoration schemes currently require many Neural Function Evaluations (NFEs) for performing well, which may be attributed to the many NFEs needed in the original generative functionality of the DMs. Recently, faster variants of DMs have been explored for image generation. These include Consistency Models (CMs), which can generate samples via a couple of NFEs. However, existing works that use guided CMs for restoration still require tens of NFEs or fine-tuning of the model per task that leads to performance drop if the assumptions during the fine-tuning are not accurate. In this paper, we propose a zero-shot restoration scheme that uses CMs and operates well with as little as 4 NFEs. It is based on a wise combination of several ingredients: better initialization, back-projection guidance, and above all a novel noise injection mechanism. We demonstrate the advantages of our approach for image super-resolution, deblurring and inpainting. Interestingly, we show that the usefulness of our noise injection technique goes beyond CMs: it can also mitigate the performance degradation of existing guided DM methods when reducing their NFE count.

Enhancing resilience in distributed networks in the face of malicious agents is an important problem for which many key theoretical results and applications require further development and characterization. This work focuses on the problem of distributed optimization in multi-agent cyberphysical systems, where a legitimate agent's dynamic is influenced both by the values it receives from potentially malicious neighboring agents, and by its own self-serving target function. We develop a new algorithmic and analytical framework to achieve resilience for the class of problems where stochastic values of trust between agents exist and can be exploited. In this case we show that convergence to the true global optimal point can be recovered, both in mean and almost surely, even in the presence of malicious agents. Furthermore, we provide expected convergence rate guarantees in the form of upper bounds on the expected squared distance to the optimal value. Finally, we present numerical results that validate the analytical convergence guarantees we present in this paper even when the malicious agents compose the majority of agents in the network.

Temporal alignment of sequences is a fundamental challenge in many applications, such as computer vision and bioinformatics, where local time shifting needs to be accounted for. Misalignment can lead to poor model generalization, especially in high-dimensional sequences. Existing methods often struggle with optimization when dealing with high-dimensional sparse data, falling into poor alignments. Feature selection is frequently used to enhance model performance for sparse data. However, a fixed set of selected features would not generally work for dynamically changing sequences and would need to be modified based on the state of the sequence. Therefore, modifying the selected feature based on contextual input would result in better alignment. Our suggested method, Conditional Deep Canonical Temporal Time Warping (CDCTW), is designed for temporal alignment in sparse temporal data to address these challenges. CDCTW enhances alignment accuracy for high dimensional time-dependent views be performing dynamic time warping on data embedded in maximally correlated subspace which handles sparsity with novel feature selection method. We validate the effectiveness of CDCTW through extensive experiments on various datasets, demonstrating superior performance over previous techniques.

Training and fine-tuning large language models (LLMs) come with challenges related to memory and computational requirements due to the increasing size of the model weights and the optimizer states. Various techniques have been developed to tackle these challenges, such as low-rank adaptation (LoRA), which involves introducing a parallel trainable low-rank matrix to the fixed pre-trained weights at each layer. However, these methods often fall short compared to the full-rank weight training approach, as they restrict the parameter search to a low-rank subspace. This limitation can disrupt training dynamics and require a full-rank warm start to mitigate the impact. In this paper, we introduce a new method inspired by a phenomenon we formally prove: as training progresses, the rank of the estimated layer gradients gradually decreases, and asymptotically approaches rank one. Leveraging this, our approach involves adaptively reducing the rank of the gradients during Adam optimization steps, using an efficient online-updating low-rank projections rule. We further present a randomized SVD scheme for efficiently finding the projection matrix. Our technique enables full-parameter fine-tuning with adaptive low-rank gradient updates, significantly reducing overall memory requirements during training compared to state-of-the-art methods while improving model performance in both pretraining and fine-tuning. Finally, we provide a convergence analysis of our method and demonstrate its merits for training and fine-tuning language and biological foundation models.

Temporal optics rises from the equivalence between light diffraction in free space and pulse dispersion in dispersive media, paving the way for the development of temporal devices and applications, such as time-lenses. A Four-wave mixing based time-lens allows single-shot measurements of ultra-short signals in high temporal resolution by imaging signals, and inducing temporal Fourier transform. We introduce a cascade time-lens by utilizing a cascade FWM process within the time-lens. We theoretically develop and experimentally demonstrate the cascade time-lens, and confirm that different cascade orders correspond to different effective temporal systems, leading to measuring in various temporal imaging configurations simultaneously with a single optical setup. This approach can simplify experiments and provide a more comprehensive view of a signal’s phase and temporal structure. Such capabilities are …

Accurate and reliable identification of the RTF between microphones with respect to a desired source is an essential component in the design of microphone array beamformers, specifically the MVDR criterion. Since an accurate estimation of the RTF in a noisy and reverberant environment is a cumbersome task, we aim at leveraging prior knowledge of the acoustic enclosure to robustify the RTF estimation by learning the RTF manifold. In this paper, we present a novel robust RTF identification method, tested and trained with real recordings, which relies on learning the RTF manifold using a GCN to infer a robust representation of the RTF in a confined area, and consequently enhance the beamformer's performance.

Most emotion recognition systems fail in real-life situations (in the wild scenarios) where the audio is contaminated by reverberation. Our study explores new methods to alleviate the performance degradation of Speech Emotion Recognition (SER) algorithms and develop a more robust system for adverse conditions. We propose processing multi-microphone signals to address these challenges and improve emotion classification accuracy. We adopt a state-of-the-art transformer model, the Hierarchical Token-semantic Audio Transformer (HTS-AT), to handle multi-channel audio inputs. We evaluate two strategies: averaging mel-spectrograms across channels and summing patch-embedded representations. Our multimicrophone model achieves superior performance compared to single-channel baselines when tested on real-world reverberant environments.

We study the question of whether a sequence of positive integers is the degree sequence of some outerplanar (a.k.a. 1-page book embeddable) graph G. If so, G is an outerplanar realization of d and d is an outerplanaric sequence. The case where is easy, as d has a realization by a forest (which is trivially an outerplanar graph). In this paper, we consider the family of all sequences d of even sum , where is the number of x’s in d. (The second inequality is a necessary condition for a sequence d with to be outerplanaric.) We partition into two disjoint subfamilies, , such that every sequence in is provably non-outerplanaric, and every sequence in is given a realizing graph G enjoying a 2-page book embedding (and moreover, one of the pages is also bipartite).

We consider the problem of characterizing degree sequences that can be realized by a bipartite graph. If a partition of the sequence into the two sides of the bipartite graph is given as part of the input, then there is a complete characterization that was established more than 60 years ago. However, the general question, in which a partition and a realizing graph need to be determined, is still open. We investigate the role of an important class of special partitions, called High-Low partitions, which separate the degrees of a sequence into two groups, the high degrees and the low degrees. We show that when the High-Low partition exists and satisfies some natural properties, analyzing the High-Low partition resolves the bigraphic realization problem. For sequences that are known to be not realizable by a bipartite graph or that are undecided, we provide approximate realizations based on the High-Low partition.

Understanding oscillatory behavior in human networks is essential for exploring synchronization, coordination, and collective dynamics. In this study, we investigate tempo oscillations in complex human networks using a system of coupled violin players with precisely controlled network parameters. Each player interacts via delayed auditory feedback, allowing us to explore the effects of connectivity, delay, and tempo on network oscillations. We identify two distinct types of oscillations: fast (2–3 seconds) and slow (5–15 seconds), and demonstrate that their periods are independent of network size and delay but are strongly correlated with the network's average tempo. Additionally, we show that increasing the number of coupled neighbors enhances oscillation damping, indicating the role of connectivity in stabilizing network dynamics. By varying the delay rate, we discover a critical decay rate where oscillation …

Temporal alignment of sequences is a fundamental challenge in many applications, such as computer vision and bioinformatics, where local time shifting needs to be accounted for. Misalignment can lead to poor model generalization, especially in high-dimensional sequences. Existing methods often struggle with optimization when dealing with high-dimensional sparse data, falling into poor alignments. Feature selection is frequently used to enhance model performance for sparse data. However, a fixed set of selected features would not generally work for dynamically changing sequences and would need to be modified based on the state of the sequence. Therefore, modifying the selected feature based on contextual input would result in better alignment. Our suggested method, Conditional Deep Canonical Temporal Time Warping (CDCTW), is designed for temporal alignment in sparse temporal data to address …

Optical nanoscopy is crucial in life and materials sciences, revealing subtle cellular processes and nanomaterial properties. Scattering-type Scanning Near-field Optical Microscopy (s-SNOM) provides nanoscale resolution, relying on the interactions taking place between a laser beam, a sharp tip and the sample. The Atomic Force Microscope (AFM) is a fundamental part of an s-SNOM system, providing the necessary probe-sample feedback mechanisms for data acquisition. In this Letter, we demonstrate that s-SNOM data can be partially inferred from AFM images. We first show that a generative artificial intelligence (AI) model (pix2pix) can generate synthetic s-SNOM data from experimental AFM images. Second, we demonstrate that virtual s-SNOM data can be extrapolated from knowledge of the tip position and, consequently, from AFM signals. To this end, we introduce an analytical model that explains the mechanisms underlying AFM-to-s-SNOM image translation. These insights have the potential to be integrated into future physics-informed explainable AI models. The two proposed approaches generate pseudo s-SNOM data without direct optical measurements, significantly expanding access to optical nanoscopy through widely available AFM systems. This advancement holds great promise for reducing both time and costs associated with nanoscale imaging.

Introducing a domain shift, such as a change in language or environment, to a well-trained speech enhancement system can cause severe performance degradation. Most current research assumes that a domain shift has already been detected and focuses on either supervised or unsupervised domain adaptation techniques. Here, we address the problem of automatically detecting when a domain shift has occurred. We present a domain shift detection method based on monitoring the confidence of a network that predicts the quality of enhanced speech. The experimental results show that our method can effectively detect a domain mismatch between the training and test sets.

Triggered by the development of exfoliation and the identification of a wide range of extraordinary physical properties in self-standing films consisting of one or few atomic layers, two-dimensional (2D) materials such as graphene, transition metal dichalcogenides (TMDs), and other van der Waals (vdW) crystals currently constitute a wide research field protruding in multiple directions in combination with layer stacking and twisting, nanofabrication, surface-science methods, and integration into nanostructured environments. Photonics encompasses a multidisciplinary collection of those directions, where 2D materials contribute with polaritons of unique characteristics such as strong spatial confinement, large optical-field enhancement, long lifetimes, high sensitivity to external stimuli (e.g., electric and magnetic fields, heating, and strain), a broad spectral range from the far infrared to the ultraviolet, and hybridization with spin and momentum textures of electronic band structures. The explosion of photonics with 2D materials as a vibrant research area is producing breakthroughs, including the discovery and design of new materials and metasurfaces with unprecedented properties as well as applications in integrated photonics, light emission, optical sensing, and exciting prospects for applications in quantum information, and nanoscale thermal transport. This Roadmap summarizes the state of the art in the field, identifies challenges and opportunities, and discusses future goals and how to meet them through a wide collection of topical sections prepared by leading practitioners.

In this paper, we study the problem of fair multi-agent multi-arm bandit learning when agents do not communicate with each other, except collision information, provided to agents accessing the same arm simultaneously. We provide an algorithm with regret O(N3f(log T)log T) (assuming bounded rewards, with unknown bound), where f(t) is any function diverging to infinity with t. In contrast to optimal algorithms which share the rewards with a selected leader, our algorithm does not require a centralized collection of the arm rewards, allowing each agent to keep its rewards private. We also significantly improved previous privacy-preserving algorithms with the same upper bound on the regret of order O(f(log T)log T) but an exponential dependence on the number of agents. Simulation results present the dependence of the regret on log T.

In this proceeding, we expand upon our recent work on the temporal Aharonov-Bohm (AB) effect, focusing on the joint temporal function (JSF) analysis to enhance the sensitivity and understanding of the system’s performance. Our original paper demonstrated the existence of a temporal analog of the Aharonov-Bohm effect using entangled photons and a temporal SU(1,1) interferometer. Here, we provide a more detailed exploration of the JSF, highlighting its role in improving signal-to-noise ratios (SNR) and its impact on measuring fast phase changes with enhanced temporal resolution. By leveraging the quantum properties of the system and utilizing the JSF between the signal and idler beams, we show that this approach allows for unprecedented sensitivity in detecting temporal phase shifts. This proceeding will also discuss how the JSF contributes to the interferometer’s ability to resolve femto-second dynamics …

In content-oblivious computation, n nodes wish to compute a given task over an asynchronous network that suffers from an extremely harsh type of noise, which corrupts the content of all messages across all channels. In a recent work, Censor-Hillel, Cohen, Gelles, and Sela (Distributed Computing, 2023) showed how to perform arbitrary computations in a content-oblivious way in 2-edge connected networks but only if the network has a distinguished node (called root) to initiate the computation. Our goal is to remove this assumption, which was conjectured to be necessary. Achieving this goal essentially reduces to performing a content-oblivious leader election since an elected leader can then serve as the root required to perform arbitrary content-oblivious computations. We focus on ring networks, which are the simplest 2-edge connected graphs. On \emph{oriented} rings, we obtain a leader election algorithm with message complexity O(n * ID_max, where ID_max is the maximal assigned ID. As it turns out, this dependency on ID_max is inherent: we show a lower bound of Omega(n log(ID_max/n)) messages for content-oblivious leader election algorithms. We also extend our results to \emph{non-oriented} rings, where nodes cannot tell which channel leads to which neighbor. In this case, however, the algorithm does not terminate but only reaches quiescence.

We study the question of whether a sequence d=(d 1,…, d n) of positive integers is the degree sequence of some outerplanar graph G. If so, G is an outerplanar realization of d and d is an outerplanaric sequence. The case where∑ d≤ 2 n− 2 is easy, as d has a realization by a forest. In this paper, we consider the family D of all sequences d of even sum 2 n≤∑ d≤ 4 n− 6− 2 ω 1, where ω x is the number of x's in d. We partition D into two disjoint subfamilies, D= D N O P∪ D 2 P B E, such that every sequence in D N O P is provably non-outerplanaric, and every sequence in D 2 P B E is given a realizing graph G enjoying a 2-page book embedding (and moreover, one of the pages is also bipartite).

Leakage-resilient cryptography aims to protect cryptographic primitives from so-called “side channel attacks” that exploit their physical implementation to learn their input or secret state. Starting from the works of Ishai, Sahai and Wagner (CRYPTO ‘03) and Micali and Reyzin (TCC ‘04), most works on leakage-resilient cryptography either focus on protecting general computations, such as circuits or multiparty computation protocols, or on specific non-interactive primitives such as storage, encryption, and signatures. This work focuses on leakage resilience for the middle ground, namely for distributed and interactive cryptographic primitives. Our main technical contribution is designing the first secret sharing scheme that is equivocal, resists adaptive probing of a constant fraction of bits from each share, while incurs only a constant blowup in share size. Equivocation is a strong leakage-resilience guarantee, recently …