Ask or search anything...
CNRSRendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method
13 Feb 2025
Université Paris-SaclayA novel zero-order federated learning method, 1P-ZOFL, integrates the inherent characteristics of wireless communication channels directly into the gradient estimation process, enabling substantial communication efficiency through scalar value transmission while offering robust, almost sure convergence for nonconvex objectives. This approach eliminates the need for costly channel state information estimation in distributed learning environments.
View blog
Joint model with latent disease age: overcoming the need for reference time
InriaIntroduction: Heterogeneity of the progression of neurodegenerative diseases
is one of the main challenges faced in developing effective therapies. With the
increasing number of large clinical databases, disease progression models have
led to a better understanding of this heterogeneity. Nevertheless, these
diseases may have no clear onset and biological underlying processes may start
before the first symptoms. Such an ill-defined disease reference time is an
issue for current joint models, which have proven their effectiveness by
combining longitudinal and survival data.
Objective In this work, we propose a joint non-linear mixed effect model with
a latent disease age, to overcome this need for a precise reference time.
Method: To do so, we utilized an existing longitudinal model with a latent
disease age as a longitudinal sub-model and associated it with a survival
sub-model that estimates a Weibull distribution from the latent disease age. We
then validated our model on different simulated scenarios. Finally, we
benchmarked our model with a state-of-the-art joint model and reference
survival and longitudinal models on simulated and real data in the context of
Amyotrophic Lateral Sclerosis (ALS).
Results: On real data, our model got significantly better results than the
state-of-the-art joint model for absolute bias (4.21(4.41) versus
4.24(4.14)(p-value=1.4e-17)), and mean cumulative AUC for right censored events
(0.67(0.07) versus 0.61(0.09)(p-value=1.7e-03)).
Conclusion: We showed that our approach is better suited than the
state-of-the-art in the context where the reference time is not reliable. This
work opens up the perspective to design predictive and personalized therapeutic
strategies.
GA-NIFS: The highly overdense system BR1202-0725 at z 4.7. A double AGN with fast outflows plus eight companion galaxies
03 Dec 2024
University of OxfordDistant quasars (QSOs) in galaxy overdensities are considered key actors in the evolution of the early Universe. In this work, we studied the kinematic and physical properties of the BR1202-0725 system at z=4.7, one of the most overdense fields known in the early Universe, consisting of a QSO, a submillimeter galaxy (SMG), and three Lyman- emitters. We used data from the JWST/NIRSpec Integral Field Unit (IFU) to analyze the rest-frame optical emission of each source in the system. We estimated a bolometric luminosity of log([erg/s]) = 47.2 0.4 and a black hole mass of log() = 10.1 0.5 for the QSO, which are consistent with previous measurements obtained with ground-based observations. The NIRSpec spectra of the SMG revealed instead unexpected [OIII] and H+[NII] profiles. The overall [OIII] line profile is blue-shifted by more than 700 km/s relative to the systemic velocity of the galaxy. Additionally, both the [OIII] and H+[NII] lines show prominent broad (1300 km/s), blueshifted wings associated with outflowing ionized gas. The analysis of NIRSpec and X-ray observations indicates that the SMG likely hosts an accreting supermassive black hole as supported by the following results: (i) the excitation diagnostic diagram is consistent with ionization from an active galactic nucleus (AGN); (ii) the X-ray luminosity is higher than erg/s; and (iii) it hosts a fast outflow ( = 5000 km/s), comparable to those observed in luminous QSOs. Therefore, the QSO-SMG pair represents one of the highest-redshift double AGN to date, with a projected separation of 24 kpc. Finally, we investigated the environment of this system and found four new galaxies at the same redshift of the QSO and within a projected distance of 5 kpc from it. This overdense system includes at least ten galaxies in only 980 kpc.
Node-layer duality in networked systems
11 May 2024
Sorbonne UniversitéReal-world networks typically exhibit several aspects, or layers, of interactions among their nodes. By permuting the role of the nodes and the layers, we establish a new criterion to construct the dual of a network. This approach allows to examine information from either a node-centric or layer-centric viewpoint. Through rigorous analytical methods and extensive simulations, we demonstrate that nodewise and layerwise connectivity measure different but related aspects of the same system. Leveraging node-layer duality provides complementary insights, enabling a deeper comprehension of diverse networks across social science, technology and biology. Taken together, these findings reveal previously unappreciated features of complex systems and provide a fresh tool for delving into their structure and dynamics.
Multiport Network Theory for Modeling and Optimizing Reconfigurable Metasurfaces
29 Nov 2024
This overview consolidates research on Multiport Network Theory (MNT) as a foundational framework for modeling and optimizing Reconfigurable Intelligent Surfaces (RIS) in wireless communication systems. It demonstrates MNT's capability to provide electromagnetically consistent models that accurately capture mutual coupling and structural scattering, facilitating the design and optimization of smart radio environments.
View blog
Strong spectral features from asymptotic giant branch stars in distant quiescent galaxies
03 Nov 2024
University of CambridgeDating the ages and weighting the stellar populations in galaxies are essential steps when studying galaxy formation through cosmic times. Evolutionary population synthesis models with different input physics are used for this purpose. Moreover, the contribution from the thermally pulsing asymptotic giant branch (TP-AGB) stellar phase, which peaks for intermediate-age 0.6-2 Gyr, has been debated for decades. Here we report the detection of strong cool-star signatures in the rest-frame near-infrared spectra of three young (~1Gyr), massive (~10^10Msun) quiescent galaxies at large look-back time, z=1-2, using JWST/NIRSpec. The coexistence of oxygen- and carbon-type absorption features, spectral edges and features from rare species, such as vanadium and possibly zirconium, reveal a strong contribution from TP-AGB stars. Population synthesis models with a significant TP-AGB contribution reproduce the observations better than those with a weak TP-AGB, which are commonly used. These findings call for revisions of published stellar population fitting results, as they point to populations with lower masses and younger ages and have further implications for cosmic dust production and chemical enrichment. New generations of improved models are needed, informed by these and future observations.
A vision transformer-based framework for knowledge transfer from multi-modal to mono-modal lymphoma subtyping models
29 May 2024
A Vision Transformer-based framework streamlines Diffuse Large B-Cell Lymphoma (DLBCL) subtyping by transferring knowledge from multi-modal (HES + IHC) training to an efficient HES-only inference model. The approach achieves 75% accuracy, surpassing current deep learning techniques and HES-based human pathologist assessments.
View blog
Compressed representation of brain genetic transcription
20 Jun 2024
University College LondonResearchers at University College London and the University of Bordeaux systematically evaluated dimensionality reduction methods for whole-brain genetic transcription, establishing deep auto-encoders as a superior approach to traditional Principal Component Analysis (PCA). The auto-encoder provided greater fidelity in data reconstruction and generated more anatomically plausible latent structures, improving the prediction of diverse neurophysiological targets.
View blog
JWST and ALMA discern the assembly of structural and obscured components in a high-redshift starburst galaxy
10 May 2024
California Institute of TechnologyWe present observations and analysis of the starburst, PACS-819, at z=1.45 ( M), using high-resolution (; 0.8 kpc) ALMA and multi-wavelength JWST images from the COSMOS-Web program. Dissimilar to HST/ACS images in the rest-frame UV, the redder NIRCam and MIRI images reveal a smooth central mass concentration and spiral-like features, atypical for such an intense starburst. Through dynamical modeling of the CO J=5--4 emission with ALMA, PACS-819 is rotation-dominated thus has a disk-like nature. However, kinematic anomalies in CO and asymmetric features in the bluer JWST bands (e.g., F150W) support a more disturbed nature likely due to interactions. The JWST imaging further enables us to map the distribution of stellar mass and dust attenuation, thus clarifying the relationships between different structural components, not discernable in the previous HST images. The CO J = 5 -- 4 and FIR dust continuum emission are co-spatial with a heavily-obscured starbursting core (<1 kpc) which is partially surrounded by much less obscured star-forming structures including a prominent arc, possibly a tidally-distorted dwarf galaxy, and a clump, either a sign of an ongoing violent disk instability or a recently accreted low-mass satellite. With spatially-resolved maps, we find a high molecular gas fraction in the central area reaching (/) and short depletion times ( 120 Myrs) across the entire system. These observations provide insights into the complex nature of starbursts in the distant universe and underscore the wealth of complementary information from high-resolution observations with both ALMA and JWST.
Compression-based inference of network motif sets
A compression-based framework, grounded in the Minimum Description Length principle, was developed to robustly infer network motif sets and overcome statistical limitations of traditional methods. The framework accurately identifies significant motifs and fundamental network features in both synthetic and real-world neural connectomes, providing a more reliable understanding of network organization.
View blog
Individual based SIS models on (not so) dense large random networks
09 Sep 2024
Université de MontréalStarting from a stochastic individual-based description of an SIS epidemic spreading on a random network, we study the dynamics when the size of the network tends to infinity. We recover in the limit an infinite-dimensional integro-differential equation studied by Delmas, Dronnier and Zitt (2022) for an SIS epidemic propagating on a graphon. Our work covers the case of dense and sparse graphs, provided that the number of edges grows faster than , but not the case of very sparse graphs with edges. In order to establish our limit theorem, we have to deal with both the convergence of the random graphs to the graphon and the convergence of the stochastic process spreading on top of these random structures: in particular, we propose a coupling between the process of interest and an epidemic that spreads on the complete graph but with a modified infection rate.
Keywords: Random graph, mathematical models of epidemics, measure-valued process, large network limit, limit theorem, graphon.
Learning Properties of Quantum States Without the I.I.D. Assumption
14 Nov 2024
We develop a framework for learning properties of quantum states beyond the assumption of independent and identically distributed (i.i.d.) input states. We prove that, given any learning problem (under reasonable assumptions), an algorithm designed for i.i.d. input states can be adapted to handle input states of any nature, albeit at the expense of a polynomial increase in training data size (aka sample complexity). Importantly, this polynomial increase in sample complexity can be substantially improved to polylogarithmic if the learning algorithm in question only requires non-adaptive, single-copy measurements. Among other applications, this allows us to generalize the classical shadow framework to the non-i.i.d. setting while only incurring a comparatively small loss in sample efficiency. We use rigorous quantum information theory to prove our main results. In particular, we leverage permutation invariance and randomized single-copy measurements to derive a new quantum de Finetti theorem that mainly addresses measurement outcome statistics and, in turn, scales much more favorably in Hilbert space dimension.
Signatures of convection in the atmospheres of cool evolved stars
31 Jan 2024
Flatiron InstituteEvolved cool stars of various masses are major cosmic engines, delivering
substantial mechanical and radiative feedback to the interstellar medium
through strong stellar winds and supernova ejecta. These stars play a pivotal
role in enriching the interstellar medium with vital chemical elements that
constitute the essential building blocks for forming subsequent generations of
stars, planets, and potentially even life. Within the complex tapestry of
processes occurring in the atmospheres of these cool and luminous stars,
convection takes center stage. Convection is a non-local, complex phenomenon
marked by non-linear interactions across diverse length scales within a
multi-dimensional framework. For these particular stars, characterized by their
considerable luminosities and extensive scale heights, convection transitions
to a global scale. This transition is facilitated by transmitting radiative
energy through the non-uniform outer layers of their atmospheres. To fully
understand this phenomenon, the application of global comprehensive 3D
radiation-hydrodynamics simulations of stellar convection is of paramount
importance. We present two state-of-the-art numerical codes: CO5BOLD and
Athena++. Furthermore, we provide a view on their applications as: pivotal
roles in enabling a comprehensive investigation into the dynamic processes
linked to convection; and critical tools for accurately modeling the emissions
produced during shock breakouts in Type II-P Supernovae.
FiLM: Visual Reasoning with a General Conditioning Layer
18 Dec 2017
MILA researchers propose FiLM (Feature-wise Linear Modulation), a general conditioning layer that enables neural networks to dynamically adapt their computation based on conditioning input. Applied to visual reasoning, FiLM achieves 97.7% accuracy on the CLEVR benchmark, significantly reducing the error rate for models without explicit architectural priors or extra supervision.
View blog
VICReg: Variance-Invariance-Covariance Regularization for
Self-Supervised Learning
28 Jan 2022
New York UniversityRecent self-supervised methods for image representation learning are based on
maximizing the agreement between embedding vectors from different views of the
same image. A trivial solution is obtained when the encoder outputs constant
vectors. This collapse problem is often avoided through implicit biases in the
learning architecture, that often lack a clear justification or interpretation.
In this paper, we introduce VICReg (Variance-Invariance-Covariance
Regularization), a method that explicitly avoids the collapse problem with a
simple regularization term on the variance of the embeddings along each
dimension individually. VICReg combines the variance term with a decorrelation
mechanism based on redundancy reduction and covariance regularization, and
achieves results on par with the state of the art on several downstream tasks.
In addition, we show that incorporating our new variance term into other
methods helps stabilize the training and leads to performance improvements.
DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment
MetaThis paper introduces a method to align DINOv2's visual features with text for both image-level and pixel-level tasks
View blog
Computational Optimal Transport
This survey provides a comprehensive review of Optimal Transport (OT) theory, with a focus on its computational methods and applications in data sciences. It highlights how entropic regularization, particularly through the Sinkhorn-Knopp algorithm, has made OT computationally feasible for large-scale problems, detailing various formulations and their use across machine learning, computer vision, and statistics.
View blog
On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity
Modern deep generative models can now produce high-quality synthetic samples that are often indistinguishable from real training data. A growing body of research aims to understand why recent methods, such as diffusion and flow matching techniques, generalize so effectively. Among the proposed explanations are the inductive biases of deep learning architectures and the stochastic nature of the conditional flow matching loss. In this work, we rule out the noisy nature of the loss as a key factor driving generalization in flow matching. First, we empirically show that in high-dimensional settings, the stochastic and closed-form versions of the flow matching loss yield nearly equivalent losses. Then, using state-of-the-art flow matching models on standard image datasets, we demonstrate that both variants achieve comparable statistical performance, with the surprising observation that using the closed-form can even improve performance.
Cluster and Predict Latent Patches for Improved Masked Image Modeling
Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this paper, we systematically analyze target representations, loss functions, and architectures, to introduce CAPI - a novel pure-MIM framework that relies on the prediction of latent clusterings. Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties. Our ViT-L backbone, CAPI, achieves 83.8% accuracy on ImageNet and 32.1% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2. We release all our code and models.
On The Landscape of Spoken Language Models: A Comprehensive Survey
11 Apr 2025
Carnegie Mellon UniversityA comprehensive survey examines the landscape of Spoken Language Models (SLMs), analyzing architectures, training strategies, and evaluation methods across pure SLMs, speech+text SLMs, and speech-aware text LMs while establishing unified terminology and identifying key challenges in developing universal speech processing systems.
View blog
There are no more papers matching your filters at the moment.