Ask or search anything...
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
14 May 2025
Anikait Singh
Haochen Shi
Google DeepMind
University of Illinois at Urbana-ChampaignThe OpenX-Embodiment Collaboration released the Open X-Embodiment (OXE) Dataset, a consolidated collection of over 1 million real robot trajectories from 22 embodiments. This work demonstrates that large RT-X models trained on such diverse data achieve positive transfer and emergent skills across different robot platforms.
View blog
OASIS: Open Agent Social Interaction Simulations with One Million Agents
23 Mar 2025
shuyue hu
Prateek Gupta
Imperial College LondonOASIS presents an open agent social interaction simulator capable of scaling to one million LLM-based agents, designed to mimic real-world social media platforms. The platform successfully replicates and investigates complex social phenomena like information propagation, group polarization, and herd effects, providing a testbed for understanding emergent behaviors at unprecedented scales.
View blog
Autoformalizing Natural Language to First-Order Logic: A Case Study in Logical Fallacy Detection
06 Mar 2025
Stanford University
ETH ZürichTranslating natural language into formal language such as First-Order Logic
(FOL) is a foundational challenge in NLP with wide-ranging applications in
automated reasoning, misinformation tracking, and knowledge validation. In this
paper, we introduce Natural Language to First-Order Logic (NL2FOL), a framework
to autoformalize natural language to FOL step by step using Large Language
Models (LLMs). Our approach addresses key challenges in this translation
process, including the integration of implicit background knowledge. By
leveraging structured representations generated by NL2FOL, we use
Satisfiability Modulo Theory (SMT) solvers to reason about the logical validity
of natural language statements. We present logical fallacy detection as a case
study to evaluate the efficacy of NL2FOL. Being neurosymbolic, our approach
also provides interpretable insights into the reasoning process and
demonstrates robustness without requiring model fine-tuning or labeled training
data. Our framework achieves strong performance on multiple datasets. On the
LOGIC dataset, NL2FOL achieves an F1-score of 78%, while generalizing
effectively to the LOGICCLIMATE dataset with an F1-score of 80%.
Hybrid Training for Vision-Language-Action Models
01 Oct 2025
Using Large Language Models to produce intermediate thoughts, a.k.a. Chain-of-thought (CoT), before providing an answer has been a successful recipe for solving complex language tasks. In robotics, similar embodied CoT strategies, generating thoughts before actions, have also been shown to lead to improved performance when using Vision-Language-Action models (VLAs). As these techniques increase the length of the model's generated outputs to include the thoughts, the inference time is negatively affected. Delaying an agent's actions in real-world executions, as in robotic manipulation settings, strongly affects the usability of a method, as tasks require long sequences of actions. However, is the generation of long chains-of-thought a strong prerequisite for achieving performance improvements? In this work, we explore the idea of Hybrid Training (HyT), a framework that enables VLAs to learn from thoughts and benefit from the associated performance gains, while enabling the possibility to leave out CoT generation during inference. Furthermore, by learning to conditionally predict a diverse set of outputs, HyT supports flexibility at inference time, enabling the model to either predict actions directly, generate thoughts or follow instructions. We evaluate the proposed method in a series of simulated benchmarks and real-world experiments.
Direct neutrino-mass measurement based on 259 days of KATRIN data
University of Washington
Carnegie Mellon UniversityThe KATRIN experiment delivers the most precise direct measurement of the effective electron antineutrino mass, establishing an upper limit of 0.45 eV at 90% confidence level based on 259 days of data. This result nearly doubles the precision of KATRIN's previous bound and provides a world-leading direct constraint on neutrino mass.
View blog
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
01 Jun 2017
University of CambridgeOff-policy model-free deep reinforcement learning methods using previously
collected data can improve sample efficiency over on-policy policy gradient
techniques. On the other hand, on-policy algorithms are often more stable and
easier to use. This paper examines, both theoretically and empirically,
approaches to merging on- and off-policy updates for deep reinforcement
learning. Theoretical results show that off-policy updates with a value
function estimator can be interpolated with on-policy policy gradient updates
whilst still satisfying performance bounds. Our analysis uses control variate
methods to produce a family of policy gradient algorithms, with several
recently proposed algorithms being special cases of this family. We then
provide an empirical comparison of these techniques with the remaining
algorithmic details fixed, and show how different mixing of off-policy gradient
estimates with on-policy samples contribute to improvements in empirical
performance. The final algorithm provides a generalization and unification of
existing deep policy gradient techniques, has theoretical guarantees on the
bias introduced by off-policy updates, and improves on the state-of-the-art
model-free deep RL methods on a number of OpenAI Gym continuous control
benchmarks.
IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detector
University of Michigan
The University of Hong KongLarge Language Models (LLMs) have attained human-level fluency in text generation, which complicates the distinguishing between human-written and LLM-generated texts. This increases the risk of misuse and highlights the need for reliable detectors. Yet, existing detectors exhibit poor robustness on out-of-distribution (OOD) data and attacked data, which is critical for real-world scenarios. Also, they struggle to provide interpretable evidence to support their decisions, thus undermining the reliability. In light of these challenges, we propose IPAD (Inverse Prompt for AI Detection), a novel framework consisting of a Prompt Inverter that identifies predicted prompts that could have generated the input text, and two Distinguishers that examine the probability that the input texts align with the predicted prompts. Empirical evaluations demonstrate that IPAD outperforms the strongest baselines by 9.05% (Average Recall) on in-distribution data, 12.93% (AUROC) on out-of-distribution data, and 5.48% (AUROC) on attacked data. IPAD also performs robustly on structured datasets. Furthermore, an interpretability assessment is conducted to illustrate that IPAD enhances the AI detection trustworthiness by allowing users to directly examine the decision-making evidence, which provides interpretable support for its state-of-the-art detection results.
Solving and visualizing fractional quantum Hall wavefunctions with neural network
22 May 2025
We introduce an attention-based fermionic neural network (FNN) to
variationally solve the problem of two-dimensional Coulomb electron gas in
magnetic fields, a canonical platform for fractional quantum Hall (FQH)
liquids, Wigner crystals and other unconventional electron states. Working
directly with the full Hilbert space of electrons confined to a disk, our
FNN consistently attains energies lower than LL-projected exact diagonalization
(ED) and learns the ground state wavefunction to high accuracy. In low LL
mixing regime, our FNN reveals microscopic features in the short-distance
behavior of FQH wavefunction beyond the Laughlin ansatz. For moderate and
strong LL mixing parameters, the FNN outperforms ED significantly. Moreover, a
phase transition from FQH liquid to a crystal state is found at strong LL
mixing. Our study demonstrates unprecedented power and universality of FNN
based variational method for solving strong-coupling many-body problems with
topological order and electron fractionalization.
Metric Reconstruction in Kerr Spacetime
28 May 2024
Metric reconstruction is the general problem of parameterizing GR in terms of its two ``true degrees of freedom'', e.g., by a complex scalar ``potential'' -- in practice mostly with the aim of simplifying the Einstein equation (EE) within perturbative approaches. In this paper, we re-analyze the metric reconstruction procedure by Green, Hollands, and Zimmerman (GHZ) [Class. Quant. Grav. \textbf{37}, 075001 (2020)], which is a generalization of the Chrzanowski-Cohen-Kegeles (CCK) approach. Contrary to the CCK method, that by GHZ is applicable not only to the vacuum, but also to the sourced linearized Einstein equation (EE). Our main innovation is a version of the GHZ method giving an efficient integration scheme for the initial value problem of the sourced linear EE. By iteration, our scheme gives the metric to as high an order in perturbation theory around Kerr as one might wish, in principle. At each order, the metric perturbation is a sum of a corrector, obtained by solving a triangular system of transport equations, a reconstructed piece, obtained from a Hertz potential as in the CCK approach, and an algebraically special perturbation, determined by the ADM quantities. As a by-product, we determine the precise relations between the asymptotic tail of the Hertz potential in the GHZ and CCK schemes, and the quantities relevant for gravitational radiation, namely, the energy flux, news- and memory tensors, and their associated BMS-supertranslations. We also discuss ways of transforming the metric perturbation to Lorenz gauge.
Virtual Sparse Convolution for Multimodal 3D Object Detection
04 Mar 2023
Texas A&M UniversityRecently, virtual/pseudo-point-based 3D object detection that seamlessly fuses RGB images and LiDAR data by depth completion has gained great attention. However, virtual points generated from an image are very dense, introducing a huge amount of redundant computation during detection. Meanwhile, noises brought by inaccurate depth completion significantly degrade detection precision. This paper proposes a fast yet effective backbone, termed VirConvNet, based on a new operator VirConv (Virtual Sparse Convolution), for virtual-point-based 3D object detection. VirConv consists of two key designs: (1) StVD (Stochastic Voxel Discard) and (2) NRConv (Noise-Resistant Submanifold Convolution). StVD alleviates the computation problem by discarding large amounts of nearby redundant voxels. NRConv tackles the noise problem by encoding voxel features in both 2D image and 3D LiDAR space. By integrating VirConv, we first develop an efficient pipeline VirConv-L based on an early fusion design. Then, we build a high-precision pipeline VirConv-T based on a transformed refinement scheme. Finally, we develop a semi-supervised pipeline VirConv-S based on a pseudo-label framework. On the KITTI car 3D detection test leaderboard, our VirConv-L achieves 85% AP with a fast running speed of 56ms. Our VirConv-T and VirConv-S attains a high-precision of 86.3% and 87.2% AP, and currently rank 2nd and 1st, respectively. The code is available at this https URL.
Logical Fallacy Detection
12 Dec 2022
Researchers from Max Planck Institute and ETH Zürich introduce the task of logical fallacy detection in natural language, constructing two novel datasets and proposing a structure-aware classification model. Their model achieved a 5.46% F1 score improvement over the best baseline on the LOGIC dataset and maintained a 5.66% F1 improvement on the challenging LOGICCLIMATE dataset for climate change claims, indicating enhanced logical reasoning.
View blog
SignX: The Foundation Model for Sign Recognition
22 Apr 2025
Boston University
Columbia UniversitySignX introduces a foundation model for sign language recognition that directly processes raw video and heterogeneous pose data, translating sign language into text. The model integrates diverse pose inputs from multiple estimation models into a unified representation, achieving high BLEU-1 scores and lower Word Error Rate on the ASLLRP SignStream®3 Corpus.
View blog
Coordination Among Neural Modules Through a Shared Global Workspace
22 Mar 2022
Deep learning has seen a movement away from representing examples with a monolithic hidden state towards a richly structured state. For example, Transformers segment by position, and object-centric architectures decompose images into entities. In all these architectures, interactions between different elements are modeled via pairwise interactions: Transformers make use of self-attention to incorporate information from other positions; object-centric architectures make use of graph neural networks to model interactions among entities. However, pairwise interactions may not achieve global coordination or a coherent, integrated representation that can be used for downstream tasks. In cognitive science, a global workspace architecture has been proposed in which functionally specialized components share information through a common, bandwidth-limited communication channel. We explore the use of such a communication channel in the context of deep learning for modeling the structure of complex environments. The proposed method includes a shared workspace through which communication among different specialist modules takes place but due to limits on the communication bandwidth, specialist modules must compete for access. We show that capacity limitations have a rational basis in that (1) they encourage specialization and compositionality and (2) they facilitate the synchronization of otherwise independent specialists.
Emergent Dominance Hierarchies in Reinforcement Learning Agents
Tel Aviv University
University of California, IrvineModern Reinforcement Learning (RL) algorithms are able to outperform humans in a wide variety of tasks. Multi-agent reinforcement learning (MARL) settings present additional challenges, and successful cooperation in mixed-motive groups of agents depends on a delicate balancing act between individual and group objectives. Social conventions and norms, often inspired by human institutions, are used as tools for striking this balance.
In this paper, we examine a fundamental, well-studied social convention that underlies cooperation in both animal and human societies: dominance hierarchies.
We adapt the ethological theory of dominance hierarchies to artificial agents, borrowing the established terminology and definitions with as few amendments as possible. We demonstrate that populations of RL agents, operating without explicit programming or intrinsic rewards, can invent, learn, enforce, and transmit a dominance hierarchy to new populations. The dominance hierarchies that emerge have a similar structure to those studied in chickens, mice, fish, and other species.
SE(3) Equivariant Augmented Coupling Flows
Coupling normalizing flows allow for fast sampling and density evaluation,
making them the tool of choice for probabilistic modeling of physical systems.
However, the standard coupling architecture precludes endowing flows that
operate on the Cartesian coordinates of atoms with the SE(3) and permutation
invariances of physical systems. This work proposes a coupling flow that
preserves SE(3) and permutation equivariance by performing coordinate splits
along additional augmented dimensions. At each layer, the flow maps atoms'
positions into learned SE(3) invariant bases, where we apply standard flow
transformations, such as monotonic rational-quadratic splines, before returning
to the original basis. Crucially, our flow preserves fast sampling and density
evaluation, and may be used to produce unbiased estimates of expectations with
respect to the target distribution via importance sampling. When trained on the
DW4, LJ13, and QM9-positional datasets, our flow is competitive with
equivariant continuous normalizing flows and diffusion models, while allowing
sampling more than an order of magnitude faster. Moreover, to the best of our
knowledge, we are the first to learn the full Boltzmann distribution of alanine
dipeptide by only modeling the Cartesian positions of its atoms. Lastly, we
demonstrate that our flow can be trained to approximately sample from the
Boltzmann distribution of the DW4 and LJ13 particle systems using only their
energy functions.
When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
27 Oct 2022
Harvard UniversityAI systems are becoming increasingly intertwined with human life. In order to effectively collaborate with humans and ensure safety, AI systems need to be able to understand, interpret and predict human moral judgments and decisions. Human moral judgments are often guided by rules, but not always. A central challenge for AI safety is capturing the flexibility of the human moral mind -- the ability to determine when a rule should be broken, especially in novel or unusual situations. In this paper, we present a novel challenge set consisting of rule-breaking question answering (RBQA) of cases that involve potentially permissible rule-breaking -- inspired by recent moral psychology studies. Using a state-of-the-art large language model (LLM) as a basis, we propose a novel moral chain of thought (MORALCOT) prompting strategy that combines the strengths of LLMs with theories of moral reasoning developed in cognitive science to predict human moral judgments. MORALCOT outperforms seven existing LLMs by 6.2% F1, suggesting that modeling human reasoning might be necessary to capture the flexibility of the human moral mind. We also conduct a detailed error analysis to suggest directions for future work to improve AI safety using RBQA. Our data is open-sourced at this https URL and code at this https URL
Temporal Difference Models: Model-Free Deep RL for Model-Based Control
24 Feb 2020
UC BerkeleyTemporal Difference Models (TDMs) bridge model-free and model-based deep reinforcement learning by introducing a family of goal-conditioned value functions with a variable planning horizon. This approach achieves the sample efficiency of model-based methods while retaining the high asymptotic performance of model-free algorithms across various continuous control tasks, including real-world robotic manipulation.
View blog
Unifying distillation and privileged information
26 Feb 2016
Distillation (Hinton et al., 2015) and privileged information (Vapnik &
Izmailov, 2015) are two techniques that enable machines to learn from other
machines. This paper unifies these two techniques into generalized
distillation, a framework to learn from multiple machines and data
representations. We provide theoretical and causal insight about the inner
workings of generalized distillation, extend it to unsupervised, semisupervised
and multitask learning scenarios, and illustrate its efficacy on a variety of
numerical simulations on both synthetic and real-world data.
Fun with
14 Jun 2008
We show that the algebra and the endomotive of the quantum statistical
mechanical system of Bost--Connes naturally arises by extension of scalars from
the "field with one element" to rational numbers. The inductive structure of
the abelian part of the endomotive corresponds to the tower of finite
extensions of that "field", while the endomorphisms reflect the Frobenius
correspondences. This gives in particular an explicit model over the integers
for this endomotive, which is related to the original Hecke algebra
description. We study the reduction at a prime of the endomotive and of the
corresponding noncommutative crossed product algebra.
Origin of the insulating and superconducting phases in molecular solid -(BEDT-TTF)Cu(CN)
24 Dec 2024
Recent studies of organic molecular solids are highlighted by their complex phase diagram and light-induced phenomena, such as Mott insulator, spin liquid phase, and superconductivity. However, a discrepancy between experimental observation and first-principle calculation on the -(BEDT-TTF)X family inhibits understanding their properties. Here, we revisit the electronic structure of -(BEDT-TTF)Cu(CN) with the recently developed DFT+GOU method to correct the energy level of molecular orbital states in the molecular solid. Our work reveals that the insulating electronic structure of -(BEDT-TTF)Cu(CN) originates from the energy gap between the highest occupied and the lowest unoccupied molecular orbital states of the BEDT-TTF dimers, that are the periodic unit of the molecular solid. We verify that our calculation result provides consistent band gap, optical conductivity, and evolution of the metal-insulator transition as a function of pressure with experimental observations. Especially, the superconducting dome of -(BEDT-TTF)Cu(CN), which originates from the flat band state at the Fermi level, is reproduced. Additionally, we constructed a new low-energy lattice model based on the ability of electronic structure data that can be used to address many-body physics, such as quantum spin liquid and double-holon dynamics. Our provides a deeper understanding of the complex phase diagram and various light-induced phenomena in the -(BEDT-TTF)X family and the other complex organic molecular solids.
There are no more papers matching your filters at the moment.