Ask or search anything...
TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Tracking
08 Oct 2025
Beijing Normal UniversityTrackVLA++ advances embodied visual tracking by integrating an efficient spatial reasoning mechanism and a robust, confidence-gated long-term memory into Vision-Language-Action models. It achieves state-of-the-art performance on multiple simulation benchmarks and demonstrates improved real-world tracking robustness against occlusions and distractors.
View blog
Quantification of Large Language Model Distillation
17 Feb 2025
Chinese Academy of Sciences
Peking UniversityModel distillation is a fundamental technique in building large language
models (LLMs), transferring knowledge from a teacher model to a student model.
However, distillation can lead to model homogenization, reducing diversity
among models and impairing their ability to robustly handle complex or novel
tasks. These limitations underscore the need to systematically quantify the
distillation process and its impact. In this work, we propose a framework to
evaluate and quantify model distillation. Our method addresses two key aspects:
(1) Identifying identity cognition contradictions to assess discrepancies in
how models perceive and represent identity-related information, and (2)
Analyzing multi-granularity response similarities across models to measure the
extent of homogenization. Experimental results demonstrate two key insights:
(1) Well-known closed-source and open-source LLMs usually exhibit high
distillation degrees, except for Claude, Doubao, and Gemini. (2) Base LLMs show
higher distillation degrees compared to aligned LLMs. By offering a systematic
approach to improve the transparency of LLM data distillation, we call for LLMs
with more independent development and more transparent technical reports to
improve LLMs' robustness and safety. The code and data are available under
this https URL
h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective
University of OxfordDeep neural networks have demonstrated remarkable performance across numerous learning tasks but often suffer from miscalibration, resulting in unreliable probability outputs. This has inspired many recent works on mitigating miscalibration, particularly through post-hoc recalibration methods that aim to obtain calibrated probabilities without sacrificing the classification performance of pre-trained models. In this study, we summarize and categorize previous works into three general strategies: intuitively designed methods, binning-based methods, and methods based on formulations of ideal calibration. Through theoretical and practical analysis, we highlight ten common limitations in previous approaches. To address these limitations, we propose a probabilistic learning framework for calibration called h-calibration, which theoretically constructs an equivalent learning formulation for canonical calibration with boundedness. On this basis, we design a simple yet effective post-hoc calibration algorithm. Our method not only overcomes the ten identified limitations but also achieves markedly better performance than traditional methods, as validated by extensive experiments. We further analyze, both theoretically and experimentally, the relationship and advantages of our learning objective compared to traditional proper scoring rule. In summary, our probabilistic framework derives an approximately equivalent differentiable objective for learning error-bounded calibrated probabilities, elucidating the correspondence and convergence properties of computational statistics with respect to theoretical bounds in canonical calibration. The theoretical effectiveness is verified on standard post-hoc calibration benchmarks by achieving state-of-the-art performance. This research offers valuable reference for learning reliable likelihood in related fields.
VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory
ByteDanceVideoSSM, developed by researchers at The University of Hong Kong and PICO, ByteDance, introduces a hybrid state-space memory architecture to enable autoregressive long video generation. The model maintains temporal consistency and dynamism over minute-scale durations, achieving superior quality and preventing motion drift or content repetition while operating with linear computational complexity.
View blog
Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning
25 Feb 2025
HKUSTIn offline reinforcement learning, it is necessary to manage
out-of-distribution actions to prevent overestimation of value functions. One
class of methods, the policy-regularized method, addresses this problem by
constraining the target policy to stay close to the behavior policy. Although
several approaches suggest representing the behavior policy as an expressive
diffusion model to boost performance, it remains unclear how to regularize the
target policy given a diffusion-modeled behavior sampler. In this paper, we
propose Diffusion Actor-Critic (DAC) that formulates the Kullback-Leibler (KL)
constraint policy iteration as a diffusion noise regression problem, enabling
direct representation of target policies as diffusion models. Our approach
follows the actor-critic learning paradigm in which we alternatively train a
diffusion-modeled target policy and a critic network. The actor training loss
includes a soft Q-guidance term from the Q-gradient. The soft Q-guidance is
based on the theoretical solution of the KL constraint policy iteration, which
prevents the learned policy from taking out-of-distribution actions. We
demonstrate that such diffusion-based policy constraint, along with the
coupling of the lower confidence bound of the Q-ensemble as value targets, not
only preserves the multi-modality of target policies, but also contributes to
stable convergence and strong performance in DAC. Our approach is evaluated on
D4RL benchmarks and outperforms the state-of-the-art in nearly all
environments. Code is available at this https URL
Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks
Carnegie Mellon University
the University of TokyoResearchers from Carnegie Mellon University, Microsoft Research Asia, and other institutions investigated how label noise in large-scale pre-training datasets affects downstream task performance. They developed NMTune, a lightweight, black-box fine-tuning method that consistently improves generalization, particularly for out-of-domain tasks, by adaptively reshaping feature representations.
View blog
G-DReaM: Graph-conditioned Diffusion Retargeting across Multiple Embodiments
Motion retargeting for specific robot from existing motion datasets is one
critical step in transferring motion patterns from human behaviors to and
across various robots. However, inconsistencies in topological structure,
geometrical parameters as well as joint correspondence make it difficult to
handle diverse embodiments with a unified retargeting architecture. In this
work, we propose a novel unified graph-conditioned diffusion-based motion
generation framework for retargeting reference motions across diverse
embodiments. The intrinsic characteristics of heterogeneous embodiments are
represented with graph structure that effectively captures topological and
geometrical features of different robots. Such a graph-based encoding further
allows for knowledge exploitation at the joint level with a customized
attention mechanisms developed in this work. For lacking ground truth motions
of the desired embodiment, we utilize an energy-based guidance formulated as
retargeting losses to train the diffusion model. As one of the first
cross-embodiment motion retargeting methods in robotics, our experiments
validate that the proposed model can retarget motions across heterogeneous
embodiments in a unified manner. Moreover, it demonstrates a certain degree of
generalization to both diverse skeletal structures and similar motion patterns.
X-GRM: Large Gaussian Reconstruction Model for Sparse-view X-rays to Computed Tomography
26 May 2025
The Chinese University of Hong Kong
EPFLComputed Tomography serves as an indispensable tool in clinical workflows,
providing non-invasive visualization of internal anatomical structures.
Existing CT reconstruction works are limited to small-capacity model
architecture and inflexible volume representation. In this work, we present
X-GRM (X-ray Gaussian Reconstruction Model), a large feedforward model for
reconstructing 3D CT volumes from sparse-view 2D X-ray projections. X-GRM
employs a scalable transformer-based architecture to encode sparse-view X-ray
inputs, where tokens from different views are integrated efficiently. Then,
these tokens are decoded into a novel volume representation, named Voxel-based
Gaussian Splatting (VoxGS), which enables efficient CT volume extraction and
differentiable X-ray rendering. This combination of a high-capacity model and
flexible volume representation, empowers our model to produce high-quality
reconstructions from various testing inputs, including in-domain and out-domain
X-ray projections. Our codes are available at:
this https URL
Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
Tsinghua UniversityMixture-of-Experts-based (MoE-based) diffusion models demonstrate remarkable scalability in high-fidelity image generation, yet their reliance on expert parallelism introduces critical communication bottlenecks. State-of-the-art methods alleviate such overhead in parallel diffusion inference through computation-communication overlapping, termed displaced parallelism. However, we identify that these techniques induce severe *staleness*-the usage of outdated activations from previous timesteps that significantly degrades quality, especially in expert-parallel scenarios. We tackle this fundamental tension and propose DICE, a staleness-centric optimization framework with a three-fold approach: (1) Interweaved Parallelism introduces staggered pipelines, effectively halving step-level staleness for free; (2) Selective Synchronization operates at layer-level and protects layers vulnerable from staled activations; and (3) Conditional Communication, a token-level, training-free method that dynamically adjusts communication frequency based on token importance. Together, these strategies effectively reduce staleness, achieving 1.26x speedup with minimal quality degradation. Empirical results establish DICE as an effective and scalable solution. Our code is publicly available at this https URL
Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG
27 May 2025
Existing Retrieval-Augmented Generation (RAG) systems often hallucinate by always attempting to answer questions, even when reliable information is unavailable. Sun et al. from the AlphaXiv Institute introduce Divide-Then-Align (DTA), a framework that teaches RAG models when to genuinely abstain by first categorizing queries based on the union of the LLM's parametric knowledge and retrieved information, then training with preference data tailored to these knowledge boundaries. DTA achieves an AF1 of 63.3% on Llama-2-7b, providing more reliable RAG responses that balance helpfulness with honesty.
View blog
What makes a good data augmentation for few-shot unsupervised image anomaly detection?
21 Apr 2023
Data augmentation is a promising technique for unsupervised anomaly detection in industrial applications, where the availability of positive samples is often limited due to factors such as commercial competition and sample collection difficulties. In this paper, how to effectively select and apply data augmentation methods for unsupervised anomaly detection is studied. The impact of various data augmentation methods on different anomaly detection algorithms is systematically investigated through experiments. The experimental results show that the performance of different industrial image anomaly detection (termed as IAD) algorithms is not significantly affected by the specific data augmentation method employed and that combining multiple data augmentation methods does not necessarily yield further improvements in the accuracy of anomaly detection, although it can achieve excellent results on specific methods. These findings provide useful guidance on selecting appropriate data augmentation methods for different requirements in IAD.
Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention
MicrosoftRecent years have witnessed the great potential of attention mechanism in graph representation learning. However, while variants of attention-based GNNs are setting new benchmarks for numerous real-world datasets, recent works have pointed out that their induced attentions are less robust and generalizable against noisy graphs due to lack of direct supervision. In this paper, we present a new framework which utilizes the tool of causality to provide a powerful supervision signal for the learning process of attention functions. Specifically, we estimate the direct causal effect of attention to the final prediction, and then maximize such effect to guide attention attending to more meaningful neighbors. Our method can serve as a plug-and-play module for any canonical attention-based GNNs in an end-to-end fashion. Extensive experiments on a wide range of benchmark datasets illustrated that, by directly supervising attention functions, the model is able to converge faster with a clearer decision boundary, and thus yields better performances.
ArtiWorld: LLM-Driven Articulation of 3D Objects in Scenes
18 Nov 2025
Mingqi Gao
Building interactive simulators and scalable robot-learning environments requires a large number of articulated assets. However, most existing 3D assets in simulation are rigid, and manually converting them into articulated objects is extremely labor- and cost-intensive. This raises a natural question: can we automatically identify articulable objects in a scene and convert them into articulated assets directly? In this paper, we present ArtiWorld, a scene-aware pipeline that localizes candidate articulable objects from textual scene descriptions and reconstructs executable URDF models that preserve the original geometry. At the core of this pipeline is Arti4URDF, which leverages 3D point cloud, prior knowledge of a large language model (LLM), and a URDF-oriented prompt design to rapidly convert rigid objects into interactive URDF-based articulated objects while maintaining their 3D shape. We evaluate ArtiWorld at three levels: 3D simulated objects, full 3D simulated scenes, and real-world scan scenes. Across all three settings, our method consistently outperforms existing approaches and achieves state-of-the-art performance, while preserving object geometry and correctly capturing object interactivity to produce usable URDF-based articulated models. This provides a practical path toward building interactive, robot-ready simulation environments directly from existing 3D assets. Code and data will be released.
Gaussian Approximation and Multiplier Bootstrap for Polyak-Ruppert Averaged Linear Stochastic Approximation with Applications to TD Learning
02 Feb 2025
In this paper, we obtain the Berry-Esseen bound for multivariate normal
approximation for the Polyak-Ruppert averaged iterates of the linear stochastic
approximation (LSA) algorithm with decreasing step size. Moreover, we prove the
non-asymptotic validity of the confidence intervals for parameter estimation
with LSA based on multiplier bootstrap. This procedure updates the LSA estimate
together with a set of randomly perturbed LSA estimates upon the arrival of
subsequent observations. We illustrate our findings in the setting of temporal
difference learning with linear function approximation.
Mastering Uncertainty: From Understanding to Prediction
18 Oct 2025
ETH ZurichDidier Sornette's work redefines uncertainty as largely remediable ignorance rather than intrinsic randomness, advocating for a shift from forecasting exact outcomes to diagnosing instability by identifying early-warning signals in complex systems. His research demonstrates that extreme events, often called "dragon-kings," can exhibit precursory patterns due to self-amplifying mechanisms.
View blog
Hysteresis Measurements as a Diagnostic Tool: A Systematic Approach for Stability Benchmarking and Performance Projection of 2D-Materials-Based MOSFETs
25 Sep 2025
Judging by its omnipresence in the literature, the hysteresis observed in the transfer characteristics of emerging transistors based on 2D-materials is widely accepted as an important metric related to the device quality. The hysteresis is often reported with attributes like "negligible" or "small" without giving any specifics as to how this was determined and against what reference the measured values were compared to. Quite surprisingly, there appears to be only a fragmentary understanding of the mechanisms actually contributing to hysteresis and the sensitivity of the actual measurement on various experimental parameters. We attempt to close this gap by first providing a comprehensive theoretical analysis of the dominant mechanisms contributing to hysteresis: charge trapping by defects from the channel or the gate, the drift of mobile charges, and eventually ferroelectricity. We continue by suggesting methods to experimentally distinguishing between these phenomena. Based on these discussions it becomes clear that previously reported hysteresis values have little meaning as they have been non-systematically recorded under arbitrary conditions. In order to resolve this predicament, we propose a standardized hysteresis measurement scheme to establish the hysteresis as a comparable metric for the assessment of device stability. Our standardized scheme ensures that hysteresis data can be effectively compared across different technologies and, most importantly, provide a means to extrapolate data obtained on thicker prototypes to subnanometer equivalent oxide thicknesses. This facilitates the systematic benchmarking of insulator/channel combinations in terms of stability, which thereby enables the screening of material systems for more stable and reliable 2D-material-based MOSFETs.
Asterinas: A Linux ABI-Compatible, Rust-Based Framekernel OS with a Small and Sound TCB
04 Jun 2025
How can one build a feature-rich, general-purpose, Rust-based operating
system (OS) with a minimal and sound Trusted Computing Base (TCB) for memory
safety? Existing Rust-based OSes fall short due to their improper use of unsafe
Rust in kernel development. To address this challenge, we propose a novel OS
architecture called framekernel that realizes Rust's full potential to achieve
intra-kernel privilege separation, ensuring TCB minimality and soundness. We
present OSTD, a streamlined framework for safe Rust OS development, and
Asterinas, a Linux ABI-compatible framekernel OS implemented entirely in safe
Rust using OSTD. Supporting over 210 Linux system calls, Asterinas delivers
performance on par with Linux, while maintaining a minimized, memory-safety TCB
of only about 14.0% of the codebase. These results underscore the practicality
and benefits of the framekernel architecture in building safe and efficient
OSes.
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents
20 Jul 2025
Stanford UniversityThe development of autonomous agents increasingly relies on Multimodal Language Models (MLMs) to perform tasks described in natural language with GUI environments, such as websites, desktop computers, or mobile phones. Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexities of constructing tasks and evaluators. To overcome these limitations, we introduce Crab, the first agent benchmark framework designed to support cross-environment tasks, incorporating a graph-based fine-grained evaluation method and an efficient mechanism for task and evaluator construction. Our framework supports multiple devices and can be easily extended to any environment with a Python interface. Leveraging Crab, we developed a cross-platform Crab Benchmark-v0 comprising 120 tasks in computer desktop and mobile phone environments. We evaluated four advanced MLMs using different single and multi-agent system configurations on this benchmark. The experimental results demonstrate that the single agent with GPT-4o achieves the best completion ratio of 38.01%. All framework code, agent code, and task datasets are publicly available at this https URL.
Commonsense Scene Graph-based Target Localization for Object Search
27 Sep 2024
Object search is a fundamental skill for household robots, yet the core problem lies in the robot's ability to locate the target object accurately. The dynamic nature of household environments, characterized by the arbitrary placement of daily objects by users, makes it challenging to perform target localization. To efficiently locate the target object, the robot needs to be equipped with knowledge at both the object and room level. However, existing approaches rely solely on one type of knowledge, leading to unsatisfactory object localization performance and, consequently, inefficient object search processes. To address this problem, we propose a commonsense scene graph-based target localization, CSG-TL, to enhance target object search in the household environment. Given the pre-built map with stationary items, the robot models the room-level knowledge with object-level commonsense knowledge generated by a large language model (LLM) to a commonsense scene graph (CSG), supporting both types of knowledge for CSG-TL. To demonstrate the superiority of CSG-TL on target localization, extensive experiments are performed on the real-world ScanNet dataset and the AI2THOR simulator. Moreover, we have extended CSG-TL to an object search framework, CSG-OS, validated in both simulated and real-world environments. Code and videos are available at this https URL.
RoboReflect: A Robotic Reflective Reasoning Framework for Grasping Ambiguous-Condition Objects
10 Mar 2025
University of WarwickAs robotic technology rapidly develops, robots are being employed in an
increasing number of fields. However, due to the complexity of deployment
environments or the prevalence of ambiguous-condition objects, the practical
application of robotics still faces many challenges, leading to frequent
errors. Traditional methods and some LLM-based approaches, although improved,
still require substantial human intervention and struggle with autonomous error
correction in complex scenarios. In this work, we propose RoboReflect, a novel
framework leveraging large vision-language models (LVLMs) to enable
self-reflection and autonomous error correction in robotic grasping tasks.
RoboReflect allows robots to automatically adjust their strategies based on
unsuccessful attempts until successful execution is achieved. The corrected
strategies are saved in the memory for future task reference. We evaluate
RoboReflect through extensive testing on eight common objects prone to
ambiguous conditions of three categories. Our results demonstrate that
RoboReflect not only outperforms existing grasp pose estimation methods like
AnyGrasp and high-level action planning techniques ReKep with GPT-4V but also
significantly enhances the robot's capability to adapt and correct errors
independently. These findings underscore the critical importance of autonomous
self-reflection in robotic systems while effectively addressing the challenges
posed by ambiguous-condition environments.
There are no more papers matching your filters at the moment.