Ask or search anything...
UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
21 Nov 2025
UniModel introduces a visual-only framework that unifies multimodal understanding and generation by representing both text and images as pixel-level data within a single diffusion transformer. This approach enables coherent text-to-image generation and image captioning, demonstrating strong cycle consistency and emergent controllability by operating entirely in a shared visual latent space.
View blog
Dome-DETR: DETR with Density-Oriented Feature-Query Manipulation for Efficient Tiny Object Detection
University of Science and Technology of ChinaTiny object detection plays a vital role in drone surveillance, remote sensing, and autonomous systems, enabling the identification of small targets across vast landscapes. However, existing methods suffer from inefficient feature leverage and high computational costs due to redundant feature processing and rigid query allocation. To address these challenges, we propose Dome-DETR, a novel framework with Density-Oriented Feature-Query Manipulation for Efficient Tiny Object Detection. To reduce feature redundancies, we introduce a lightweight Density-Focal Extractor (DeFE) to produce clustered compact foreground masks. Leveraging these masks, we incorporate Masked Window Attention Sparsification (MWAS) to focus computational resources on the most informative regions via sparse attention. Besides, we propose Progressive Adaptive Query Initialization (PAQI), which adaptively modulates query density across spatial areas for better query allocation. Extensive experiments demonstrate that Dome-DETR achieves state-of-the-art performance (+3.3 AP on AI-TOD-V2 and +2.5 AP on VisDrone) while maintaining low computational complexity and a compact model size. Code is available at this https URL.
Emotional Support with LLM-based Empathetic Dialogue Generation
17 Jul 2025
Researchers from TeleAI, China Telecom Corp Ltd, developed an effective solution for Emotional Support Conversation (ESC) by fine-tuning Qwen2.5 Large Language Models with advanced prompt engineering. Their approach, which includes both LoRA and full-parameter fine-tuning, achieved a second-place ranking in the NLPCC 2025 Task 8 evaluation, with their best model yielding a total score of 39.62 and a G-score of 87.20.
View blog
TableZoomer: A Collaborative Agent Framework for Large-scale Table Question Answering
National University of Singapore
Beihang UniversityDeveloped by TeleAI, TableZoomer is an LLM-powered agent framework designed for large-scale Table Question Answering, efficiently addressing the challenges of massive and heterogeneous tables. It achieved absolute accuracy improvements of 19.34% on DataBench by employing schema-based representation and query-aware zooming, significantly reducing input tokens while maintaining high performance.
View blog
MLM: Learning Multi-task Loco-Manipulation Whole-Body Control for Quadruped Robot with Arm
Shanghai Jiao Tong UniversityWhole-body loco-manipulation for quadruped robots with arms remains a challenging problem, particularly in achieving multi-task control. To address this, we propose MLM, a reinforcement learning framework driven by both real-world and simulation data. It enables a six-DoF robotic arm-equipped quadruped robot to perform whole-body loco-manipulation for multiple tasks autonomously or under human teleoperation. To address the problem of balancing multiple tasks during the learning of loco-manipulation, we introduce a trajectory library with an adaptive, curriculum-based sampling mechanism. This approach allows the policy to efficiently leverage real-world collected trajectories for learning multi-task loco-manipulation. To address deployment scenarios with only historical observations and to enhance the performance of policy execution across tasks with different spatial ranges, we propose a Trajectory-Velocity Prediction policy network. It predicts unobservable future trajectories and velocities. By leveraging extensive simulation data and curriculum-based rewards, our controller achieves whole-body behaviors in simulation and zero-shot transfer to real-world deployment. Ablation studies in simulation verify the necessity and effectiveness of our approach, while real-world experiments on a Go2 robot with an Airbot robotic arm demonstrate the policy's good performance in multi-task execution.
VLMPlanner: Integrating Visual Language Models with Motion Planning
Researchers at the University of Science and Technology of China developed VLMPlanner, a hybrid framework integrating visual language models with real-time motion planning for autonomous driving. It processes raw multi-view images and employs a Context-Adaptive Inference Gate for computational efficiency, achieving state-of-the-art performance on the nuPlan benchmark and reducing collision probability.
View blog
TableReasoner: Advancing Table Reasoning Framework with Large Language Models
10 Jul 2025
Wenhan Chang
Researchers from the Institute of Artificial Intelligence (TeleAI), China Telecom Corp Ltd, developed TableReasoner, a framework for Table Question Answering that processes large and complex tables by generating a concise JSON schema. The system effectively mitigates numerical hallucinations through program-assisted reasoning and achieved first place in both subtasks of SemEval-2025 Task 8, demonstrating robust scalability across table sizes.
View blog
Depth Helps: Improving Pre-trained RGB-based Policy with Depth Information Injection
09 Aug 2024
Researchers from Shanghai Artificial Intelligence Laboratory and Renmin University of China introduce the Depth Information Injection (DI²) framework, which enhances pre-trained RGB-based robotic manipulation policies with 3D perception by integrating predicted depth features. The method achieves an average 63.15% success rate in simulated tasks and 66.67% in real-world scenarios, maintaining robust performance when deployed with RGB-only input.
View blog
Personalized Quantum Federated Learning for Privacy Image Classification
03 Oct 2024
Quantum federated learning has brought about the improvement of privacy image classification, while the lack of personality of the client model may contribute to the suboptimal of quantum federated learning. A personalized quantum federated learning algorithm for privacy image classification is proposed to enhance the personality of the client model in the case of an imbalanced distribution of images. First, a personalized quantum federated learning model is constructed, in which a personalized layer is set for the client model to maintain the personalized parameters. Second, a personalized quantum federated learning algorithm is introduced to secure the information exchanged between the client and this http URL, the personalized federated learning is applied to image classification on the FashionMNIST dataset, and the experimental results indicate that the personalized quantum federated learning algorithm can obtain global and local models with excellent performance, even in situations where local training samples are imbalanced. The server's accuracy is 100% with 8 clients and a distribution parameter of 100, outperforming the non-personalized model by 7%. The average client accuracy is 2.9% higher than that of the non-personalized model with 2 clients and a distribution parameter of 1. Compared to previous quantum federated learning algorithms, the proposed personalized quantum federated learning algorithm eliminates the need for additional local training while safeguarding both model and data this http URL may facilitate broader adoption and application of quantum technologies, and pave the way for more secure, scalable, and efficient quantum distribute machine learning solutions.
Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective
17 May 2025
Parameter-efficient fine-tuning for continual learning (PEFT-CL) has shown promise in adapting pre-trained models to sequential tasks while mitigating catastrophic forgetting problem. However, understanding the mechanisms that dictate continual performance in this paradigm remains elusive. To unravel this mystery, we undertake a rigorous analysis of PEFT-CL dynamics to derive relevant metrics for continual scenarios using Neural Tangent Kernel (NTK) theory. With the aid of NTK as a mathematical analysis tool, we recast the challenge of test-time forgetting into the quantifiable generalization gaps during training, identifying three key factors that influence these gaps and the performance of PEFT-CL: training sample size, task-level feature orthogonality, and regularization. To address these challenges, we introduce NTK-CL, a novel framework that eliminates task-specific parameter storage while adaptively generating task-relevant features. Aligning with theoretical guidance, NTK-CL triples the feature representation of each sample, theoretically and empirically reducing the magnitude of both task-interplay and task-specific generalization gaps. Grounded in NTK analysis, our framework imposes an adaptive exponential moving average mechanism and constraints on task-level feature orthogonality, maintaining intra-task NTK forms while attenuating inter-task NTK forms. Ultimately, by fine-tuning optimizable parameters with appropriate regularization, NTK-CL achieves state-of-the-art performance on established PEFT-CL benchmarks. This work provides a theoretical foundation for understanding and improving PEFT-CL models, offering insights into the interplay between feature representation, task orthogonality, and generalization, contributing to the development of more efficient continual learning systems.
Dual-Bounded Nonlinear Optimal Transport for Size Constrained Min Cut Clustering
30 Jan 2025
Min cut is an important graph partitioning method. However, current solutions to the min cut problem suffer from slow speeds, difficulty in solving, and often converge to simple solutions. To address these issues, we relax the min cut problem into a dual-bounded constraint and, for the first time, treat the min cut problem as a dual-bounded nonlinear optimal transport problem. Additionally, we develop a method for solving dual-bounded nonlinear optimal transport based on the Frank-Wolfe method (abbreviated as DNF). Notably, DNF not only solves the size constrained min cut problem but is also applicable to all dual-bounded nonlinear optimal transport problems. We prove that for convex problems satisfying Lipschitz smoothness, the DNF method can achieve a convergence rate of . We apply the DNF method to the min cut problem and find that it achieves state-of-the-art performance in terms of both the loss function and clustering accuracy at the fastest speed, with a convergence rate of . Moreover, the DNF method for the size constrained min cut problem requires no parameters and exhibits better stability.
SSCM: A Spatial-Semantic Consistent Model for Multi-Contrast MRI Super-Resolution
Multi-contrast Magnetic Resonance Imaging super-resolution (MC-MRI SR) aims to enhance low-resolution (LR) contrasts leveraging high-resolution (HR) references, shortening acquisition time and improving imaging efficiency while preserving anatomical details. The main challenge lies in maintaining spatial-semantic consistency, ensuring anatomical structures remain well-aligned and coherent despite structural discrepancies and motion between the target and reference images. Conventional methods insufficiently model spatial-semantic consistency and underuse frequency-domain information, which leads to poor fine-grained alignment and inadequate recovery of high-frequency details. In this paper, we propose the Spatial-Semantic Consistent Model (SSCM), which integrates a Dynamic Spatial Warping Module for inter-contrast spatial alignment, a Semantic-Aware Token Aggregation Block for long-range semantic consistency, and a Spatial-Frequency Fusion Block for fine structure restoration. Experiments on public and private datasets show that SSCM achieves state-of-the-art performance with fewer parameters while ensuring spatially and semantically consistent reconstructions.
Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues
Chinese Academy of Sciences
Renmin University of ChinaRecently, fake news with text and images have achieved more effective diffusion than text-only fake news, raising a severe issue of multimodal fake news detection. Current studies on this issue have made significant contributions to developing multimodal models, but they are defective in modeling the multimodal content sufficiently. Most of them only preliminarily model the basic semantics of the images as a supplement to the text, which limits their performance on detection. In this paper, we find three valuable text-image correlations in multimodal fake news: entity inconsistency, mutual enhancement, and text complementation. To effectively capture these multimodal clues, we innovatively extract visual entities (such as celebrities and landmarks) to understand the news-related high-level semantics of images, and then model the multimodal entity inconsistency and mutual enhancement with the help of visual entities. Moreover, we extract the embedded text in images as the complementation of the original text. All things considered, we propose a novel entity-enhanced multimodal fusion framework, which simultaneously models three cross-modal correlations to detect diverse multimodal fake news. Extensive experiments demonstrate the superiority of our model compared to the state of the art.
DS-TTS: Zero-Shot Speaker Style Adaptation from Voice Clips via Dynamic Dual-Style Feature Modulation
Communication University of ChinaRecent advancements in text-to-speech (TTS) technology have increased demand
for personalized audio synthesis. Zero-shot voice cloning, a specialized TTS
task, aims to synthesize a target speaker's voice using only a single audio
sample and arbitrary text, without prior exposure to the speaker during
training. This process employs pattern recognition techniques to analyze and
replicate the speaker's unique vocal features. Despite progress, challenges
remain in adapting to the vocal style of unseen speakers, highlighting
difficulties in generalizing TTS systems to handle diverse voices while
maintaining naturalness, expressiveness, and speaker fidelity. To address the
challenges of unseen speaker style adaptation, we propose DS-TTS, a novel
approach aimed at enhancing the synthesis of diverse, previously unheard
voices. Central to our method is a Dual-Style Encoding Network (DuSEN), where
two distinct style encoders capture complementary aspects of a speaker's vocal
identity. These speaker-specific style vectors are seamlessly integrated into
the Dynamic Generator Network (DyGN) via a Style Gating-Film (SGF) mechanism,
enabling more accurate and expressive reproduction of unseen speakers' unique
vocal characteristics. In addition, we introduce a Dynamic Generator Network to
tackle synthesis issues that arise with varying sentence lengths. By
dynamically adapting to the length of the input, this component ensures robust
performance across diverse text inputs and speaker styles, significantly
improving the model's ability to generalize to unseen speakers in a more
natural and expressive manner. Experimental evaluations on the VCTK dataset
suggest that DS-TTS demonstrates superior overall performance in voice cloning
tasks compared to existing state-of-the-art models, showing notable
improvements in both word error rate and speaker similarity.
Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer
The increased integration of renewable energy poses a slew of technical challenges for the operation of power distribution networks. Among them, voltage fluctuations caused by the instability of renewable energy are receiving increasing attention. Utilizing MARL algorithms to coordinate multiple control units in the grid, which is able to handle rapid changes of power systems, has been widely studied in active voltage control task recently. However, existing approaches based on MARL ignore the unique nature of the grid and achieve limited performance. In this paper, we introduce the transformer architecture to extract representations adapting to power network problems and propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency and facilitating the representation learning of the transformer-based model. We couple T-MAAC with different multi-agent actor-critic algorithms, and the consistent improvements on the active voltage control task demonstrate the effectiveness of the proposed method.
Bridging the Gap between Continuous and Informative Discrete Representations by Random Product Quantization
08 Nov 2025
Self-supervised learning (SSL) has become a core technique in speech processing, but the high dimensionality of its representations makes discretization essential for improving efficiency. However, existing discretization methods still suffer from significant information loss, resulting in a notable performance gap compared to continuous representations. To overcome these limitations, we propose two quantization-based discretization methods: Product Quantization (PQ) and Random Product Quantization (RPQ). PQ partitions the original feature space into multiple subspaces and independently quantizes each sub-vector, producing a fused set of discrete units that retain diverse information from different subspaces, thereby mitigating the loss associated with single-cluster quantization. RPQ further enhances representation diversity by randomly sampling a fixed proportion of feature dimensions multiple times to construct sub-vectors, thereby better capturing the variability in the data distribution. Theoretical analysis shows that RPQ reduces the correlation coefficient rho (where 0 <= rho <= 1) between sub-quantizers. Its quantization error is lower-bounded by the product of rho and epsilon-kms, where epsilon-kms denotes the quantization error of a single K-means quantizer. Experimental results on a combined dataset built from LibriSpeech and ML-SUPERB show that PQ and RPQ outperform standard K-means discretization, achieving relative improvements of 21.8 percent and 20.0 percent in WER on LibriSpeech, and 24.1 percent and 19.6 percent in CER on ML-SUPERB, respectively. Moreover, their performance is competitive with, and in some cases even surpasses, that of continuous SSL representations.
Tele-FLM Technical Report
25 Apr 2024
Nanyang Technological UniversityLarge language models (LLMs) have showcased profound capabilities in language understanding and generation, facilitating a wide array of applications. However, there is a notable paucity of detailed, open-sourced methodologies on efficiently scaling LLMs beyond 50 billion parameters with minimum trial-and-error cost and computational resources. In this report, we introduce Tele-FLM (aka FLM-2), a 52B open-sourced multilingual large language model that features a stable, efficient pre-training paradigm and enhanced factual judgment capabilities. Tele-FLM demonstrates superior multilingual language modeling abilities, measured by BPB on textual corpus. Besides, in both English and Chinese foundation model evaluation, it is comparable to strong open-sourced models that involve larger pre-training FLOPs, such as Llama2-70B and DeepSeek-67B. In addition to the model weights, we share the core designs, engineering practices, and training details, which we expect to benefit both the academic and industrial communities.
Context-aware Event Forecasting via Graph Disentanglement
Event forecasting has been a demanding and challenging task throughout the
entire human history. It plays a pivotal role in crisis alarming and disaster
prevention in various aspects of the whole society. The task of event
forecasting aims to model the relational and temporal patterns based on
historical events and makes forecasting to what will happen in the future. Most
existing studies on event forecasting formulate it as a problem of link
prediction on temporal event graphs. However, such pure structured formulation
suffers from two main limitations: 1) most events fall into general and
high-level types in the event ontology, and therefore they tend to be
coarse-grained and offers little utility which inevitably harms the forecasting
accuracy; and 2) the events defined by a fixed ontology are unable to retain
the out-of-ontology contextual information. To address these limitations, we
propose a novel task of context-aware event forecasting which incorporates
auxiliary contextual information. First, the categorical context provides
supplementary fine-grained information to the coarse-grained events. Second and
more importantly, the context provides additional information towards specific
situation and condition, which is crucial or even determinant to what will
happen next. However, it is challenging to properly integrate context into the
event forecasting framework, considering the complex patterns in the
multi-context scenario. Towards this end, we design a novel framework named
Separation and Collaboration Graph Disentanglement (short as SeCoGD) for
context-aware event forecasting. Since there is no available dataset for this
novel task, we construct three large-scale datasets based on GDELT.
Experimental results demonstrate that our model outperforms a list of SOTA
methods.
Enhancing Zero-Shot Brain Tumor Subtype Classification via Fine-Grained Patch-Text Alignment
Fudan UniversityThe fine-grained classification of brain tumor subtypes from histopathological whole slide images is highly challenging due to subtle morphological variations and the scarcity of annotated data. Although vision-language models have enabled promising zero-shot classification, their ability to capture fine-grained pathological features remains limited, resulting in suboptimal subtype discrimination. To address these challenges, we propose the Fine-Grained Patch Alignment Network (FG-PAN), a novel zero-shot framework tailored for digital pathology. FG-PAN consists of two key modules: (1) a local feature refinement module that enhances patch-level visual features by modeling spatial relationships among representative patches, and (2) a fine-grained text description generation module that leverages large language models to produce pathology-aware, class-specific semantic prototypes. By aligning refined visual features with LLM-generated fine-grained descriptions, FG-PAN effectively increases class separability in both visual and semantic spaces. Extensive experiments on multiple public pathology datasets, including EBRAINS and TCGA, demonstrate that FG-PAN achieves state-of-the-art performance and robust generalization in zero-shot brain tumor subtype classification.
From General Relation Patterns to Task-Specific Decision-Making in Continual Multi-Agent Coordination
Beijing Jiaotong UniversityContinual Multi-Agent Reinforcement Learning (Co-MARL) requires agents to address catastrophic forgetting issues while learning new coordination policies with the dynamics team. In this paper, we delve into the core of Co-MARL, namely Relation Patterns, which refer to agents' general understanding of interactions. In addition to generality, relation patterns exhibit task-specificity when mapped to different action spaces. To this end, we propose a novel method called General Relation Patterns-Guided Task-Specific Decision-Maker (RPG). In RPG, agents extract relation patterns from dynamic observation spaces using a relation capturer. These task-agnostic relation patterns are then mapped to different action spaces via a task-specific decision-maker generated by a conditional hypernetwork. To combat forgetting, we further introduce regularization items on both the relation capturer and the conditional hypernetwork. Results on SMAC and LBF demonstrate that RPG effectively prevents catastrophic forgetting when learning new tasks and achieves zero-shot generalization to unseen tasks.
There are no more papers matching your filters at the moment.