Ask or search anything...
Aalborg UniversityLarge Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities
22 Sep 2025
A comprehensive survey proposes a novel two-dimensional taxonomy to organize research on integrating Large Language Models and Knowledge Graphs for Question Answering, detailing methods by complex QA categories and the specific functions KGs serve. The work reviews state-of-the-art techniques, highlights their utility in mitigating LLM weaknesses such as factual inaccuracy and limited reasoning, and identifies critical future research directions.
View blog
Determinantal point process models and statistical inference : Extended version
Statistical models and methods for determinantal point processes (DPPs) seem
largely unexplored. We demonstrate that DPPs provide useful models for the
description of spatial point pattern datasets where nearby points repel each
other. Such data are usually modelled by Gibbs point processes, where the
likelihood and moment expressions are intractable and simulations are time
consuming. We exploit the appealing probabilistic properties of DPPs to develop
parametric models, where the likelihood and moment expressions can be easily
evaluated and realizations can be quickly simulated. We discuss how statistical
inference is conducted using the likelihood or moment properties of DPP models,
and we provide freely available software for simulation and statistical
inference.
TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment
Peking University
Nanyang Technological UniversityTimeCMA introduces a framework for multivariate time series forecasting that leverages large language models (LLMs) through a novel cross-modality alignment module to generate disentangled yet robust time series embeddings. This approach, combined with efficient last token embedding storage, consistently outperforms state-of-the-art baselines across eight datasets while significantly reducing computational overhead.
View blog
TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods
An automated benchmark framework, TFB, addresses critical shortcomings in time series forecasting evaluation by offering comprehensive data coverage, fair comparisons across diverse method paradigms, and consistent experimental pipelines. Its extensive evaluation challenges existing assumptions about method efficacy and provides guidance for selecting appropriate models based on time series characteristics.
View blog
Foundation Models for Video Understanding: A Survey
University of Central FloridaA comprehensive survey of over 200 Video Foundation Models (ViFMs) is presented, categorizing them by architecture and analyzing various training strategies and their performance across tasks. The survey reveals that image-based models show competitive performance, multimodal approaches consistently outperform unimodal ones on complex tasks, and increased model scale generally yields superior results.
View blog
Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting
15 Sep 2024
Alibaba GroupTransformers for time series forecasting mainly model time series from limited or fixed scales, making it challenging to capture different characteristics spanning various scales. We propose Pathformer, a multi-scale Transformer with adaptive pathways. It integrates both temporal resolution and temporal distance for multi-scale modeling. Multi-scale division divides the time series into different temporal resolutions using patches of various sizes. Based on the division of each scale, dual attention is performed over these patches to capture global correlations and local details as temporal dependencies. We further enrich the multi-scale Transformer with adaptive pathways, which adaptively adjust the multi-scale modeling process based on the varying temporal dynamics of the input, improving the accuracy and generalization of Pathformer. Extensive experiments on eleven real-world datasets demonstrate that Pathformer not only achieves state-of-the-art performance by surpassing all current models but also exhibits stronger generalization abilities under various transfer scenarios. The code is made available at this https URL.
DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting
DUET is a framework for multivariate time series forecasting that enhances accuracy by employing dual clustering on both temporal and channel dimensions. It achieves state-of-the-art performance, demonstrating an average reduction of 7.1% in Mean Squared Error (MSE) and 6.5% in Mean Absolute Error (MAE) compared to the next best baseline.
View blog
Autoformalization in the Era of Large Language Models: A Survey
03 Jul 2025
Northeastern UniversityAutoformalization, the process of transforming informal mathematical propositions into verifiable formal representations, is a foundational task in automated theorem proving, offering a new perspective on the use of mathematics in both theoretical and applied domains. Driven by the rapid progress in artificial intelligence, particularly large language models (LLMs), this field has witnessed substantial growth, bringing both new opportunities and unique challenges. In this survey, we provide a comprehensive overview of recent advances in autoformalization from both mathematical and LLM-centric perspectives. We examine how autoformalization is applied across various mathematical domains and levels of difficulty, and analyze the end-to-end workflow from data preprocessing to model design and evaluation. We further explore the emerging role of autoformalization in enhancing the verifiability of LLM-generated outputs, highlighting its potential to improve both the trustworthiness and reasoning capabilities of LLMs. Finally, we summarize key open-source models and datasets supporting current research, and discuss open challenges and promising future directions for the field.
Closed-form Continuous-time Neural Models
02 Mar 2022
MITContinuous-time neural processes are performant sequential decision-makers
that are built by differential equations (DE). However, their expressive power
when they are deployed on computers is bottlenecked by numerical DE solvers.
This limitation has significantly slowed down the scaling and understanding of
numerous natural physical phenomena such as the dynamics of nervous systems.
Ideally, we would circumvent this bottleneck by solving the given dynamical
system in closed form. This is known to be intractable in general. Here, we
show it is possible to closely approximate the interaction between neurons and
synapses -- the building blocks of natural and artificial neural networks --
constructed by liquid time-constant networks (LTCs) efficiently in closed-form.
To this end, we compute a tightly-bounded approximation of the solution of an
integral appearing in LTCs' dynamics, that has had no known closed-form
solution so far. This closed-form solution substantially impacts the design of
continuous-time and continuous-depth neural models; for instance, since time
appears explicitly in closed-form, the formulation relaxes the need for complex
numerical solvers. Consequently, we obtain models that are between one and five
orders of magnitude faster in training and inference compared to differential
equation-based counterparts. More importantly, in contrast to ODE-based
continuous networks, closed-form networks can scale remarkably well compared to
other deep learning instances. Lastly, as these models are derived from liquid
networks, they show remarkable performance in time series modeling, compared to
advanced recurrent models.
An overview of neural architectures for self-supervised audio representation learning from masked spectrograms
In recent years, self-supervised learning has amassed significant interest for training deep neural representations without labeled data. One such self-supervised learning approach is masked spectrogram modeling, where the objective is to learn semantically rich contextual representations by predicting removed or hidden portions of the input audio spectrogram. With the Transformer neural architecture at its core, masked spectrogram modeling has emerged as the prominent approach for learning general purpose audio representations, a.k.a. audio foundation models. Meanwhile, addressing the issues of the Transformer architecture, in particular the underlying Scaled Dot-product Attention operation, which scales quadratically with input sequence length, has led to renewed interest in recurrent sequence modeling approaches. Among them, Selective structured state space models (such as Mamba) and extended Long Short-Term Memory (xLSTM) are the two most promising approaches which have experienced widespread adoption. While the body of work on these two topics continues to grow, there is currently a lack of an adequate overview encompassing the intersection of these topics. In this paper, we present a comprehensive overview of the aforementioned research domains, covering masked spectrogram modeling and the previously mentioned neural sequence modeling architectures, Mamba and xLSTM. Further, we compare Transformers, Mamba and xLSTM based masked spectrogram models in a unified, reproducible framework on ten diverse downstream audio classification tasks, which will help interested readers to make informed decisions regarding suitability of the evaluated approaches to adjacent applications.
TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting
12 Jun 2025
Time Series Forecasting (TSF) is key functionality in numerous fields, such
as financial investment, weather services, and energy management. Although
increasingly capable TSF methods occur, many of them require domain-specific
data collection and model training and do not generalize well when applied in
other domains. Time Series Foundation Models (TSFMs) that are pre-trained on
massive heterogeneous time series data aim to overcome these limitations. The
prospects for generalizability have spurred the development of a new generation
of TSFMs. This study proposes a benchmark, TSFM-Bench, to facilitate
comprehensive and unified evaluation of TSFMs. TSFM-Bench covers a wide range
of TSFMs, including those based on large language models and those pre-trained
on time series data. TSFM-Bench supports multiple forecasting scenarios,
including zero-shot, few-shot, and full-shot, enabling assessment across the
full range of adaptation strategies. TSFM-Bench also provides a standardized
experimental protocols for critical evaluation processes such as dataset
splitting, loading, normalization, and few-shot sampling, facilitating
consistency and fairness. We report on an extensive evaluation of TSFMs across
a diverse range of datasets spanning multiple domains and exhibiting varied
statistical characteristics. Specifically, we identify pros and cons and
inherent limitations of existing TSFMs, and we propose potential directions for
new model designs.
Large Language Models and Knowledge Graphs: Opportunities and Challenges
11 Aug 2023
University of Oxford
Zhejiang UniversityAn international team of researchers analyzes the convergence of Large Language Models (LLMs) and Knowledge Graphs (KGs), proposing a hybrid approach that leverages their complementary strengths to advance Knowledge Computing, aiming for more robust and accurate knowledge systems.
View blog
Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting
Time Series Forecasting (TSF) is a crucial task in various domains, yet existing TSF models rely heavily on high-quality data and insufficiently exploit all available data. This paper explores a novel self-supervised approach to re-label time series datasets by inherently constructing candidate datasets. During the optimization of a simple reconstruction network, intermediates are used as pseudo labels in a self-supervised paradigm, improving generalization for any predictor. We introduce the Self-Correction with Adaptive Mask (SCAM), which discards overfitted components and selectively replaces them with pseudo labels generated from reconstructions. Additionally, we incorporate Spectral Norm Regularization (SNR) to further suppress overfitting from a loss landscape perspective. Our experiments on eleven real-world datasets demonstrate that SCAM consistently improves the performance of various backbone models. This work offers a new perspective on constructing datasets and enhancing the generalization of TSF models through self-supervised learning. The code is available at this https URL.
AimTS: Augmented Series and Image Contrastive Learning for Time Series Classification
14 Apr 2025
AimTS provides a pre-training framework for time series classification, learning generalizable representations from multi-source data through prototype-based and series-image contrastive learning. The method achieved superior accuracy against state-of-the-art techniques across 125 UCR and 30 UEA datasets and demonstrated improved computational efficiency.
View blog
Logical Consistency of Large Language Models in Fact-checking
In recent years, large language models (LLMs) have demonstrated significant
success in performing varied natural language tasks such as language
translation, question-answering, summarizing, fact-checking, etc. Despite LLMs'
impressive ability to generate human-like texts, LLMs are infamous for their
inconsistent responses - a meaning-preserving change in the input query results
in an inconsistent response and attributes to vulnerabilities of LLMs such as
hallucination. Consequently, existing research focuses on simple
paraphrasing-based consistency assessment of LLMs, and ignores complex queries
that necessitate an even better understanding of logical reasoning by an LLM.
Our work therefore addresses the logical inconsistency of LLMs under complex
logical queries with primitive logical operators, e.g., negation, conjunction,
and disjunction. As a test bed, we consider retrieval-augmented LLMs on a
fact-checking task involving propositional logic queries from knowledge graphs
(KGs). Our contributions are threefold. Benchmark: We introduce three logical
fact-checking datasets over KGs for community development towards logically
consistent LLMs. Assessment: We propose consistency measures of LLMs on
propositional logic queries and demonstrate that existing LLMs lack logical
consistency, especially on complex queries. Improvement: We employ supervised
fine-tuning to improve the logical consistency of LLMs on the complex
fact-checking task with KG contexts. We have made our source code and
benchmarks available.
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
29 Apr 2025
Marzieh Fadaee
Shivalika Singh
Azmine Toushik Wasi
University of California, Santa Barbara
University of CopenhagenThe evaluation of vision-language models (VLMs) has mainly relied on
English-language benchmarks, leaving significant gaps in both multilingual and
multicultural coverage. While multilingual benchmarks have expanded, both in
size and languages, many rely on translations of English datasets, failing to
capture cultural nuances. In this work, we propose Kaleidoscope, as the most
comprehensive exam benchmark to date for the multilingual evaluation of
vision-language models. Kaleidoscope is a large-scale, in-language multimodal
benchmark designed to evaluate VLMs across diverse languages and visual inputs.
Kaleidoscope covers 18 languages and 14 different subjects, amounting to a
total of 20,911 multiple-choice questions. Built through an open science
collaboration with a diverse group of researchers worldwide, Kaleidoscope
ensures linguistic and cultural authenticity. We evaluate top-performing
multilingual vision-language models and find that they perform poorly on
low-resource languages and in complex multimodal scenarios. Our results
highlight the need for progress on culturally inclusive multimodal evaluation
frameworks.
Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching
15 Jan 2025
Traditional similarity-based schema matching methods are incapable of resolving semantic ambiguities and conflicts in domain-specific complex mapping scenarios due to missing commonsense and domain-specific knowledge. The hallucination problem of large language models (LLMs) also makes it challenging for LLM-based schema matching to address the above issues. Therefore, we propose a Knowledge Graph-based Retrieval-Augmented Generation model for Schema Matching, referred to as the KG-RAG4SM. In particular, KG-RAG4SM introduces novel vector-based, graph traversal-based, and query-based graph retrievals, as well as a hybrid approach and ranking schemes that identify the most relevant subgraphs from external large knowledge graphs (KGs). We showcase that KG-based retrieval-augmented LLMs are capable of generating more accurate results for complex matching cases without any re-training. Our experimental results show that KG-RAG4SM outperforms the LLM-based state-of-the-art (SOTA) methods (e.g., Jellyfish-8B) by 35.89% and 30.50% in terms of precision and F1 score on the MIMIC dataset, respectively; KG-RAG4SM with GPT-4o-mini outperforms the pre-trained language model (PLM)-based SOTA methods (e.g., SMAT) by 69.20% and 21.97% in terms of precision and F1 score on the Synthea dataset, respectively. The results also demonstrate that our approach is more efficient in end-to-end schema matching, and scales to retrieve from large KGs. Our case studies on the dataset from the real-world schema matching scenario exhibit that the hallucination problem of LLMs for schema matching is well mitigated by our solution.
Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
07 Sep 2025
With the growing availability of multi-domain time series data, there is an increasing demand for general forecasting models pre-trained on multi-source datasets to support diverse downstream prediction scenarios. Existing time series foundation models primarily focus on scaling up pre-training datasets and model sizes to enhance generalization performance. In this paper, we take a different approach by addressing two critical aspects of general forecasting models: (1) how to derive unified representations from heterogeneous multi-domain time series data, and (2) how to effectively capture domain-specific features to enable adaptive transfer across various downstream scenarios. To address the first aspect, we propose Decomposed Frequency Learning as the pre-training task, which leverages frequency-based masking and reconstruction to decompose coupled semantic information in time series, resulting in unified representations across domains. For the second aspect, we introduce the Time Series Register, which captures domain-specific representations during pre-training and enhances adaptive transferability to downstream tasks. Our model achieves the state-of-the-art forecasting performance on seven real-world benchmarks, demonstrating remarkable few-shot and zero-shot capabilities.
SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation
Carnegie Mellon UniversityInspired by how humans reason over discrete objects and their relationships, we explore whether compact object-centric and object-relation representations can form a foundation for multitask robotic manipulation. Most existing robotic multitask models rely on dense embeddings that entangle both object and background cues, raising concerns about both efficiency and interpretability. In contrast, we study object-relation-centric representations as a pathway to more structured, efficient, and explainable visuomotor control. Our contributions are two-fold. First, we introduce LIBERO+, a fine-grained benchmark dataset designed to enable and evaluate object-relation reasoning in robotic manipulation. Unlike prior datasets, LIBERO+ provides object-centric annotations that enrich demonstrations with box- and mask-level labels as well as instance-level temporal tracking, supporting compact and interpretable visuomotor representations. Second, we propose SlotVLA, a slot-attention-based framework that captures both objects and their relations for action decoding. It uses a slot-based visual tokenizer to maintain consistent temporal object representations, a relation-centric decoder to produce task-relevant embeddings, and an LLM-driven module that translates these embeddings into executable actions. Experiments on LIBERO+ demonstrate that object-centric slot and object-relation slot representations drastically reduce the number of required visual tokens, while providing competitive generalization. Together, LIBERO+ and SlotVLA provide a compact, interpretable, and effective foundation for advancing object-relation-centric robotic manipulation.
Efficient Multivariate Time Series Forecasting via Calibrated Language Models with Privileged Knowledge Distillation
TimeKD introduces an efficient framework for multivariate time series forecasting that leverages the robust representation capabilities of Large Language Models while mitigating their high inference costs. This is achieved through a novel privileged knowledge distillation method, which enables a lightweight student model to effectively learn from a calibrated LLM teacher that processes privileged information (future ground truth data) during training, leading to improved forecasting accuracy and superior computational efficiency.
View blog
There are no more papers matching your filters at the moment.